Success

User tests: Successful: Unsuccessful:

avatar elkuku
elkuku
23 May 2013

Here is some new stuff I hacked together during the last week. Open for review ;)

Access control

Acces control is limited and very specific:

  • Only Projects are tracked.
  • There are only 4 "hard wired" Actions:
    view, create, edit and manage.
  • It is based on groups a user automatically belongs to or can be assigned to.

    For every project two system groups are created by default:
    Public and User.

The special admin user role that is assigned using the config.json file is granted global access.

Following is an example setup for a security tracker with non public access and two additionally created custom groups:
Access groups

However, this is a very first step... lots of optimization and testing required here.

Pagination

I searched the web for "PHP pagination MySQL" or similar, and this came up on the first page:
http://www.awcore.com/dev/1/3/Create-Awesome-PHPMYSQL-Pagination_en

I liked the way the author solved "the problem", the code looked acceptable,... so I "forked" and Joomla!"d it :)
The CSS was somewhat conflicting (for me), so I took one of the beautiful styles from the author, and now it looks like this:

Pagination

For now it only generates "plain links" adding a &page=n parameter to the current URL.
When we implement the search functionality, this must be revised.
I can also imagine a JavaScript (AJAX) based solution using this.

Note: The license on this is very unclear. Seems that is provided as a "snippet". Maybe our legal department should have a look at this ;)

Database updates

Obsolete tables and table fields have been removed while others have been added ;)

Full reinstall required !

Avatar support

On the first log in, the application tries to fetch the users avatar and store it locally.

This requires cURL.

Debugging and Logging

Added a TrackerDebugger class that... handles debugging :P
Added a CallbackLogger class to provide database query logging.
A log file can be written to log queries even during redirects ;)
Exception rendering has been improved to include clickable file links using the xdebug protocol.


Happy testing
=;)

avatar elkuku elkuku - open - 23 May 2013
avatar elkuku elkuku - open - 23 May 2013
avatar elkuku
elkuku - comment - 23 May 2013

The Travis build failed with the following error: Connection to 'pear.phpmd.org:80' failed: Connection timed out...
The other day a build failed by exceeding the GitHub rate limit :D

avatar b2z
b2z - comment - 24 May 2013

What a great additions we have here! Huge job done, thanks Nikolai! I am now looking into the code and will write my comments on it :)

avatar b2z
b2z - comment - 24 May 2013

Ok, seems that everything looks fine :)

Some thoughts:

  • create /logs/database.log before trying to put the contents. Without it I had a fatal error.
  • I don't like iterating avatar(activity.user), because this will make a load on a database. I think we need to get the path of avatar joining the users table. To do that we need user_id in activities table not only username.
avatar elkuku
elkuku - comment - 24 May 2013

Nice ;)
I fixed 1) yesterday (including display of other log files etc.) and working on 2) right now - the avatar thingy is much more simpler than I thought, because all avatars are stored as png - I missed that somehow.. so no more iterating..

I'll push that later today - Thanks a bunch for testing :smile:

avatar b2z
b2z - comment - 24 May 2013

Great! BTW I like the Pagination - nice one ;) And we could try to AJAXify it. I did something similar with the default JPagination in my eSports extension. Here for example.

avatar elkuku
elkuku - comment - 24 May 2013

OK, here is some more new stuff.
Note that a reinstall and config modifications are required.

If you find more bugs / have more of those good suggestions - feel free ;)

Oh, BTW, @mbabker I did some changes to the hooks scripts due to db field renaming and, since I have not been able to test them properly, they might throw some sweet errors... Please don't hate me too much :tongue:

avatar b2z
b2z - comment - 25 May 2013

Ok some my fixes are pushed ;)

BTW in the TrackerDebugger there are a couple of config vars like debug.logging, but these does not present in the config file.

avatar b2z
b2z - comment - 25 May 2013

If anyone wants to see our current work in action check out my live JTracker example site. Currently there are only JTracker bugs imported :)

avatar elkuku
elkuku - comment - 25 May 2013

Thanks for providing the online demo - I hope we will not run in any security issues or hurt your server ;)

The Travis fails are valid (surprise..) - fixing...

avatar mbabker
mbabker - comment - 25 May 2013

I (finally) got around to checking this out. Looking good.

I only get to play with the hooks after the changes go live. I'm hoping that one of the loggers somewhere catch whatever errors may occur, that's the only way I know they're broken.

Someone just say when to merge this (or click the button) so I know to start loading up the j.org server with the latest changes. I really should look into having Jenkins do this syncing for me...

avatar elkuku
elkuku - comment - 25 May 2013

@mbabker wrote

I'm hoping that one of the loggers somewhere catch whatever errors may occur...

I'm not quite sure if we should use J\Log or bring in a PSR logger (like monolog or something) for that matter ?
Oh, btw.., I just found whoops (thanks to @dongilbert for that) - I think we might bring this in and use it "opt in" - concerns ? (composer is a dangerous toy...)

Someone just say when to merge this (or click the button) so I know to start loading up the j.org server with the latest changes.

I must say I really enjoy the discussions here on GitHub about new features and code (thanks @b2z). We just should try to keep the issue subjects smaller :tongue:

I really should look into having Jenkins do this syncing for me...

Could we start playing with Jenkins please (maybe just by creating separate build.xml files) ? I also find it more reliable than Travis... maybe we can push the Jenkins results to GitHub using our App, like Travis does now ;)

avatar mbabker
mbabker - comment - 25 May 2013

I have access on build.joomla.org now to manage jobs, so I can look at tying in Jenkins with this repo (maybe use it as a POC for pull testing on the CMS too).

I like Travis, but it just isn't as flexible as Jenkins. Nothing wrong with that, both serve their purposes well.

avatar mbabker
mbabker - comment - 25 May 2013

As for the logger, the Framework Log package needs an overhaul to get in compliance with PSR-3. So if we use that, we're in a "be forewarned, it might change" state (not like we haven't already gone down that path by using a framework that's deemed to be in an alpha state). AbstractApplication is already linked towards the PSR interface so we'd have to override the **Logger methods to not use a PSR logger. All that said, I think we're probably better off going with a PSR compatible package. @realityking had mentioned to me in the past that we should use Monolog, so there's your outside opinion as well.

avatar AmyStephen
AmyStephen - comment - 27 May 2013

WHAT IN HEAVENS NAME IS GOING ON HERE???? SHOULDN'T WE TALK ABOUT THIS IN A LONG THREAD ON LIST BEFORE YOU CODE ANYTHING?

@b2z thanks for the link, too http://tracker.j-esports.com/tracker/jtracker-bugs

Seriously impressed.

avatar elkuku
elkuku - comment - 27 May 2013

I am seriously impressed of the long threads on the list(s) about what Joomla! might become... I WANT IT NOW !

:smile:

(((Yes, of course, I'm aware that this will be a "mission critical" app - but as sooner we'll get something to test, we'll get something the JBS might work with.)))

avatar mbabker
mbabker - comment - 27 May 2013

@AmyStephen there were a few threads on various lists spanning well over a year of discussion. Therefore, I believe the project's prerequisite of "do a crapton of talking yielding little to no results then accomplishing something productive" has been met. Unfortunately, we still have the "doing another crapton of talking for no apparent reason before finally agreeing that the effort that has been made on the presented proposal is good enough for us" phase to go through. :laughing:

avatar AmyStephen
AmyStephen - comment - 27 May 2013

I like your style. Learning. This is an amazing team.

On Mon, May 27, 2013 at 3:47 PM, Michael Babker notifications@github.comwrote:

@AmyStephen https://github.com/AmyStephen there were a few threads on
various lists spanning well over a year of discussion. Therefore, I believe
the project's prerequisite of "do a crapton of talking yielding little to
no results then accomplishing something productive" has been met.
Unfortunately, we still have the "doing another crapton of talking for no
apparent reason before finally agreeing that the effort that has been made
on the presented proposal is good enough for us" phase to go through. [image:
:laughing:]


Reply to this email directly or view it on GitHub#111 (comment)
.

avatar b2z
b2z - comment - 29 May 2013

So, will we merge it or we are still in the process of making a decision regarding Logger to bring in?

avatar elkuku
elkuku - comment - 30 May 2013

Let me push some latest changes (hope they won't break anything badly)... and it should be good ;)

avatar elkuku
elkuku - comment - 30 May 2013

Here you go ;)
Note I added nostalgiaz / bootstrap-switch which is Apache licensed, and should be consulted with our legal department before merging ;)

avatar mbabker
mbabker - comment - 30 May 2013

Bootstrap 2 is Apache licensed and we were clear to include and distribute it in the CMS. I can find the discussion, but it all boiled down to it not being PHP code (and the PHP code to support its use is code written for and copyrighted by OSM) IIRC. So as long as we're just using it and not including its pure source in our PHP code, then there shouldn't be any issue.

avatar elkuku
elkuku - comment - 30 May 2013

Ah right (I remember @AmyStephen saying that lately).
BTW this is only used to toggle the states in the access groups edit view. There might be a much more simpler solution here (maybe just the check boxes)..

avatar mbabker
mbabker - comment - 30 May 2013

If we wanted to update to Bootstrap 2.3.2, IIRC, they have added native support for different button states, but not quite the same as the switch. The LESS files for the template are in the master branch, so theoretically someone could do the update if they wanted (we've been using Bootstrap 2.1 since that's what the CMS is (still) shipping and that's what the template was built from).

avatar elkuku
elkuku - comment - 30 May 2013

I believe the big difference is that the switch thingy uses a standard <input type="checkbox"> with an additional <div> (and optionally its associated <label> so it also supports keyboard usage) while the bootstrap toggle uses a <button> thingy...

I would like to care more about PHP and leave CSS and JavaScript stuff to someone more experienced ;)

avatar b2z
b2z - comment - 31 May 2013

Synced my deployment and everything looks fine :+1: Only Stats gives a blank page, need to investigate it later.

avatar elkuku
elkuku - comment - 31 May 2013

The blank page is a fatal :P... The stats thingy is only a small test. We need to investigate how to run those requests in the background (a cron maybe) and store the results to db. @todo in another PR - not a blocker imo ;)

avatar dongilbert
dongilbert - comment - 31 May 2013

Searching / filtering ends up with an UnexpectedValueException

avatar elkuku
elkuku - comment - 31 May 2013

Searching isn't even supposed to work yet, but gives interesting new errors - thanks for reporting :)

Umm... seems that I'll do another (small) round of bug killings...

avatar wilsonge
wilsonge - comment - 4 Jun 2013

CMS is now using Bootstrap 2.3.2 as of today so would now make sense to update your guys bootstrap version especially if it allows you to drop the switch (although I know @mbabker said it wasn't a direct replacement)

avatar mbabker
mbabker - comment - 4 Jun 2013

I've left a note also with those with a direct interest in keeping the j.org template updated about this change (in case they don't watch all the GitHub activity). So, we should be able to just pull in that media when the template is updated and adapt to the Twig output.

avatar - close - 9 Jun 2013
avatar elkuku elkuku - reference | f5fcab5 - 9 Jun 13
avatar elkuku elkuku - merge - 9 Jun 2013
avatar elkuku elkuku - close - 9 Jun 2013
avatar elkuku
elkuku - comment - 9 Jun 2013

Seems we are good here, so I'm merging this to find more issues on the live server ;)

avatar elkuku elkuku - head_ref_deleted - 9 Jun 2013
avatar mbabker
mbabker - comment - 9 Jun 2013

And on that note, I'll be busy re-syncing that :-)

avatar elkuku
elkuku - comment - 9 Jun 2013

Please note that I also merged the hooks branch which could be interesting on the live server ;)

avatar mbabker
mbabker - comment - 9 Jun 2013

Gah, I just finished syncing with just the some-new-stuff branch merged. Well, here we go again... LOL

avatar elkuku
elkuku - comment - 9 Jun 2013

Ups... seems that we need more communication ;)

avatar mbabker
mbabker - comment - 9 Jun 2013

M'eh, it happens.

avatar mbabker
mbabker - comment - 9 Jun 2013

And we're live. You've got admin rights, don't kill my server :-P

avatar dongilbert
dongilbert - comment - 9 Jun 2013

Where is this server?

On Sun, Jun 9, 2013 at 3:19 PM, Michael Babker notifications@github.comwrote:

And we're live. You've got admin rights, don't kill my server :-P


Reply to this email directly or view it on GitHub#111 (comment)
.

avatar mbabker
mbabker - comment - 9 Jun 2013

issues dot j.org (not intended for mass use yet, hence the reason it isn't a real link)

avatar mbabker
mbabker - comment - 9 Jun 2013

@elkuku The comments hook is broken, none of these comments are showing up. And for once it ain't my fault :-P

avatar mbabker
mbabker - comment - 9 Jun 2013

Just dug in some, and the GitHub bot is getting a 404 whenever it's requesting /hooks/comments. I tried it myself, got the expected white screen, and the appropriate log entry filed.

avatar mbabker
mbabker - comment - 9 Jun 2013

Added a new RequestBin. Looking at the raw access logs, it seems as though the last few requests are larger than requests from earlier that did work.

192.30.252.54 - - [09/Jun/2013:15:13:11 -0500] "POST /hooks/comments HTTP/1.1" 200 162 "-" "GitHub Hookshot 107b5cb"
192.30.252.58 - - [09/Jun/2013:15:45:01 -0500] "POST /hooks/comments HTTP/1.1" 404 12671 "-" "GitHub Hookshot 107b5cb"

avatar elkuku
elkuku - comment - 9 Jun 2013

Bad stuff :(
Try to set debug.hooks to 1

Seems that the log has been started but no entry has been added... @todo enhance logging even more :P

avatar mbabker
mbabker - comment - 9 Jun 2013

Hooks debugging enabled. Let's see what happens this time.

avatar mbabker
mbabker - comment - 9 Jun 2013

Still got a 404. I also enabled debug.logging and got an entry in 404.log. Bad route.

avatar mbabker
mbabker - comment - 9 Jun 2013

And turned on the full system debug now. Let's see what I get ;-)

avatar mbabker
mbabker - comment - 9 Jun 2013

Found the problem. It's choking over my use of an @ in my bot account password.

avatar elkuku
elkuku - comment - 9 Jun 2013

Huh - sounds ugly... I hope it's an easy fix ;)

avatar jissues-bot
jissues-bot - comment - 9 Jun 2013

It was ;-)

avatar elkuku
elkuku - comment - 9 Jun 2013

Nice - but seems that some data got scrambled :tongue: maybe we should leave debug.hooks on for a while to get all the nice JSON data for local debugging ;)

avatar mbabker
mbabker - comment - 9 Jun 2013

Consider it done

avatar elkuku
elkuku - comment - 9 Jun 2013

Heh - that looks sweeet :smile:

Add a Comment

Login with GitHub to post a comment