? ? Success

User tests: Successful: Unsuccessful:

avatar rdeutz
rdeutz
2 Oct 2016

Situation today

While we are in Alpha, Beta, RC development mode we don’t have a real overview how the packages are testet. We get some results when people submit issues but we don’t have an idea if people are testing in real live environments or if people are testing at all

One of the problems is that there is no place someone can report a successful test result. This PR should solve this problem.

Idea

The idea is to have a specific extension for reporting test results. The date will be sent to the statistics server and we don’t ask people for personal data (they can submit personal data, I will explain the idea behind this later)

It is as easy as possible, people ask simple question like „Is this a copy of a production site or a vanilla test site“, „Does the site run in a production like environment“, „Did you update or is it a clean install“ and so on. Just some easy questions. The next step is to report if someone tested a part of the CMS Yes with or without success or no.

The testing result allow us to have a better understanding, what people are testing and where the package has problems. This will give us the chance to work on specific areas and improve the quality.

The extension will only be enabled when we are in development mode (alpha, beta, rc) it doesn’t makes sense to have the extension enabled for a stable version of the CMS.

I added some fields for personal data but these are not required, but if people give us there name we can list them as „PreReleaseTesters“ or if they give us the github account we can add points for testing so that people get some „fame“ back.

How to test

  • Apply patch
  • Make a discover install
  • play around

If you hit send then you will see what would be send to the server. The endpoint isn’t developed at the moment, I though I will look first if people like the idea.

avatar rdeutz rdeutz - open - 2 Oct 2016
avatar rdeutz rdeutz - change - 2 Oct 2016
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 2 Oct 2016
Category Administration Components Language & Strings
avatar joomla-cms-bot joomla-cms-bot - change - 2 Oct 2016
Labels Added: ? ?
avatar mbabker
mbabker - comment - 2 Oct 2016

Does this need to be a part of core? What are we gaining having an API endpoint where sites can send a "this works" or "this does not work" (basically a boolean flag) and a duplicate of the stats server data object without additional details about what the user's doing?

I honestly don't see much value in this as is. It's too generic and doesn't give any way for a user to explain what they are testing to validate a successful change or a way to report issues if they feel they've reached a fail state.

avatar rdeutz
rdeutz - comment - 2 Oct 2016

It might be generic at this state but it gives us information we don't have now. And we can make it more specific and add checklists for extensions to guide people to a process of testing. It is also a bit more then a boolean flag, you can say for each extension if you have tested it or not.

joomla test releases - staging dev 3 7 x - administration

avatar ghazal ghazal - test_item - 4 Oct 2016 - Tested successfully
avatar ghazal
ghazal - comment - 4 Oct 2016

I have tested this item successfully on 413ba22

Tested on joomla 3.7

Overall it's a nice addition. It works allright.
But, as I tested on PHP7, I got these notices:
Notice : Undefined variable: disabled in layouts/joomla/form/field/radio.php on line 58

Notice : Undefined variable: required in layouts/joomla/form/field/radio.php on line 59

Notice : Undefined variable: autofocus in layouts/joomla/form/field/radio.php on line 60

Notice : Undefined variable: disabled in layouts/joomla/form/field/radio.php on line 68

Notice : Undefined variable: required in layouts/joomla/form/field/radio.php on line 78


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258.

avatar rdeutz
rdeutz - comment - 4 Oct 2016

Thanks for testing, the notice came from not proper testing if a var is set before using it. I am going to fix it later.

avatar jeckodevelopment
jeckodevelopment - comment - 8 Oct 2016

Good idea @rdeutz ! ?
Why don't we ship it as official extension? And promote it when we release RCs and Beta versions, instead of having it in the core?

avatar rdeutz
rdeutz - comment - 8 Oct 2016

I have my doubts that people will go find the extensions install it and then submit the results, if we don't ship it with core this all is a waste of time. Furthermore we only need to ship it while we are in alpha/beta/rc it don't has to be in a stable release. And even it is in a stable release, it can be disabled.

avatar brianteeman
brianteeman - comment - 8 Oct 2016

Wouldnt this work the same as com_patchtester. People who want to contribute to testing will install the component and use it


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258.

avatar brianteeman
brianteeman - comment - 8 Oct 2016

Furthermore we only need to ship it while we are in alpha/beta/rc it don't has to be in a stable release

We could use the same logic for including com_patchtester in those releases - which seems a good idea to me


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258.

avatar dgt41
dgt41 - comment - 8 Oct 2016

@brianteeman I guess both this and com_patch_tester could somehow live in this branch for beta/rc. The only thing will be that people will have to do a discover installation, so the whole thing would remain simple (from db perspective)

avatar brianteeman
brianteeman - comment - 8 Oct 2016

@dgt41 yes

avatar brianteeman
brianteeman - comment - 8 Oct 2016

@rdeutz The libraries are listed in the unenabled extensions


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258.

avatar mbabker
mbabker - comment - 8 Oct 2016

We've kept the repo structure in a state that it only includes production assets (to the point that the Composer development dependencies like PHPUnit aren't included which makes maintaining that aspect of things more complex) with minimal extras (those extras basically being consolidated to the build and tests directories). I'm not convinced we should be littering the repo full of development only tools like this or patch tester, especially if don't even come installed by default (which makes packaging stable releases even more complex because you'd have to start stripping lines of code out of certain files). If it's in this repo it should be part of all packages and set up properly.

avatar csthomas
csthomas - comment - 8 Oct 2016

What about only add link(-s) on backend (somewhere) and by one click user can install developer/tester component.

Something like: Quick install one of the Joomla tester extensions.

For example links can be visible only if error reporting is set to maximum or if joomla version contains some dev chars.

avatar mbabker
mbabker - comment - 8 Oct 2016

Just use (new JVersion)->isInDevelopmentState(), don't try to parse the version string or rely on error reporting.

avatar coolcat-creations
coolcat-creations - comment - 28 Oct 2016

I like the idea!
Just some suggestions:

i could have google it to put the shame away from me but:
What is Vanilla Test? -> maybe a tooltip for me ;)

Site updated from -> First one the language string is missing.

Layout is broken at the bottom (Chrome)
image

and following error on certain points:
Notice: Undefined property: stdClass::$tested in /Applications/MAMP/htdocs/joomla-cms/administrator/components/com_jtestreport/views/default/tmpl/send.php on line 110

Is the list a list of the changes in the system? Would be perfect then to link to the specific Area (but i guess it´s too much effort to include this links)

Later on on this page should be maybe an explanation what is happening with the data etc.

avatar brianteeman
brianteeman - comment - 28 Oct 2016

Vanilla test is like vanilla ice cream. Clean and pure with nothing added

avatar coolcat-creations
coolcat-creations - comment - 28 Oct 2016

Let´s add this into the tooltip :-)

avatar brianteeman
brianteeman - comment - 28 Oct 2016

Still not sure myself that this is something to add to the CMS myself

On 28 October 2016 at 19:21, coolcat-creations notifications@github.com
wrote:

Let´s add this into the tooltip :-)


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#12258 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABPH8W1BU_hBABQERDet6IqQk2YS4A5_ks5q4j0RgaJpZM4KMDc5
.

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
https://brian.teeman.net/ http://brian.teeman.net/

avatar brianteeman brianteeman - change - 29 Oct 2016
Labels Added: ?
Removed: ?
avatar brianteeman brianteeman - change - 29 Oct 2016
Labels Added: ?
Removed: ?
avatar brianteeman brianteeman - change - 5 Dec 2016
Status Pending Needs Review
Labels
avatar brianteeman
brianteeman - comment - 5 Dec 2016

Setting to needs review. A decision needs to be made if this is something we want to have merged into the core - even if only distributed in staging and not in final releases


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258.

avatar brianteeman brianteeman - change - 5 Dec 2016
Labels Removed: ? ?
avatar brianteeman brianteeman - edited - 5 Dec 2016
avatar franz-wohlkoenig
franz-wohlkoenig - comment - 11 Jan 2017

I have tested this item successfully on 413ba22

Missing String:
bildschirmfoto 2017-01-11 um 19 26 06


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12258.
avatar franz-wohlkoenig franz-wohlkoenig - test_item - 11 Jan 2017 - Tested successfully
avatar rdeutz rdeutz - change - 12 May 2017
Status Needs Review Closed
Closed_Date 0000-00-00 00:00:00 2017-05-12 12:12:51
Closed_By rdeutz
avatar rdeutz rdeutz - close - 12 May 2017

Add a Comment

Login with GitHub to post a comment