I don't know how I feel about this honestly. In theory whether you've patched a live site, are testing on a backup of a live site, or testing with a bare Joomla install shouldn't matter. The differences in those environments are too great to accurately catch the conditions someone tested in. My component configuration for com_content may be different than every other tester's config, which is good in that it's an additional possible combination of how things are set up but at the same time it means that because there are so many possible combinations of configurations that you would practically have to attach the relevant configurations for the item you're testing and any possible influencers to the test report too (Viewing an article? Does it have its own menu item or is it part of a category's menu item? What's the article configuration? What's the parent category configuration? What's the menu item configuration? etc, etc.).

Hi,
I forgot to give the context of the suggestion : it came from a discussion at the JoomlaDays Netherlands last week-end with Roland Dalmulder, Brian Teeman and others.

We were discussing a couple of ideas to make Testing easier/better.
There was first an idea about a "one-click installation with Sample Data & Patch Tester included"(same principle as the "Quick Joomla Test Drive" on https://demo.joomla.org)
But it appeared that this was indeed not such a good idea, a.o. because that would mean that everybody would use exactly the same environment.
So this is how we came to speak about the importance of testing on different environments, not only with Joomla Sample Test Data.
For example, the future Joomla Router could work perfectly well on an empty or basic Joomla installation, but not necessarily on real websites because of third-party.

Actually, in the Documentation, it is asked to test on Sample Test Data.
But testing on (Duplicates of) real websites is also very important because it allows to discover more potential side-effects of proposed Patches.

So maybe a first quick win would be to adapt Documentation (https://docs.joomla.org/Testing_Joomla!_patches) to say that testing

• can of course be done on Joomla Staging Version with Sample Test Data
• but should also be done on (duplicates of) real websites

And a second thing would be to incorporate some extra options on the Feedback screen, so that we can assess more easily the "robustness" of the (so-called) Successfull Tests.

But this is just a suggestion of course... and it can certainly be refined or improved upon.

It'd be helpful if more information about the test environment were shared, but a high level "tested on clone of website" or "tested on live website" option to me seems too wide open and not much more informative than saying "I've tested this item". How www.joomla.org and www.babdev.com are configured is quite different (different hosting platforms, active PHP versions, installed extensions, template, etc.) so me saying I tested on my live site or a clone of www.joomla.org without saying either I've tested on one of those sites or posting the configs of those sites makes little difference.

In general I'm +1 on making it possible to include more info about the test environment, but I'm 0 on this suggestion as is because I personally don't feel like those general statements are adding any value to the testing module.

Maybe we can add a default text like a template that asks for a few parts of the hosting envoirment as optional point.

######### extra data

I have [un]successfully tested this on using

• PHP Version:
• Database:
• Database- Version:
• Joomla build
• 3Patry Extensions:

*
*

Just my 2cts following @mbabker

"For example, the future Joomla Router could work perfectly well on an empty or basic Joomla installation, but not necessarily on real websites because of third-party."

Then it would be a third-party issue that should be reported to the devs of the extension and not to the core?

IMHO, the Project doesn't have to ensure a proper operation regarding all the possible configurations and settings, but only on those that meet the technical requirements.

Maybe we can add a default text like a template that asks for a few parts of the hosting envoirment as optional point.

Something like #760 ?

Sysinfo uploads should be working OK, so something suggesting attaching that to a test report may be helpful.

And bouncing back on the router thing specifically, just because it doesn't work with an extension does not necessarily mean that it's a core issue. Sure posting on the issue tracker can probably help figure that out, but ultimately Joomla can't patch every extension. Having all the relevant info when an issue is posted or a test is performed definitely helps things in the long run, and I think we're already supporting that to the extent practical without making it mandatory (just could possibly use some UI nudges recommending it).

Mmmh, maybe some misunderstanding, let me restate the idea quickly :

• the idea is not to make a test specifically to test compatibility of third-party extensions
• but the idea is that you can quicly miss problems if people stick to Sample Data or little site

Let me take an example : do you see the Item Counter (articles counter, ...) introduced with J!3.5 ?
Actually, it had been happily tested by many during months.
And only two weeks before 3.5, someone came and said "hey, I have just tested on my website which happens to have thousands & thousands of articles... and there is a huge performance issue, like 30 seconds to open the Category page".

So, not even speaking of interactions with other things, even sticking strictly to the Core, it does make a difference whether you only test on standard Sample Data or in "real life" :-)

But indeed, what is the most relevant information we could add to the Feedback screen, maybe it is not what I initially suggested, maybe like mentionned above it is more "PHP version, ...".
This I leave it to you, more experimented guys :-)

My point was just to say "hey, we could improve quality of testing, both for testers and for people having to approve the Patches, if the Feedback screen was a bit enriched"

My point was just to say "hey, we could improve quality of testing, both for testers and for people having to approve the Patches, if the Feedback screen was a bit enriched"

I'm all for that. Just need to figure out what to do I guess.