By the way, there's more in this, like: incremental commits per PR, clean up of the S3 maybe some caching issues, etc but I guess the idea is implementable without much pain

If the PR testing platform ever launches, that's all that is needed. com_patchtester is still suitable for purely PHP changes but it will never work efficiently on a tagged release package, and what you're proposing is essentially running the CMS' build script for each commit of a pull request.

Personally I'd rather re-evaluate this app as a whole and what benefits it is bringing the project and not start adding major features to it at the moment.

@mbabker where is that PR?

Also by app you mean the CMS or patchtester?

https://github.com/joomla-projects/gsoc17_pr_testing_platform

This app === issue tracker

@mbabker who's in charge for the testing platform?

I honestly don't even know anymore.

Personally I'd rather re-evaluate this app as a whole and what benefits it is bringing the project and not start adding major features to it at the moment.

By the way this doesn't need to be related to the issue tracker if the Drone can use some Webhook to write back to Github the link for the installable zip. Also, we really don't need to run the build script just zip the contents without the node_modules folders, the easy way

Actually digging more into this it seems like a very easy implementation:
There is already a plugin for the aws:
http://plugins.drone.io/drone-plugins/drone-s3/

So all we need is a simple zip command either in bash or in js that will run after npm install

If we are going to put a ZIP somewhere it needs to be generated by the build script so that an end user can either clone a new install from it (which is what the PR testing platform would do) or upgrade a testing site with it, it should not be something with a bunch of random files. The entire point of the build script is to ensure composer install --no-dev and npm install is run on the CMS, that node_modules is cleaned, and a boatload of junk files cleaned out of the Composer installation, otherwise we could just run git archive and call it a day.

With 4.0 a "only changed files" package isn't an option either given none of media or libraries/vendor is version controlled anymore, and THAT is where the struggle with testing lies.

@mbabker I think that if we agree on the approach of delivering an installable zip out of Drone the other parts are implementation details (eg run build.php or another script). Also since there is no progress on the testing platform for many months I could say that this is a more down to earth approach that can be reached without major pain

Drone is outside my area of responsibility, and if an integration isn't being added to the issue tracker then this is best moved to the CMS repo for the automated testing team to discuss.

@mbabker I had to express this idea and my first thought that this was a good fit for the issue tracker. Well I need to find that other repo and copy paste it there (won't happen, @rdeutz is the only one that could approve/reject this and he's already mentioned a few times already)

It comes down to a cost thing as well @dgrammatiko our existing AWS downloads are costing the project give or take $25k a year - to start hosting the PR's there as well would need a cost evaluation as well

@wilsonge so the test platform will be cheaper than an S3 instance? I think not

we shouldnt be paying a-z for anything - it should be a sponsorship

@dgrammatiko I had the same idea a while ago but at the end it was a question of money. We might have now a 2nd chance because there are some changes with our releation to AWS S3. I will try to find out what we can do.

I think that it 's better if we improve our current efforts to document and introduce the current tools (and avoid new efforts on different paths).

For instance, this is the main documentation page for Patchtester https://docs.joomla.org/Component_Patchtester_for_Testers and it has a video created in 2013.

Videos and Tutorials about how to use Patchtester on Joomla 4, for every use case, localized in different languages, would be of great help. At the end of the day, that's what we need to reach more testers.

Effort needs to go into replacing the current tool, i.e. com_patchtester. It was never a comprehensive tool, realistically only works correctly if you're testing on a nightly build package or running from git (and only on pull requests without SQL changes because core does not have a rollback ability, meaning once you apply a pull with a SQL change the only way to roll back is to completely re-install Joomla), and with the changes in 4.0 the number of pull requests that it can work with are fewer and fewer because there is no sane way to make the component able to work with NPM or Composer (and if you know how to use those tools you probably aren't using com_patchtester). It's a stopgap measure aimed to help less technical users (who don't have the expertise or patience to be able to run a git apply command) that IMO is reaching its end of useful lifetime. Better alternatives have already been worked on, how about finishing deploying those alternatives (i.e. the PR testing platform) instead of continuing to waste contributor time on things that get created but never used (as is the case with way too many GSoC projects).

how about finishing deploying those alternatives

Well, the production team decided to remove the composer and npm pulled dependencies last April and iirc we agreed at that time that this will be merged and gave a grace period where the common workflow for testers will be broken.

Nine (9) months later there's zero change in the status. The workflow is still broken.
If we do want to convince people that changes are needed and that we're not doing things without planning, this whole situation is working against us. People are even more frightened about the upcoming changes and the trust is broken. Basically, it makes all the production team and the contributors that pushing the state to look like the silly folks that have no clue what they're doing.

Fixing the testers workflow will bring back some of that lost trust. This is already delayed way too much. The production team needs to allocate devs, funds or whatever else is needed and make this happen. ASAP

All I can do is say that from the webmasters aspect of things everything I was asked for has been completed (server space mainly). If someone who knows that platform is serious about deploying it, contact me and I will grant access so they can configure the server.

I think that it 's better if we improve our current efforts to document and introduce the current tools (and avoid new efforts on different paths).

For instance, this is the main documentation page for Patchtester https://docs.joomla.org/Component_Patchtester_for_Testers and it has a video created in 2013.

Videos and Tutorials about how to use Patchtester on Joomla 4, for every use case, localized in different languages, would be of great help. At the end of the day, that's what we need to reach more testers.

Actually there's updated(ish) page here https://docs.joomla.org/Special:MyLanguage/Testing_Joomla!_patches including DETAILED video's on patch testing I created last year.
I agree very strongly though on the need for far better documentation

It'd be good to refresh the com_patchtester page a little bit and somehow tie together that, the testing patches page, and https://docs.joomla.org/J4.x:Setting_Up_Your_Local_Environment especially for 4.0 contributors (along with whatever other resources are relevant). Those three pages are at a high level going to target enough potential contributors of a wide enough degree of skillsets that they should be good entry points into the docs on their own. https://docs.joomla.org/Portal:Bug_Squad would probably be another one to bring into the fold since that's the team's portal page.

Ok, this issue fulfilled its purpose, closing