[#1088] - [RFC] Auto Labeling on issues.joomla.org and GitHub to help first time testers and PBF / Bugs and Fun @home
- Closed
- 25 Aug 2020
- Medium
- Build: master
- # 1088
Problem statement / Abstract
We need a solution that is centered around GitHub to identify patches to test at PBF and BF@H and the best point to do this is when the patch is added rather than a review needing to be done after. It is beneficial for those new to testing to know that their setup is capable of completing a test. If they start several and cannot finish them, we will have discouraged testers. But if there is a matching of tests to the ability and environment of the tester it should make for more completed tests.
It also means that the lists will be up to date whereas those held in a spreadsheet soon become out of date and require regular checking and deletion.
It also allows for the simplification of PBF and BF@H as a simple link will bring the issues that match the three criteria set.
Conversations with the bug squad have had a favourable response as well.
Proposed solution
Proposed Github Pull request template changes
Pull Request for Issue # .
### Summary of Changes
### Testing Questions
<!-- Please answer this questions by adding an X inside the [ ] right before the questions to get the right testers for this issue. -->
<!-- Feel free to ask whether you are not sure how to answer the questions. -->
- [ ] Can this PR be tested with [patchtester](https://docs.joomla.org/Special:MyLanguage/Component_Patchtester_for_Testers)?
- [ ] Can this PR be tested with the [prebuild package](https://docs.joomla.org/Testing_Joomla!_patches#Prebuild_packages)?
- [ ] Will the tester need to have NPM or Composer installed on their system?
- [ ] Are actions on the part of the tester needed in order to complete this test ie, Run a script or make a database change?
### Testing Instructions
[...]
On PR submission and PR Updates the issue tracker parses the initial post for the mention template and checks whether the developer ticked the boxes and add the labels as assigned
| Question | Checked | Label |
|---|---|---|
| Can this PR be tested with patchtester? | Yes | tests:pachtester |
| Can this PR be tested with patchtester? | No | tests:not-testable-pachtester |
| Can this PR be tested with the prebuild package? | Yes | tests:packages |
| Can this PR be tested with the prebuild package? | No | tests:not-testable-packages |
| Will the tester need to have NPM or Composer installed on their system? | Yes | tests:npm-composer |
| Will the tester need to have NPM or Composer installed on their system? | No | No additional label |
| Are actions on the part of the tester needed in order to complete this test ie, Run a script or make a database change? | Yes | tests:requires-special-setup |
| Are actions on the part of the tester needed in order to complete this test ie, Run a script or make a database change? | No | No additional label |
Open questions
- General feedback on that idea
- Do you have suggestions for additional or different labels or questions?
Other Infos
This RFC is based on this document: https://docs.google.com/document/d/1xkmjoOp0iPR9Gj7iDOpZVdgZ2Ug4lYivNuG2noP1f6E/edit?usp=sharing setup by @softforge and me.
Sure it is not fool prove. But to me it would be one step in the right direction. And would allowing us to go away from that google sheet and also does not require manuall labeling of all issues.
I get your point about free text and i'm not happy about that too but that was the best idea i could come up with to move forward. Do you have a hint where we might should look to improve the process?
When the regex finds nothing we don't do any labels so to me it would be an improvment for all PRs following the template but would also take care of PRs not following the template.
What would be the difference when the PR creation would go via the issue tracker?
I'm sure there are no plans right now to restore joomlacode nor to selfhost the code again right now.
| Status | New | ⇒ | Closed |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2020-08-25 01:48:04 |
| Closed_By | ⇒ | zero-24 |
Anything that relies on parsing free-form text areas for key phrases is flaky at best and unreliable at worst. The rest of my opinions about how Joomla is wasting an excessive amount of time and energy focusing on flawed testing workflows don't really matter right now, but from a technological perspective the only way to effectively pull this off would be to have all issue and pull request creation go through the issue tracker and not GitHub, and at that point the project might as well invest the time in modernizing JoomlaCode and go back to self hosting its version control repository and issue tracking software while not using any third party solution.