User tests: Successful: Unsuccessful:
The purpose of this is to introduce project specific behaviours consisting of some common PHP code and project specific settings stored in the database.
In the Joomla! world this would be PluginsĀ®
I am not sure about the implementation, so this is just a first scratch (but it should work) and I am wide open for suggestions ;)
So this would be the first implementation.
It works similar to those automated services like Travis, Scrutinizer etc. but for human test results created by our JTracker Application with the help of our community members from the JBS and others - it wouldn't work without them :)
GitHub allows four different status states called pending
, success
, error
and failure
.
So we might use three of them and map our human test results to something like:
pending
- less than 2
successful and 0
failing tests.success
- 2
or more successful and 0
failing tests.failure
- 0
or more successful and 1
or more failing tests.See: GitHub API - Create a status
This would be the database entry to activate the update status action for a project.
You should probably use a test project for testing...
INSERT INTO `#__tracker_actions` (`id`, `project_id`, `type`, `name`, `params`) VALUES
(1, 4, 'GitHub', 'UpdateStatus', '
{
"humanTestResults": {
"active": "1",
"pending": {
"comp_success": "<",
"cnt_success": "2",
"comp_failure": "=",
"cnt_failure": "0"
},
"success": {
"comp_success": ">=",
"cnt_success": "2",
"comp_failure": "=",
"cnt_failure": "0"
},
"error": {
"comp_success": "",
"cnt_success": "",
"comp_failure": "",
"cnt_failure": ""
},
"failure": {
"comp_success": ">=",
"cnt_success": "0",
"comp_failure": ">=",
"cnt_failure": "1"
}
}
}
');
Currently our human test results are not bound to a specific commit as the CI services like Travis or Scrutinizer are.
This means if a PR receives new commits, the test results would be automatically "invalidated" and testers would be required to submit their results again.
I believe this is good since it does not bother the testers when a PR receives only code style changes, on the other hand the code changes might require new tests.
So there are two solutions:
It sounds useful but I am struggling to get my head around your examples
@elkuku I like the idea and code looks good
I've not tested it yet, but my question will be more general - as I understand this code can be used not only for the automatic processing, but also to run some common actions in the different points of our Apps? For example #598 - does it make sense to create some kind of AddLabel
action?
Oh, got it! Take care and get well soon!
Status | New | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2015-09-03 18:18:34 |
Closed_By | ⇒ | b2z |
@brianteeman as you are one of the most frequent users of our JTracker thingy - do you think that this would be a useful addition (from the functionality point of view)?
@infograf768 as one of the less frequent users.. could this be useful to you and the JBS workflow?