Success

User tests: Successful: Unsuccessful:

avatar elkuku
elkuku
24 Jan 2015

Project specific actions

The purpose of this is to introduce project specific behaviours consisting of some common PHP code and project specific settings stored in the database.
In the Joomla! world this would be PluginsĀ®

I am not sure about the implementation, so this is just a first scratch (but it should work) and I am wide open for suggestions ;)

Update GitHub status for human test results (JBS)

So this would be the first implementation.

bildschirmfoto von 2015-01-24 15 58 38

It works similar to those automated services like Travis, Scrutinizer etc. but for human test results created by our JTracker Application with the help of our community members from the JBS and others - it wouldn't work without them :)

GitHub allows four different status states called pending, success, error and failure.

So we might use three of them and map our human test results to something like:

  • pending - less than 2 successful and 0 failing tests.
  • success - 2 or more successful and 0 failing tests.
  • failure - 0 or more successful and 1 or more failing tests.

See: GitHub API - Create a status

@test

This would be the database entry to activate the update status action for a project.

You should probably use a test project for testing...

INSERT INTO `#__tracker_actions` (`id`, `project_id`, `type`, `name`, `params`) VALUES
(1, 4, 'GitHub', 'UpdateStatus', '
{
  "humanTestResults": {
    "active": "1",
    "pending": {
      "comp_success": "<",
      "cnt_success": "2",
      "comp_failure": "=",
      "cnt_failure": "0"
    },
    "success": {
      "comp_success": ">=",
      "cnt_success": "2",
      "comp_failure": "=",
      "cnt_failure": "0"
    },
    "error": {
      "comp_success": "",
      "cnt_success": "",
      "comp_failure": "",
      "cnt_failure": ""
    },
    "failure": {
      "comp_success": ">=",
      "cnt_success": "0",
      "comp_failure": ">=",
      "cnt_failure": "1"
    }
  }
}
');

Test results are NOT bound to a specific commit

Currently our human test results are not bound to a specific commit as the CI services like Travis or Scrutinizer are.
This means if a PR receives new commits, the test results would be automatically "invalidated" and testers would be required to submit their results again.

I believe this is good since it does not bother the testers when a PR receives only code style changes, on the other hand the code changes might require new tests.

So there are two solutions:

  1. Whenever the PR gets new commits use the currently available test results for the latest commit and submit the results automatically to GitHub (currently implemented)
  2. Make the test results bound to a specific commit as the other services. This would require testers to resubmit their test results whenever a PR gets new commits (to be implemented).
avatar elkuku elkuku - open - 24 Jan 2015
avatar elkuku
elkuku - comment - 28 Jan 2015

@brianteeman as you are one of the most frequent users of our JTracker thingy - do you think that this would be a useful addition (from the functionality point of view)?
@infograf768 as one of the less frequent users.. could this be useful to you and the JBS workflow?

avatar brianteeman
brianteeman - comment - 28 Jan 2015

It sounds useful but I am struggling to get my head around your examples

avatar b2z
b2z - comment - 28 Jan 2015

@elkuku sorry I am currently quite busy and do not have much spare time to dig in. I will take a look on it on this weekend. From the first look I would say it is quite interesting.

avatar b2z
b2z - comment - 1 Feb 2015

@elkuku I like the idea and code looks good :+1:

I've not tested it yet, but my question will be more general - as I understand this code can be used not only for the automatic processing, but also to run some common actions in the different points of our Apps? For example #598 - does it make sense to create some kind of AddLabel action?

avatar b2z
b2z - comment - 1 Feb 2015

Testing now. Found a bug - when I am submitting the test from the test module then a test status is not visible in the activity
test1

After page refresh everything is ok.

And the question - why human tests named as default for my installation? See here.
test2

avatar b2z
b2z - comment - 2 Mar 2015

@elkuku are you working on it? :wink:

avatar elkuku
elkuku - comment - 3 Mar 2015

@elkuku are you working on it? :wink:

I have a bit of a health issue that keeps me away from my PC currently, so sadly I see myself limited to posting comments here and there..

If someone else could take this over, improve the idea if its useful, that would be great ;)

avatar b2z
b2z - comment - 3 Mar 2015

Oh, got it! Take care and get well soon!

avatar b2z b2z - change - 3 Sep 2015
Status New Closed
Closed_Date 0000-00-00 00:00:00 2015-09-03 18:18:34
Closed_By b2z
avatar b2z b2z - close - 3 Sep 2015
avatar b2z b2z - close - 3 Sep 2015
avatar elkuku elkuku - head_ref_deleted - 5 Sep 2015

Add a Comment

Login with GitHub to post a comment