? ? Success

User tests: Successful: Unsuccessful:

avatar andrepereiradasilva
andrepereiradasilva
23 Aug 2016

Summary of Changes

Run CS Tests in a specific 5.6 build to improved speed of travis checking.
Also reorders execution matrix for faster tests (nodejs to last, cs to first).
Also disables xdebug is cs tests for faster performance.

Improvements

  • Mandatory tests time: From 8:00 min to 4:45 min (aroud 40% improvement)
  • All tests time: From 20:00 min to 13:00 min (aroud 35% improvement)

  • CS Results in: 7:00/8:00 min to 2:00/3:00 min (more than 60% improvement)

Before

image

After

image

Testing Instructions

Unit test + cs tests pass.
Code review.

Documentation Changes Required

None.

avatar joomla-cms-bot joomla-cms-bot - change - 23 Aug 2016
Category Unit Tests Repository
avatar andrepereiradasilva andrepereiradasilva - open - 23 Aug 2016
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 23 Aug 2016
Labels Added: ? ?
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

@photodude @mbabker what do you think?

avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
Title
[Proposal] Run CS Tests in a specific 5.6 build
[Proposal] Run CS Tests in a specific 5.6 build (and reorder execution matrix)
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

BTW this is also as the advantage that the CS checks are the first to be run so in no more than 4 minutes (instead of aprox 8 minutes before) we have the CS test results. Where actually most times travis fails.

avatar zero-24 zero-24 - change - 23 Aug 2016
Status Pending Ready to Commit
avatar zero-24
zero-24 - comment - 23 Aug 2016

Looks good and travis is happy too.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11757.

avatar joomla-cms-bot joomla-cms-bot - change - 23 Aug 2016
Labels Added: ?
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

@zero-24 now i disabled xdebug in cs tests let's see how much will improve further.

avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

nice even faster 2-3 min and you have the cs results :)
See https://travis-ci.org/joomla/joomla-cms/builds/154430337

avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
Title
[Proposal] Run CS Tests in a specific 5.6 build (and reorder execution matrix)
[Proposal] Run CS Tests in a specific 5.6 build + disable xdebug for cs tests (and reorder execution matrix)
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar photodude
photodude - comment - 23 Aug 2016

Probably a good choice to run the code standards checks separate from the unit tests.

avatar mbabker
mbabker - comment - 23 Aug 2016

We can probably disable XDebug on PHPUnit too because we aren't generating
code coverage on Travis. And it may help with PHP 7.1's build segfaulting.

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

ok php debug disabled in unit tests too. testing ...

avatar photodude
photodude - comment - 23 Aug 2016

@andrepereiradasilva Since you are disabling xdebug for everything now, IMO the cs-tests.sh file is no longer needed.

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

now time is like this:
image

So in around 5 minutes we have all mandotory tests done :)

avatar photodude
photodude - comment - 23 Aug 2016

On a side note: the total test time is a skewed value and is highly dependent on the hhvm test and is highly variable. Tests will be marked complete long before the total test time completes since the matrix build is marked complete when the build jobs pass even though the allowed failure tests are still running.

Our actual test start to completion is just the longest time listed in the build jobs portion of the Travis CI matrix.

avatar zero-24 zero-24 - change - 23 Aug 2016
Status Ready to Commit Pending
avatar yvesh
yvesh - comment - 23 Aug 2016
avatar joomla-cms-bot joomla-cms-bot - change - 23 Aug 2016
Labels Removed: ?
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

On a side note: the total test time is a skewed value and is highly dependent on the hhvm test and is highly variable. Tests will be marked complete long before the total test time completes since the matrix build is marked complete when the build jobs pass even though the allowed failure tests are still running.
Our actual test start to completion is just the longest time listed in the build jobs portion of the Travis CI matrix.

I know that, but as long as the hhvm test don't finish it's not released to do other tests in other PR in the queue ...

avatar photodude
photodude - comment - 23 Aug 2016

It should be doing all the tests in the build matrix and marking the build complete before testing the allowed failures; unless travis isn't fully following their build matrix and documentation about the "fast finish" directive.

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

i don't meand the tests in the same commit i mean when you do another commit that is added to the tests queue.

avatar rdeutz
rdeutz - comment - 23 Aug 2016

I tried to test this but the xdebug disable doesn't work, anything I missed? Here is the travis-job: https://travis-ci.org/rdeutz/joomla-cms/jobs/154519045

avatar rdeutz
rdeutz - comment - 23 Aug 2016

ahh but why not removing earlier not needed for composer

avatar rdeutz
rdeutz - comment - 23 Aug 2016

played around with it a bit seems you can't disable xdebug for the composer self-update but for anything else. I also tried to let all 5.6 together, because from I have seen is that travis starts 5 jobs and then the next one when one is finished. With two 5.6 jobs it took 17 minutes and in combination 13 miniutes. Can't really say what is better because I don't have so much tests run, so we might need to check both configurations.

@andrepereiradasilva that's is a good improvement, great job!

avatar yvesh
yvesh - comment - 23 Aug 2016

@andrepereiradasilva imo you can remove xdebug always no reason for that weird if ;-)

avatar photodude
photodude - comment - 23 Aug 2016

@andrepereiradasilva would you disable Xdebug in the hhvm unit tests as follows
if [[ $TRAVIS_PHP_VERSION = hhv* ]]; then echo xdebug.enable=0 >> /etc/hhvm/php.ini; fi

avatar rdeutz
rdeutz - comment - 23 Aug 2016

I have created a remove xdebug script and called it Branch: https://github.com/rdeutz/joomla-cms/tree/test11757

avatar photodude
photodude - comment - 23 Aug 2016

@yvesh the if is necessary since HHVM and PHP 7.1 will error as they have a different format to configure the environment settings.

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

ok will add it

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

@mbabker it seems php 7.1 doesn't have xdebug enabled.

did a test to check the php conf in 7.1

ls -la ~/.phpenv/versions/$(phpenv version-name)/etc/
cat ~/.phpenv/versions/$(phpenv version-name)/etc/php.ini
ls -la ~/.phpenv/versions/$(phpenv version-name)/etc/conf.d/
cat ~/.phpenv/versions/$(phpenv version-name)/etc/conf.d/travis.ini

And didn't found any xdebug.

Note. didn't found it in hhvm too...

ls -la /etc/hhvm/
cat /etc/hhvm/php.ini
cat /etc/hhvm/server.ini
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

Latest test result: We are now below 5min to do all required tests ?

image

avatar mbabker
mbabker - comment - 23 Aug 2016

Well, that's interesting, since it's there by default on the other PHP builds IIRC. Still doesn't explain now why PHPUnit segfaults on the Travis environment, but I guess that's something to bang my head over on the existing issue.

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

ok. so @wilsonge @rdeutz can this be merged so all tests run faster?
We can do more improvements later ...

avatar andrepereiradasilva andrepereiradasilva - change - 23 Aug 2016
The description was changed
avatar photodude
photodude - comment - 23 Aug 2016

I'm still of the option that the added cs.tests.sh File is unnecessary since that change was added to the original bash build file. We should try to avoid making our Travis set up unnecessarily complex.

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

ok let's see if it works like this (file cs-tests.sh removed)

e988441 23 Aug 2016 avatar andrepereiradasilva ups
8201e87 23 Aug 2016 avatar andrepereiradasilva cs
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

ok so i reverted back to have the cs plus unit in 5.6 in one go. i think it's better now

avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

hum ... it seems it's faster to have one php 5.6 for the unit tests.

If tests of the latest commit pass IMO is ready to merge.

avatar rdeutz rdeutz - change - 23 Aug 2016
Status Pending Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2016-08-23 19:34:29
Closed_By rdeutz
avatar andrepereiradasilva
andrepereiradasilva - comment - 23 Aug 2016

thanks @rdeutz finnaly we don't have to wait almost 10 minutes for the cs results!

Add a Comment

Login with GitHub to post a comment