Joomla! Issue Tracker - CMS

Download
Launch
Feeling Lucky

[#7445] - Fix Unit test inter-dependencies

? ?
avatar wilsonge
wilsonge
15 Jul 2015

Using the tool at https://packagist.org/packages/fiunchinho/phpunit-randomizer

1) JDocumentHTMLTest::testEnsureSetHeadDataReturnsThisObject
Exception: Application Instantiation Error

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\factory.php:122
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\html.php:334
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\html.php:680
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\html.php:236
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\html.php:104
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\behavior.php:88
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\html.php:236
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\html\html.php:138
C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\language\text.php:389
C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\document\html\html.php:177
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\joomla\document\html\JDocumentHTMLTest.php:117
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

2) JTwitterTest::test__GetDirectMessages
InvalidArgumentException: Argument directMessages produced an invalid class name: JTwitterDirectMessages

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\twitter\twitter.php:154
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\joomla\twitter\JTwitterTest.php:185
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

3) JGoogleAuthOauth2Test::testAuth
Illegal string offset 'code'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\joomla\google\JGoogleAuthOauth2Test.php:91
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

4) JHelpTest::testCreateURL
Illegal string offset 'option'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\application\helper.php:53
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\help\help.php:38
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\help\JHelpTest.php:83
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

5) JRouterSiteTest::testParseRawRoute with data set #2 ('index.php?option=com_test&Ite...tvalue', true, array(), array('com_test', '42', 'testvalue'))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:393
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

6) JRouterSiteTest::testParseRawRoute with data set #3 ('index.php?Itemid=42', true, array('com_test', 'test'), array(42))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:393
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

7) JRouterSiteTest::testParseSefRoute with data set #8 ('', false, array(), array('45', 'com_test3', 'test3'), array('45', 'com_test3', 'test3'))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:501
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

8) JRouterSiteTest::testParseSefRoute with data set #11 ('?testvar=testvalue', false, array(), array('45', 'com_test3', 'test3'), array('45', 'com_test3', 'test3'))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:501
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

9) JRouterSiteTest::testParseSefRoute with data set #26 ('test2/sub-menu', true, array(), array('com_test2', 44), array('com_test2', 44))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:501
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

10) JRouterSiteTest::testParseSefRoute with data set #27 ('test2/sub-menu', false, array(), array('com_test2', 44), array('com_test2', 44))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:501
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

11) JRouterSiteTest::testParseSefRoute with data set #30 ('test2/sub-menu', true, array(true), array('com_test2', 44), array('com_test2', 44))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:501
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

12) JRouterSiteTest::testParseSefRoute with data set #31 ('test2/sub-menu', false, array(true), array('com_test2', 44), array('com_test2', 44))
Illegal string offset 'Itemid'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\router\JRouterSiteTest.php:501
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

13) JToolbarButtonHelpTest::testFetchButton
Illegal string offset 'option'

C:\Users\George\Documents\GitHub\joomla-cms\libraries\joomla\input\input.php:277
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\application\helper.php:53
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\help\help.php:38
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\toolbar\button\help.php:79
C:\Users\George\Documents\GitHub\joomla-cms\libraries\cms\toolbar\button\help.php:42
C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\toolbar\button\JToolbarButtonHelpTest.php:100
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

14) JPluginHelperTest::testGetPlugin
Trying to get property of non-object

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\plugin\JPluginHelperTest.php:97
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

--

There were 12 failures:

1) JApplicationWebTest::testRedirect
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (...)
     1 => Array (
-        0 => 'Location: http://mydomain.com/index.php'
+        0 => 'Location: http://example.com/index.php'
         1 => true
         2 => null
     )
     2 => Array (...)
 )

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\joomla\application\JApplicationWebTest.php:1058
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

2) JApplicationWebTest::testRedirectWithHeadersSent
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<script>document.location.href='http://mydomain.com/index.php';</script>
+'<script>document.location.href='http://example.com/index.php';</script>
 '

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\joomla\application\JApplicationWebTest.php:1087
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

3) JLogEntryTest::testDefaultValues
Line: 56.
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'2015-07-15T08:20:06+00:00'
+'2015-07-15T08:20:07+00:00'

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\joomla\log\JLogEntryTest.php:57
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

4) JHtmlFormbehaviorTest::testChosen
Verify that the chosen method initialises jQuery as well
Failed asserting that an array has the key '/media/jui/js/jquery.min.js'.

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\html\JHtmlFormbehaviorTest.php:87
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

5) JApplicationCmsTest::testRedirect
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (...)
     1 => Array (
-        0 => 'Location: http://mydomain.com/index.php'
+        0 => 'Location: http://example.com/index.php'
         1 => true
         2 => null
     )
     2 => Array (...)
 )

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\application\JApplicationCmsTest.php:402
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

6) JApplicationCmsTest::testRedirectLegacy
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (...)
     1 => Array (
-        0 => 'Location: http://mydomain.com/index.php'
+        0 => 'Location: http://example.com/index.php'
         1 => true
         2 => null
     )
     2 => Array (...)
 )

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\application\JApplicationCmsTest.php:451
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

7) JApplicationCmsTest::testRedirectLegacyWithEmptyMessageAndEmptyStatus
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (...)
     1 => Array (
-        0 => 'Location: http://mydomain.com/index.php'
+        0 => 'Location: http://example.com/index.php'
         1 => true
         2 => null
     )
     2 => Array (...)
 )

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\application\JApplicationCmsTest.php:496
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

8) JApplicationCmsTest::testRedirectWithHeadersSent
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<script>document.location.href='http://mydomain.com/index.php';</script>
+'<script>document.location.href='http://example.com/index.php';</script>
 '

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\application\JApplicationCmsTest.php:526
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

9) JPluginHelperTest::testGetPluginGroup
Method should return all plugins in a group
Failed asserting that 0 matches expected 7.

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\plugin\JPluginHelperTest.php:115
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

10) JPluginHelperTest::testIsEnabled
plg_content_loadmodule should be enabled
Failed asserting that false is true.

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\plugin\JPluginHelperTest.php:130
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

11) JHtmlSelectTest::testRadiolist with data set #0 ('<div class="controls">\n  <labe.../div>\n', array(array('1', 'Yes', 'yesId'), array('0', 'No'), array('-1', 'Maybe')), 'myRadioListName')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
 '<div class="controls">
        <label for="yesId" id="yesId-lbl" class="radio">

        <input type="radio" name="myRadioListName" id="yesId" value="1"  >Yes
        </label>
        <label for="myRadioListName0" id="myRadioListName0-lbl" class="radio">

        <input type="radio" name="myRadioListName" id="myRadioListName0" value="0"  >No
        </label>
        <label for="myRadioListName-1" id="myRadioListName-1-lbl" class="radio">

        <input type="radio" name="myRadioListName" id="myRadioListName-1" value="-1"  >Maybe
        </label>
 </div>
 '

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\html\JHtmlSelectTest.php:489
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

12) JHtmlSelectTest::testRadiolist with data set #1 ('<div class="controls">\n  <labe.../div>\n', array(array('foo', 'FOO', 'fooId'), array('bar', 'BAR')), 'myFooBarListName', array('i am radio', 'jsfunc();'), 'key',
 'val')
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
 '<div class="controls">
        <label for="fooId" id="fooId-lbl" class="radio">

        <input type="radio" name="myFooBarListName" id="fooId" value="foo" class="i am radio" onchange="jsfunc();" >FOO
        </label>
        <label for="myFooBarListNamebar" id="myFooBarListNamebar-lbl" class="radio">

        <input type="radio" name="myFooBarListName" id="myFooBarListNamebar" value="bar" class="i am radio" onchange="jsfunc();" >BAR
        </label>
 </div>
 '

C:\Users\George\Documents\GitHub\joomla-cms\tests\unit\suites\libraries\cms\html\JHtmlSelectTest.php:489
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\TestRunner.php:23
C:\Users\George\Documents\GitHub\joomla-cms\libraries\vendor\fiunchinho\phpunit-randomizer\src\PHPUnitRandomizer\Command.php:13

FAILURES!
Tests: 5709, Assertions: 10112, Errors: 14, Failures: 12, Skipped: 306, Incomplete: 1, Risky: 8.

Randomized with seed: 1020

The last two failures are existing for windows due to line ending differences. The rest are a result of randomizing the test order.

The majority seem to be a result of $_SERVER variables not being set up and reset correctly

avatar wilsonge wilsonge - open - 15 Jul 2015
avatar zero-24 zero-24 - change - 15 Jul 2015
Labels Added: ? ?
avatar zero-24 zero-24 - change - 15 Jul 2015
Labels Added: ? ?
avatar zero-24 zero-24 - change - 15 Jul 2015
Category ⇒ Unit Tests
avatar mbabker
mbabker - comment - 19 Jul 2015

Anything hitting the Illegal string offset message with JInput::set() seems to be a false positive.

JGoogleAuthOauth2Test just needs to be completely rewritten. Running the test class in isolation results in several failures because the test is dependent on JOAuth2ClientTest.

avatar mbabker
mbabker - comment - 19 Jul 2015

Also, it would seem that randomizer has a bug in it; some setup code seems to not being executed by it as all the failures with JPluginHelperTest look to be that the test database stuff wasn't created.

avatar mbabker mbabker - reference | 9c795ec - 19 Jul 15
avatar vdespa vdespa - change - 24 Jul 2015
Priority Medium ⇒ Low
Status New ⇒ Confirmed
avatar wilsonge
wilsonge - comment - 26 Jul 2015

I think some of the tests with the incorrect URL might be fixed by #7545 (I don't have the time to check right now)

avatar brianteeman
brianteeman - comment - 17 Feb 2016

@wilsonge its been 9months since the last comment - is this still valid?

This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/7445.

avatar brianteeman brianteeman - change - 17 Feb 2016
Status Confirmed ⇒ Information Required
Labels
avatar wilsonge
wilsonge - comment - 18 Feb 2016

We fixed most of them. I don't have this tool installed anymore, so closing. We can always come back to it!

avatar wilsonge wilsonge - change - 18 Feb 2016
Status Information Required ⇒ Closed
Closed_Date 0000-00-00 00:00:00 ⇒ 2016-02-18 00:16:10
Closed_By ⇒ wilsonge
Labels
avatar wilsonge wilsonge - close - 18 Feb 2016

Add a Comment

Login with GitHub to post a comment