I was surprised that this test spec fails on my Docker-based test for all four branches with the following:

  2 failing

  1) Test in backend that the action logs
       can display a list of actions:
     AssertionError: Timed out retrying after 4000ms: Expected to find content: 'User ci-admin logged in to admin' but never did.
      at Context.eval (webpack://joomla/./tests/System/integration/administrator/components/com_actionlogs/Actionlogs.cy.js:21:7)

  2) Test in backend that the action logs
       can delete selected logs:
     AssertionError: Timed out retrying after 4000ms: Expected to find element: `thead input[name='checkall-toggle']`, but never found it.
      at Context.checkAllResults (webpack://joomla/./node_modules/joomla-cypress/src/support.js:140:0)

After some investigation:

• The reason is not the docker-based test, it is running the overall test suite twice.
• The plugin 'Action Log - Joomla' is deactivated and therefore no action log is written.

1. administrator/components/com_plugins/Plugins.cy.js disables and enables plugin 'Action Log - Joomla', if by fault this test is interrupted, the plugin may stay disabled and then there is no action log – but this was not the reason it was:
2. api/com_plugins/Plugins.cy.js this test spec, as last step, disables the first plugin in the list and this in in alphabetical order 'Action Log' ?

It should be reproducible by

 npx cypress run --spec tests/System/integration/api/com_plugins/Plugins.cy.js,tests/System/integration/administrator/components/com_plugins/Plugins.cy.js

I would recommend to add enable 'Action Log - Joomla' in the beginning of the test spec, after check if the plugin is disabled. This would also cover the first case with the interrupted test. And of course extend the api test by enabling the first plugin in the list as last step.

@alikon what do you think about the suggestion from @muhme?

i'll check in the week-end hopefully

should be more safe than before now
?

should be more safe than before now ?

✅ yes, it is ? successfully tested this additional test spec administrator/components/com_plugins/Plugins.cy.js and the improved api/com_plugins/Plugins.cy.js on the branches dev-4.4, dev-5.1, dev-5.2 and dev-6.0 by running tree times:

tests/System/integration/api/com_plugins/Plugins.cy.js,tests/System/integration/administrator/components/com_plugins/Plugins.cy.js

Sorry, those were not the appropriate test specifications, one more test:

✅ this additional test specification Actionlogs.cy.js together with Plugins.cy.js (this time only executable in 4.4-dev) three times as well as once the entire test suite for 4.4-dev successfully

tests/System/integration/api/com_plugins/Plugins.cy.js,tests/System/integration/administrator/components/com_actionlogs/Actionlogs.cy.js

Should the plugin state not be restored after the tests were running through? After a test, the system should be left in the same stage as before the test. This is because we do not restore an initial database dump before every test runs.

if you refer to this line https://github.com/joomla/joomla-cms/pull/43531/files#diff-8265cec42f94079a40c9f1675f71d2290441d6e25a8225c1b12ae1de7383d07eR3

this was added to cover the scenario:

administrator/components/com_plugins/Plugins.cy.js disables and enables plugin 'Action Log - Joomla', if by fault this test is interrupted, the plugin may stay disabled and then there is no action log

Fine by me then. Thanks!