The system tests for tuf_metadata are completely wrong. I dont really know enough about the tests but after they have run the #tuf_metadata table is empty
db_setValidTufRoot deletes any existing tuf_metadata instead of creating metadata
it looks like its just a copy paste of db_setInvalidTufRoot
joomla-cms/tests/System/support/commands/db.mjs
Lines 654 to 666 in a89c179
Labels |
Added:
No Code Attached Yet
|
Can't follow you. The test indeed removes any previous metadata and inserts either valid or invalid root metadata in a new row.
Did I inspect the DB content during a test? Nope. However, the corresponding system tests check for different system messages that depend on the existence of the respective rows in the database. An empty metadata table would create a different error - that's why I'm assuming that the tests work as expected in our testing setup:
after running the tests the db is empty
So, maybe I'm missing something here: what's the usecase of those system tests? So far I was under the assumption that they would be run "from scratch", creating a new installation with sample content for each and every run; if that's the case, where's the problem with the empty TUF table in the first place?
And if it's really an issue, wouldn't it be the proper fix to restore the default, valid root data after each test?
The tests should be repeatable. As far as I can assert they are not because the command to create valid tufdata which should run after each individual test doesn't do that.
Status | New | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2025-02-12 07:14:43 |
Closed_By | ⇒ | SniperSister |
For performance reasons we do not do a new installation before every test, but a cleanup afterwards (actually it would be better to prepare the system in a state before the test runs in the actual spec file). But we decided for a cleanup afterwards (on some point we probably change that, but for now it is as it is). So I do agree here that in after each, we should leave the database in the state as it was before the test run.
cc @SniperSister as it was your PR #42799 that introduced this faulty test