What is difficult to test here? Updates have to work from 3.10 to that on both kinds of databases without any SQL error and after an update or a new installation the database checker (System - Information - Database) shall not show any database errors.

Drone even builds you the update container for the update test.

Have you tested that yourself first?

Regarding reordering of statements: I would have preferred to keep the order, because otherwise you have to pretty much know what you are doing. E.g. if there is an insert into the extensions table with using real null values for the checked_out_time before this column has been modified to allow real nulls, it causes an error, so in some of the update sql we had to use the old fashioned pseudo null dates still. Same with existance of columns when doing insert statements.

Of course you can optimize that, too, and reorder, but then you have to test it carefully with the new installation and update tests like I said.

Do we even need any of these.
As there was no official support for updates before beta don't we just need to delete them all and just replace them all by a single sql for the update from 3.10 and then from the point this is merged continue to create new sql files

Do we even need any of these.
As there was no official support for updates before beta don't we just need to delete them all and just replace them all by a single sql for the update from 3.10 and then from the point this is merged continue to create new sql files

We can do that if chosing a file name with version and date so it doesn't run the script again when updating a 4.0 Beta 1 to something later (this can be fulfilled by chosing a name with version and date not younger than the youngest script to be combined), and we need to make sure the combined script exactly does the same as the bunch of scripts before.

Since the updater and also the fixer both run the script not a a whole but split it up into statements and then runs statement by stament, there should not be any new timeout issue coming when combining the scripts.

P.S.: But as I said in my previous comment, the order of the statements has to fit together, structure and data changes, and it would be safest just to keep the existing order because of this we know that it works.

Hmm, but maybe memory limit could be a problem when all is in one update sql script? As I said, the complete file is split into single SQL statements and these are saved in an array.

These are really a lot of changes and from a maintenance position, I wouldn't want to just squash everything in one big file. I of course paid attention to check for the order of the queries and especially changed columns. I also unified the inserts. I fear that we will have an issue with upgrading large sites and here it would be good to have this split up into several parts, so that we can execute this one step at a time.

One test I have forgotten in my post before: It needs to test if an update of current 4.0-dev or a nightly or 4.0 Beta 1 to a later 4.0 version (i.e. the update package or custom update URL built for this PR) works.

So it needs 3 tests, each of the to be tested on each kind of database MySQL (or MariaDB) and PostgreSQL:

• New installation: Apply patch and make new installation (or make new install with the installation package built for this PR by Drone) and check in "System - Information - Database" that there are no errors.
• Update from 3.10: Update a current 3.10-dev branch or latest 3.10 nightly build to the update package built for this PR and check in "System - Information - Database" that there are no errors.
• Update from 4.0: Update a current 4.0-dev branch or latest 4.0 nightly build or a 4.0 Beta 1 to the update package built for this PR and check in "System - Information - Database" that there are no errors.

During all 3 tests it needs to watch the database server log files. On MySQL there should be no errors or warnings, on PostgreSQL there should only be lots of warnings about double backslash in strings but nothing else.

Maybe one may wonder why it needs the first test with new installation when this PR doesn't change anything on the installation sql files. The answer is clear: Also after a new installation, the database checker checks the update sql scripts when going to "System - Information - Database", and if there is some error in them compared to a new installation, the fixer will find database problems.

@Hackwar Please update your testing instructions accordingly so people test completely and correctly.

@Hackwar Just tested updating a 3.10-dev to the update package built by drone for this PR on MySQL, and I got following error: Column count doesn't match value count at row 52. That means you are doing an insert statement somewhere where the count of columns in the columns list is not equal to the count of values. No idea which script it is, but it seems to insert at least 52 rows, because that's the row number within the insert statement. Find the script yourself.

Did the Postgres update files as well and fixed a few bugs.

The build artifacts are no longer online, alternative test instructions or run the ci again?

The build artifacts are no longer online, alternative test instructions or run the ci again?

I started a rebuild, new artifacts are available from https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/29374/downloads/34477/

The build artifacts are no longer online, alternative test instructions or run the ci again?

I started a rebuild, new artifacts are available from https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/29374/downloads/34477/

I've updated the link in the testing instructions to reflect this.

I have tested this item ✅ successfully on ebbba74

Tested with updaing 3.10-dev to this version and MySQL.

The database checker throws no errors, just a single warning about mismatching joomla version (your PR instead of actual 4.0), which
i think can be ignored.