? Language Change PR-5.0-dev Pending

User tests: Successful: Unsuccessful:

avatar richard67
richard67
6 Aug 2022

Pull Request for Issue # .

Summary of Changes

This pull request (PR) removes all code related to the utf8mb4 conversion.

Joomla 4.0 and later require a minimum database version for MySQL or MariaDB which supports utf8mb4 and so would not run on databases which do not support it.

But as a result of a sever migration when updating from 3.10 to 4.x, it could still be that the database tables of the core haven't been converted to utf8mb4 yet, so the update to 4.x had to make that conversion.

Given that people have to be on a Joomla 4.x before they can update to 5.0, so updates from 3.10.x to 5.0 will not be possible, it is granted that the database has been converted in past when they had updated from 3.10.x to 4.y.

So for updating to 5.0 all code and SQL scripts related to that conversion are obsolete.

Testing Instructions

The tests make only sense with MySQL or MariaDB databases but not with PostgreSQL.

Update any Joomla 4.x version to the custom update URL or the update package which have been built by drone for this PR and make sure that the update works.

After that, make sure that the database checker ("System -> Maintenance -> Database", i.e. administrator/index.php?option=com_installer&view=database) still works.

Actual result BEFORE applying this Pull Request

Obsolete, unused code and SQL scripts for utf8mb4 conversion of MySQL or MariaDB databases.

Expected result AFTER applying this Pull Request

No obsolete, unused code and SQL scripts for utf8mb4 conversion of MySQL or MariaDB databases.

Updating from any Joomla 4.x version works as well as without this PR.

The database checker works as well as without this PR.

Documentation Changes Required

None.

avatar richard67 richard67 - open - 6 Aug 2022
avatar richard67 richard67 - change - 6 Aug 2022
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 6 Aug 2022
Category Administration com_admin com_installer Language & Strings Libraries
avatar richard67 richard67 - change - 6 Aug 2022
The description was changed
avatar richard67 richard67 - edited - 6 Aug 2022
avatar richard67 richard67 - change - 6 Aug 2022
Title
[5.0] [WiP] Remove utf8mb4 conversion
[5.0] Remove utf8mb4 conversion
avatar richard67 richard67 - edited - 6 Aug 2022
avatar richard67 richard67 - change - 6 Aug 2022
The description was changed
avatar richard67 richard67 - edited - 6 Aug 2022
avatar brianteeman
brianteeman - comment - 6 Aug 2022

This will mean that any j3 site will have to be updated via j4 - is that the intention?

avatar richard67
richard67 - comment - 6 Aug 2022

This will mean that any j3 site will have to be updated via j4 - is that the intention?

@brianteeman That's what I assume, and as far as I know that's the plan.

@HLeithner @nibra please correct me if that is wrong.

avatar brianteeman
brianteeman - comment - 6 Aug 2022

Just wanted it to be clear. In the absence of any road map its hard to know what the plans are

avatar HLeithner
HLeithner - comment - 6 Aug 2022

Yes there will me no direct update path between 3 and 5

avatar richard67
richard67 - comment - 6 Aug 2022

In the absence of any road map its hard to know what the plans are

@brianteeman As long as we are not on the road to nowhere ... or the highway to hell ... ?

avatar HLeithner
HLeithner - comment - 6 Aug 2022

Depending on the progress we will see if a upgrade to 4.4 is needed (like 3.10) or direct update from any 4.x version is possible.

avatar richard67
richard67 - comment - 6 Aug 2022

Depending on the progress we will see if a upgrade to 4.4 is needed (like 3.10) or direct update from any 4.x version is possible.

@HLeithner This depends mainly on if we want to keep the old 4.x update SQL scripts - then we can update from any J4 version - or if we throw them away like we threw away the 3.x update SQL scripts in J4 - then we have to come from the latest J4 version.

avatar richard67
richard67 - comment - 6 Aug 2022

@HLeithner P.S.: To throw old update SQL scripts away has the advantage to be more flexible with new update SQL scripts regarding the database checker, because that does not understand when you modify the same thing again in the history of update SQL scripts. E.g. you cannot remove an index and later add it back with the same name, or remove a column and later add it back to the same table with the same same, such things, without the database checker showing false errors which cannot be fixed.

avatar HLeithner
HLeithner - comment - 6 Aug 2022

@HLeithner P.S.: To throw old update SQL scripts away has the advantage to be more flexible with new update SQL scripts regarding the database checker, because that does not understand when you modify the same thing again in the history of update SQL scripts. E.g. you cannot remove an index and later add it back with the same name, or remove a column and later add it back to the same table with the same same, such things, without the database checker showing false errors which cannot be fixed.

having a better database management system would also be great^^

avatar Abernyte-Git
Abernyte-Git - comment - 6 Aug 2022

I have tested this item successfully on aa13e59

Works as expected.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/38406.

avatar Abernyte-Git Abernyte-Git - test_item - 6 Aug 2022 - Tested successfully
avatar wilsonge
wilsonge - comment - 9 Aug 2022

We should probably remove the #__utf8_conversion table as part of this PR in this case

avatar richard67
richard67 - comment - 9 Aug 2022

We should probably remove the #__utf8_conversion table as part of this PR in this case

@wilsonge The table is already removed (dropped) after a successful conversion here: https://github.com/joomla/joomla-cms/blob/4.2-dev/administrator/components/com_admin/script.php#L7969-L7974

If the table still exists for some reason in 4.x, the database checker will show that as an error, see https://github.com/joomla/joomla-cms/blob/4.2-dev/libraries/src/Schema/ChangeSet.php#L97-L124 .

Using the "Fix" button will then run the conversion, see https://github.com/joomla/joomla-cms/blob/4.2-dev/administrator/components/com_installer/src/Model/DatabaseModel.php#L305 .

So at the end the table does not exist anymore, or the conversion has failed.

I could add a 5.0.0 update SQL script with a DROP TABLE IF EXISTS ... for MySQL in addition to be sure the table is deleted in any case.

But is that really necessary? If you say yes, I will add the script.

avatar RickR2H
RickR2H - comment - 23 Sep 2022

I have tested this item successfully on aa13e59


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/38406.

avatar RickR2H RickR2H - test_item - 23 Sep 2022 - Tested successfully
avatar richard67 richard67 - change - 5 Oct 2022
Status Pending Ready to Commit
Labels Added: Language Change PR-5.0-dev
avatar richard67
richard67 - comment - 5 Oct 2022

RTC (seems nobody else is here who checks if a PR has 2 tests)


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/38406.

avatar HLeithner HLeithner - change - 9 Oct 2022
Status Ready to Commit Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2022-10-09 19:08:33
Closed_By HLeithner
Labels Added: ?
avatar HLeithner HLeithner - close - 9 Oct 2022
avatar HLeithner HLeithner - merge - 9 Oct 2022
avatar HLeithner
HLeithner - comment - 9 Oct 2022

thanks

avatar zero-24
zero-24 - comment - 9 Oct 2022

Joomla 4.0 and later require a minimum database version for MySQL or MariaDB which supports utf8mb4 and so would not run on databases which do not support it.

Just FYI this should be the case but the minimum database version is not enforced you can still install and manually update. The Core Updater checks the versions but the CMS itself does not. And we can not enforce this before 5.0 as that would be an unexpected major break while installing an update.

Add a Comment

Login with GitHub to post a comment