User tests: Successful: Unsuccessful:
Pull Request for Issue #9361 .
Convert #__finder_xxx
tables to utf8mb4_unicode_ci collation
See #9361
Upgrade joomla from version 3.4.8, 3.5.x or 3.7.2 still works.
Joomla should contain contents with words like:
Details:
#__finder_xxx tables has unicode collation.
Upgrade OK
Missing unicode collation in #__finder_xxx tables.
I do not know.
Status | New | ⇒ | Pending |
Category | ⇒ | Administration com_admin Installation SQL Libraries |
Labels |
Added:
?
|
We had excluded the change from general to unicode collation for the com_finder tables because in some languages it has caused "duplicate index" problem after the conversion. To fix that it would have required to force a complete rebuild of the index. We have had discussions with the com_finder guru, @chrisdavenport , and we concluded to leave com_finder tables with general collations in order not to provoke problems in certain languages (I think it was Greece) where things have been OK before.
I know - the example with
ausschließlich
ausschliesslich
was the example we struggled with. The code in script.php here is designed to fix that problem we encountered.
should i mark the Test successfully?
To be honest I am not sure yet if that will work, and I am not sure yet if I will find to review those many changes.
Well a first look on the code looks OK to me. But I have not checked yet how it will be shown in the "Extensions -> Manage -> Database" view. It could be confusing if people are told they have not been converted to utf8mb4 yet if they were, so maybe there should be a separate check with a separate message text telling that it is conversion to unicode collation of the com_finder tables which has to be done.
And I still would like to have @chrisdavenport 's opinion on if that could cause problems e.g. for Greece.
@franz-wohlkoenig From my point of view your test is unsuccessful because it seems that @csthomas 's scripts did not run.
I have tested this item
Looks like it should work in theory (haven't tested it). My only concern would be on sites with substantial numbers of entries in the terms table. Copying all that data from one table to another and back again and then building a new index could take a lot of time. What are the consequences if the request times out?
Category | Administration com_admin Installation SQL Libraries | ⇒ | Administration com_admin Language & Strings Installation SQL Libraries |
Labels |
Added:
?
|
Based on this https://stackoverflow.com/questions/4685173/delete-all-duplicate-rows-except-for-one-in-mysql I think this is not a bad solution.
Maybe creating a copy of structure and then copy data only once would be better. I will check it later.
At above commits first I want to use mainly UPDATE
and DELETE
queries but this queries does not use indexed column term
and require some joins which can slow down all conversion.
Then I back to simpler INSERT INTO ... SELECT ... ON DUPLICATE KEY UPDATE
.
It takes more memory (another table for a moment) but should be faster.
Disadvantage:
This PR does not update #__finder_links_terms[0-9a-f]
tables.
As nobody need it I close it.
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2017-11-04 12:26:34 |
Closed_By | ⇒ | csthomas |
Update from 3.5.1 > 3.7.2, no Database-Fix necessary, "ausschließlich", "ausschliesslich" and "hätte" is found as before Update.
But in phpMyAdmin Tabel looks same as before Update: