? Pending

User tests: Successful: Unsuccessful:

avatar wilsonge
wilsonge
9 Mar 2016

Summary of Changes

If we haven't performed the UTF8MB4 conversion on core tables then we should still strip utf8mb4 characters

Testing Instructions

Update to Joomla 3.5 beta 5 through extension manager on a utf8mb4 compatible server (do not manually apply the database fixer in extension manager to do the utf8mb4 upgrade). Try and insert some 4 byte characters into the database in a sentence - at first this will cause data loss (all characters will be stripped after the 4 byte character will vanish) as the server hasn't been utf8mb4 provisioned yet. Apply patch and perform the same test -the utf8mb4 character will now be replaced with \xe2\xAf\x91 character (reference: http://www.utf8-chartable.de/unicode-utf8-table.pl?start=11136&number=128&names=-&utf8=string-literal) and you won't loose any other characters.

Do the database extension fix to upgrade the tables to utf8mb4 - the 4 byte charaters can now be inserted without any issues.

Note this adds effectively a extra DB query to every single page load - is it really required for this edge case?

avatar wilsonge wilsonge - open - 9 Mar 2016
avatar wilsonge wilsonge - change - 9 Mar 2016
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 9 Mar 2016
Labels Added: ?
avatar andrepereiradasilva
andrepereiradasilva - comment - 9 Mar 2016

Deleted previous comments and made new test. I don't see any characters disappearing in any case.

Update to Joomla 3.5 beta 5 through extension manager on a utf8mb4 compatible server (do not manually apply the database fixer in extension manager to do the utf8mb4 upgrade).

Done

Try and insert some 4 byte characters into the database in a sentence - at first this will cause data loss (all characters will be stripped after the 4 byte character will vanish) as the server hasn't been utf8mb4 provisioned yet.

Result:
image

Apply patch and perform the same test -the utf8mb4 character will now be correctly stripped and you won't loose any other characters.

Result
image

Do the database extension fix to upgrade the tables to utf8mb4 - the 4 byte charaters can now be inserted without any issues.

image

BTW used always the same 4 byte character ????

avatar wilsonge
wilsonge - comment - 10 Mar 2016

So parts 2 and 3 are good for me (amended test instructions as was my bad - we don't strip the utf8mb4 put replace it with that "box" symbol). I don't understand why you get the question mark in the first step tho :/

avatar andrepereiradasilva
andrepereiradasilva - comment - 10 Mar 2016

Don't know, this is the test server info:

  • Database Version: 5.5.5-10.0.24-MariaDB
  • Database Collation: utf8_unicode_ci
  • Database Connection Collation: utf8mb4_general_ci
  • PHP 5.6.18

Tests made in chrome.

avatar richard67 richard67 - test_item - 10 Mar 2016 - Tested successfully
avatar richard67
richard67 - comment - 10 Mar 2016

I have tested this item :white_check_mark: successfully on dd90196

My test looked exactly the same as andre's, i.e. all as described, and in step 1 a "?".

Database Version: 5.5.47-0+deb7u1-log
Database Collation: utf8_general_ci
Database Connection Collation: utf8mb4_general_ci when looking after the conversion.
PHP Version: 5.6.18
Browser: Firefox 44.0.2

For me this is a positive test result, because the new code after patch works as described, even if the thing in general is worth a discussion.

I did not see any performance problems when testing, so maybe we could live with it like done with this PR?

And I would not rely on the conversion always be successful after using update component to update.

Also I would not say we should not stop support updating with extension installer anymore as long as we have it working (except of this necessary database fix to do the conversion). Some people still might need it, and so they will also need this PR.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9354.

avatar brianteeman brianteeman - change - 10 Mar 2016
Category Libraries
avatar wilsonge wilsonge - change - 11 Mar 2016
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2016-03-11 00:01:10
Closed_By wilsonge
avatar wilsonge wilsonge - close - 11 Mar 2016
avatar wilsonge wilsonge - close - 11 Mar 2016
avatar wilsonge wilsonge - head_ref_deleted - 11 Mar 2016

Add a Comment

Login with GitHub to post a comment