? ? Language Change ? Pending

User tests: Successful: Unsuccessful:

avatar richard67
richard67
25 Oct 2021

Pull Request for Issue # .

Replaces PR #35838 and #35847 .

Summary of Changes

This pull request (PR) fixes wrong display of strings which should be shown with direction LTR, like alphanumeric version numbers or table names, when the current language has direction RTL.

It fixes wrong display of versions which contain characters (e.g. development, alpha, beta or RC versions, or versions like "1.2.3.v4" like they are used by some Joomla 4 language pack) when the current language is an RTL language in diverse list views in the backend as well as in the pre-update checker's extension details and the database checkers information and error messages.

The implementation follows the recommendations by w3c which can be found here: https://www.w3.org/International/articles/inline-bidi-markup/#whattodo :

  • Add dir="ltr" attribute to HTML elements (spans only up to now) where we know the direction in advance and where it applies to the complete element.
  • Wrap inline text or markup which shall be LTR in the middle of RTL text into span elements with a dir="ltr" and optionally a lang attribute.
  • Avoid using the Unicode RTL marks '‎'.

A new file with a new class and a method "inlineBidirectional" is added to the HTMLHelper (JHtml) which wraps inline text or markup only if its direction is different to the document's direction. The class can later be extended by other methods useful for internationalization.

In the "Live Update" tab of the Joomla Update component, this PR replaces the usage of the RTL mark '‎' for the installed Joomla version and the latest Joomla version by using the above approach. There might be more places where this could and should be done, but that might also be done with a future PR.

Hint for maintainers: Due to changes in J4 it will not just an easy upmerge of this PR into 4.0-dev, so I'm preparing a separate PR for 4.0-dev. The 2 language strings which are added by this PR here will not be needed in J4 because the database checker has been changed in J4.

Request for Comment (RFC)

@bembelimen @wilsonge @zero-24 Questions:

Can this considered to be a progressive enhancement so it can go into 3.10? Or is that a new feature which has to go into 4.1? Or something in the middle, 4.0.x?

I'll rebase or make a replacement PR if necessary.

Testing Instructions

  1. On a current 3.10-dev branch or the latest 3.10 nightly build, download and install the following modified language packs and weblinks package.
  • https://test5.richard-fath.de/de-DE_joomla_lang_full_3.10.1v1.zip
    This is an unmodified but older German language pack so you will get an update to a newer version with a "normal" version number. If you have already installed a newer German language pack, uninstall it and then install the one from the link here.
  • https://test5.richard-fath.de/de-XX_joomla_lang_full_3.10.1v1_test.zip
    This is a modified German language pack which uses a different language tag so it can be installed in parallel to the German language pack mentioned before. It is modified so that both the installed and the available update versions are shown with release candidate "-rcX" version numbers.
  • https://test5.richard-fath.de/fa-XX_joomla_lang_full_3.10.2v1_test.zip
    This is a modified Persian language which uses a different language tag so it can be installed in parallel to any present Persian language pack. It is modified so that both the installed and the available update versions are shown with version numbers like e.g. "3.10.2v1". The Persian language pack for Joomla 4 currently uses such versions, for the test here with Joomla 3 it needs that modified pack.
  • https://test5.richard-fath.de/pkg-weblinks-3.9.0-rc1_test.zip
    This is a modified weblinks package to get installed and update versions shown with release candidate "-rcX" version numbers for having test for different extension types than languages.
  1. Create a new, empty file "3.10.3-2021-10-26.sql" in the folder for update SQL scripts for your database type, i.e. folder "administrator/components/com_admin/sql/updates/mysql" or "administrator/components/com_admin/sql/updates/postgresql".
    This makes sure that in the database checker we get a database error shown about not matching schema versions.

  2. Modify the CMS version in the database so it doesn't match the version given by the "libraries/src/Version.php" file.
    This makes sure that in the database checker we get a database error shown about not matching update versions.
    Example SQL statement for a current 3.10-dev branch and MySQL or MariaDB:

UPDATE `#__extensions` SET `manifest_cache` = REPLACE(`manifest_cache`, '"version":"3.10.4-dev"', '"version":"3.10.3-dev"')
 WHERE `name` = 'files_joomla' AND `type` = 'file' AND `element` = 'joomla' AND `client_id` = 0;
  1. Go to the Joomla Update Component and there to the "Live Update" tab and check the message telling that no update was found. Check with both LTR and RTL language.

Result: See section "Actual result BEFORE applying this Pull Request" below.

  1. In the options, set the update channel to "Custom URL", the minimum stability to "Development" and the custom URL to https://test5.richard-fath.de/next_patch_list_pr-35902.xml . This makes sure that an update for the core will be found for later tests and that this update's version contains some characters. I made this custom update site to point to a 4.0.5-dev version (different to the screenshots used later), but in fact it points to the current nightly build for J4.

  2. Check in the following views all version numbers which contain strings for both kinds of backend languages LTR and RTL. For RTL you can use the previously installed "Persian (Test)".

  • "Pre-Update Check" tab of the Joomla Update component with "More Detail" expanded for the extensions
  • "Live Update" tab of the Joomla Update component
  • Languages: Installed
  • Extensions: Update (Use the "Clear Cache" and then the "Find Updates" button if necessary to find all updates.)
  • Extensions: Manage

Result: See section "Actual result BEFORE applying this Pull Request" below.

  1. Go to "Extensions -> Manage -> Database" and there to the tab which shows the problems created by the previous 2 steps.
    Check the version numbers shown in the problems about not matching versions.

Result: See section "Actual result BEFORE applying this Pull Request" below.

  1. Go to the "Other Information" tab and check the versions and the table names "#__extensions" and "'__schemas".

Result: See section "Actual result BEFORE applying this Pull Request" below.

  1. In database in the extensions table, delete the records for the 'de-DE' and the 'fa-XX' site languages and the weblinks editor button.
  • In phpMyAdmin using the GUI:
    j3-extensions-delete-records-for-discover-in-phpmyadmin
  • With SQL, replace #__ with your database prefix.:
DELETE FROM `#__extensions` WHERE `type` = 'language' AND `element` = 'de-DE' AND `client_id` = 0;
DELETE FROM `#__extensions` WHERE `type` = 'language' AND `element` = 'fa-XX' AND `client_id` = 0;
DELETE FROM `#__extensions` WHERE `type` = 'plugin' AND `element` = 'weblink' AND `folder` = 'editors-xtd';
  1. Go to the "Extensions: Discover" view and check the version numbers which contain strings. Use the "Discover" button if necessary to discover all extensions. Check with LTR and RTL backend languages.

Result: See section "Actual result BEFORE applying this Pull Request" below.

  1. Select the discovered extensions and install them using the "Install" button.

  2. In the options of the Joomla Update Component, switch back the update channel so that no update is found.

  3. Apply the patch of this PR.

  4. Repeat steps 4 to 7. In addition, inspect the markup of versions with developer tools of your browser. Check with both LTR and RTL language.

Result: See section "Expected result AFTER applying this Pull Request" below.

  1. Repeat step 8.

Result: Because the modified Persian language pack doesn't contain the new language string added by this PR you see the English texts, see the orange mark in the following image:

j3-database-check-other-information_rtl-before-upd

  1. Update the modified Persian language pack to the available update. This will update it to a newer modified version which contains the new language string.

  2. Repeat again step 8 and then steps 7 to 10. In addition, inspect the markup of versions and in case of the "Other Information" tab of the database checker also the 2 table names in the version information.

Result: See section "Expected result AFTER applying this Pull Request" below.

  1. Go to "Extensions -> Manage - Install Languages" and inspect the markup of versions with developer tools of your browser. Check with both LTR and RTL language.

Result: See section "Expected result AFTER applying this Pull Request" below.

Actual result BEFORE applying this Pull Request

"Live Update" tab of the Joomla Update component if no update has been found

LTR
j3-joomla-update-live-update-no-update_ltr

RTL
j3-joomla-update-live-update-no-update_rtl-bad

"Pre-Update Check" tab of the Joomla Update component

LTR
j3-joomla-update-pre-update-check-extensions_ltr

RTL - The messed position of the "More Detail" toggle is another issue to be solved in another PR
j3-joomla-update-pre-update-check-extensions_rtl-bad

"Live Update" tab of the Joomla Update component if an update has been found

LTR
j3-joomla-update-live-update_ltr

RTL - here the version numbers are already correct
j3-joomla-update-live-update_rtl-ok

Languages: Installed

LTR
j3-languages-installed_ltr

RTL
j3-languages-installed_rtl-bad

Extensions: Update

LTR
j3-extensions-update_ltr

RTL
j3-extensions-update_rtl-bad

Extensions: Manage

LTR
j3-extensions-manage_ltr

RTL
j3-extensions-manage_rtl-bad

Extensions: Database - Tab "X Database Problems Found"

LTR
j3-database-check-errors-found_ltr

RTL
j3-database-check-errors-found_rtl-bad

Extensions: Database - Tab "Other information"

LTR
j3-database-check-other-information_ltr

RTL
j3-database-check-other-information_rtl-bad

Extensions: Discover

LTR
j3-extensions-discover_ltr

RTL
j3-extensions-discover_rtl-bad

Expected result AFTER applying this Pull Request

For LTR there should be no changes compared to before applying this PR.

For RTL the versions should look the same as for LTR. See the images below.

When the language has RTL direction, versions are wrapped into a span element with dir="ltr" attribute.

"Live Update" tab of the Joomla Update component if no update has been found

j3-joomla-update-live-update-no-update_rtl-ok

"Pre-Update Check" tab of the Joomla Update component

j3-joomla-update-pre-update-check-extensions_rtl-ok

The messed position of the "More Detail" toggle is another issue to be solved in another PR.

"Live Update" tab of the Joomla Update component

There should be no change, both LTR and RTL should work as well as without this PR for this view.

Languages: Installed

j3-languages-installed_rtl-ok

Extensions: Update

j3-extensions-update_rtl-ok

Extensions: Manage

j3-extensions-manage_rtl-ok

Extensions: Database - Tab "X Database Problems Found"

j3-database-check-errors-found_rtl-ok

Extensions: Database - Tab "Other information"

j3-database-check-other-information_rtl-ok

Extensions: Discover

j3-extensions-discover_rtl-ok

Documentation Changes Required

None.

avatar richard67 richard67 - open - 25 Oct 2021
avatar richard67 richard67 - change - 25 Oct 2021
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 25 Oct 2021
Category Administration com_installer com_joomlaupdate com_languages Language & Strings Libraries JavaScript
avatar richard67 richard67 - change - 25 Oct 2021
Labels Added: ? Language Change ?
avatar brianteeman
brianteeman - comment - 25 Oct 2021

Fix the overrides of the Hathor template, too.

No need to do that as we made a statement a long time ago regarding no hathor updates

avatar richard67
richard67 - comment - 25 Oct 2021

Fix the overrides of the Hathor template, too.

No need to do that as we made a statement a long time ago regarding no hathor updates

@brianteeman Should I keep the old language sting for Hathor at least so we don't get an untranslated string?

avatar brianteeman
brianteeman - comment - 25 Oct 2021

Is it ok that I remove the old language key from the "administrator/language/en-GB/en-GB.com_installer.ini" file?
I don't see any need to keep it as these strings are only used by the database checker.

This is J3 where the custom was not to remove strings

avatar richard67 richard67 - change - 25 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 25 Oct 2021
avatar richard67
richard67 - comment - 25 Oct 2021

Is it ok that I remove the old language key from the "administrator/language/en-GB/en-GB.com_installer.ini" file?
I don't see any need to keep it as these strings are only used by the database checker.

This is J3 where the custom was not to remove strings

@brianteeman Shall I add some deprecated comment then? Or just leave it as it is?

avatar richard67 richard67 - change - 25 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 25 Oct 2021
avatar richard67
richard67 - comment - 25 Oct 2021

@brianteeman Does it need to add some deprecation comment to the 2 unused, old language strings so we can remove them in J4? If so: Above the line like we did in past in J3, or at line end like we meanwhile do it do for J4?

avatar brianteeman
brianteeman - comment - 25 Oct 2021

technically yes.

avatar richard67 richard67 - change - 25 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 25 Oct 2021
avatar Fedik
Fedik - comment - 26 Oct 2021

@richard67 one more suggestion. To prevent placing all inside HTMLHelper, maybe better make HTMLlanguage helper.
Look for example one of existing helpers in /libraries/cms/html

Create libraries/cms/html/language.php with class JHtmlLanguage and with inlineBidirectional() method.
Then call it like JHtml::_('language.inlineBidirectional', ....).

avatar richard67
richard67 - comment - 26 Oct 2021

@Fedik Thanks for the suggestion. I will do that later today.

avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67
richard67 - comment - 26 Oct 2021

@Fedik Done. Do you know if I could and should add unit tests for the new helper method?

avatar richard67 richard67 - change - 26 Oct 2021
Title
[3.10] [RFC] [WiP] Fix display of LTR strings like e.g. version number or table names in RTL languages
[3.10] [RFC] [WiP] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
avatar richard67 richard67 - edited - 26 Oct 2021
avatar Fedik
Fedik - comment - 26 Oct 2021

This I do not know

avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar joomla-cms-bot joomla-cms-bot - change - 26 Oct 2021
Category Administration com_installer com_joomlaupdate com_languages Language & Strings Libraries JavaScript Administration com_installer com_joomlaupdate com_languages Language & Strings Libraries JavaScript Unit Tests
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67
richard67 - comment - 26 Oct 2021

Found it out myself. Unit tests added. Now I only have to complete a few things in testing instructions, then it will be ready.

avatar brianteeman
brianteeman - comment - 26 Oct 2021

זה מבחן

avatar richard67 richard67 - change - 26 Oct 2021
Labels Added: ?
avatar richard67
richard67 - comment - 26 Oct 2021

I think you should be using an actual rtl string when testing rtl

@brianteeman Done.

avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
Title
[3.10] [RFC] [WiP] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
[3.10] [RFC] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
Title
[3.10] [RFC] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
[3.10] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
Title
[3.10] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
[3.10] [RFC] Fix display of LTR strings like e.g. version numbers or table names in RTL languages
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67
richard67 - comment - 26 Oct 2021

Ready for testing.

avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67 richard67 - change - 26 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 26 Oct 2021
avatar richard67
richard67 - comment - 28 Oct 2021

I've noticed there were a few things missing which I've added with the last commits. I have to update screenshots and testing instructions.

avatar richard67 richard67 - change - 28 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 28 Oct 2021
avatar richard67 richard67 - change - 28 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 28 Oct 2021
avatar richard67 richard67 - change - 28 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 28 Oct 2021
avatar richard67 richard67 - change - 28 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 28 Oct 2021
avatar richard67 richard67 - change - 28 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 28 Oct 2021
avatar richard67
richard67 - comment - 28 Oct 2021

Done. Ready for testing again.

avatar richard67 richard67 - change - 28 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 28 Oct 2021
avatar infograf768 infograf768 - test_item - 29 Oct 2021 - Tested successfully
avatar infograf768
infograf768 - comment - 29 Oct 2021

I have tested this item successfully on 399751b


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/35902.

avatar richard67
richard67 - comment - 29 Oct 2021

Meanwhile I'm working on a J4 version of this PR because due to changes in J4 it will not be just an easy upmerge of this PR which will do the thing for J4.

avatar richard67
richard67 - comment - 29 Oct 2021

I just notice that in Joomla 4 the 2 new language strings added by this PR will not be needed (as well as the 2 old ones for which I've added the deprecation message here), because in J4 the database checker has been changed to work for 3rd party extensions too and not just for the CMS core.

I'll update the deprecation comments in this PR accordingly so it shows that right.

This doesn't invalidate the previous test result so I'll restore that in the tracker.

avatar richard67 richard67 - change - 29 Oct 2021
The description was changed
avatar richard67 richard67 - edited - 29 Oct 2021
avatar richard67 richard67 - alter_testresult - 29 Oct 2021 - infograf768: Tested successfully
avatar richard67
richard67 - comment - 30 Oct 2021

The last commit has only changed the function calls in unit tests from JHtml::_('language.inlineBidirectional', ...) to JHtmlLanguage::inlineBidirectional(...) to be consistent with the other unit tests for JHtml, so the previous human test result is still valid. I'll restore it in the issue tracker.

avatar richard67 richard67 - alter_testresult - 30 Oct 2021 - infograf768: Tested successfully
avatar khu5h1 khu5h1 - test_item - 8 Nov 2021 - Tested successfully
avatar richard67
richard67 - comment - 9 Nov 2021

@khu5h1 Have you tested this PR? I got a notification that you have, and the tracker shows you in the user tests, but there is not really a test result post. Have you used the "Test this" button like it should be done? Or have you used the "Alter test" button?

avatar khu5h1 khu5h1 - test_item - 9 Nov 2021 - Tested successfully
avatar khu5h1
khu5h1 - comment - 9 Nov 2021

I have tested this item successfully on cae0e2e

@richard67 I have tested this PR. Also, I tested the same yesterday. But I was trying to edit the tested successfully comment but by mistake, It got deleted.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/35902.
avatar richard67
richard67 - comment - 9 Nov 2021

@khu5h1 Thanks.

avatar richard67 richard67 - change - 9 Nov 2021
Status Pending Ready to Commit
avatar richard67
richard67 - comment - 9 Nov 2021

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/35902.

avatar richard67
richard67 - comment - 15 Nov 2021

To be honest I am not sure anymore if this PR implements the new thing in the right way. Maybe it's a bad idea to have a html helper method having a dependency on the document? Hoping for reviews and feedback.

Maybe I should remote RTC or change this PR back to draft or both so it's not merged by accident?

avatar richard67 richard67 - change - 21 Nov 2021
Status Ready to Commit Pending
avatar richard67
richard67 - comment - 21 Nov 2021

Back to pending for now because I think I have to change the way how it's implemented. Sorry to the testers for having wasted your time.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/35902.

avatar richard67
richard67 - comment - 21 Nov 2021

Will also move to draft status.

avatar brianteeman
brianteeman - comment - 21 Nov 2021

I strongly recommend that you abandon working on this for J3 and concentrate on j4 only.

For example I noticed today that some existing rtl mitigations in the j4 codebase are no longer required as they were only for old browsers

avatar zero-24
zero-24 - comment - 21 Nov 2021

Thats also what I have said. Focus on a 4.1 patch and when thats finally accepted we can check whether it makes sense to port it back to 3.10 and 4.0

avatar richard67
richard67 - comment - 21 Nov 2021

Yes, I only leave this open for documentation of the issues until I have time to make a PR for 4.1. Feel free to close it if it is disturbing.

avatar richard67
richard67 - comment - 28 Jan 2022

Closing as it won't be fixed in 3.10 anymore. I'll check sooner or later which of the issues are still there on 4.1-dev and then make PR's if necessary.

avatar richard67 richard67 - close - 28 Jan 2022
avatar richard67 richard67 - change - 28 Jan 2022
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2022-01-28 19:33:10
Closed_By richard67
avatar brianteeman
brianteeman - comment - 28 Jan 2022

@richard67 ping me before you start on it. Lets do it right and not the previous mish-mash

avatar richard67
richard67 - comment - 28 Jan 2022

Oh, that can take a while because it has a very low priority on my list. Highest priority have the updating issues. But I'll let you know when I have something, either an issue or a PR.

avatar richard67
richard67 - comment - 28 Jan 2022

In my next attempt I will not create a new helper function but do the wrap into a span with dir=ltr depending on the document's direction directly locally where it's needed and the right display of LTR version with numbers and letters cannot be achieved with other means.

Add a Comment

Login with GitHub to post a comment