User tests: Successful: Unsuccessful:
[EDIT]
This PR now includes a first stab at introducing a server-assisted client-side JText.plural() implementation.
The initial reason of this PR, is also resolved in the same PR.
Makes Extension Updater display single and multiple update-availability-text correctly in Dashboard (Control Panel):
When a single update is found:
When multiple updates are found:
Note: This PR now has a proposed implementation of a javascript JText.plural method, and I'd like to read your thoughts on this proposal.
Implementation details to allow for client side plural implementation:
Thoughts & comments, welcome.
Code Review
Apply patch
Test the following:
The tests below should be tested each against none, a single and multiple updates for extensions.
For the tests I have included some zipped files here: testfiles.zip
This zip file includes 4 files
*localise.php
files, one for French and one for Russian.Note: To avoid any false errors, it is best to disable the browser cache.
Test pluralization with the new en-GB pluralizers. (files are ready)
Result: The new client side pluralization should show the correct pluralization.
Test in other languages where the localise.php files have not been patched.
Result: The new client side pluralization should still show results the old way (not correctly pluralized).
Test pluralization and fallback to en-GB for other languages. For this test please use one or both of the patched *localise.php
files. Please copy to the appropriate language directories in admin and site.
Result: The new client side pluralization should still show correct pluralization with a fallback to en-GB strings.
Note: If testing with the Russian language, note that en-GB does not have the _2
suffix, so there is no fallback and the language string will be presented.
Test pluralization with patched extensionupdater language files. I have patched them both to display [FR] with the equivalent English text and [RU*] where * is replaced with the suffixes generated. Please put those in the appropriate directory in administrator/languages.
Result: It should correctly display the pluralized strings for that language.
[EDIT] Yes, as for the language files *localise.php
and the description of the new functionality.
Status | New | ⇒ | Pending |
Category | ⇒ | Administration Language & Strings JavaScript Front End Plugins |
Wonderful! Thanks. Will adapt appropriately.
I thought that last night too (using JText::plural()
) but it's not that simple. The message is actually assembled and displayed in the JavaScript API as a success AJAX handler and that's the only part of the process that is aware of the item count. Also, Joomla.JText
in the JavaScript API doesn't have plural support.
OK , I haven't looked at the whole picture, yet, but I appreciate both your input. I'll read and try to find a good solution.
Also, Joomla.JText in the JavaScript API doesn't have plural support.
Would be nice to implement this.
@infograf768 I am coding something in that direction... stay tuned.
Labels |
Added:
?
?
|
Category | Administration Language & Strings JavaScript Front End Plugins | ⇒ | Administration com_contact Language & Strings Libraries JavaScript Front End Plugins |
@frankmayer please look at the conflicting file:
media/system/js/core-uncompressed.js
This PR now includes a first stab at introducing a server-assisted client-side JText.plural() implementation.
The initial reason of this PR, is also resolved in the same PR. More info in the edited PR-Description.
@jeckodevelopment OK, thanks, had seen this already ;)
@infograf768 took a stab at this.
@Bakual, @mbabker Since the back-end JText wouldn't cut it in this case, I implemented a server-assisted client-side solution.
Thoughts welcome.
Category | Administration Language & Strings JavaScript Front End Plugins com_contact Libraries | ⇒ | Administration Language & Strings Libraries JavaScript Front End Plugins |
Found a bug. Fix coming up
Bug fixed... Ready to get tested
Title |
|
Title |
|
Will test tomorrow, but I can already say that any change in the xx-XX.localise.php will NOT be B/C
any change in the xx-XX.localise.php will NOT be B/C
Explain please. The newly added method returns the same type of result as what exists already. The one potential problem would be using a 3.6 or earlier language pack on 3.7 and the method not existing which would cause an empty array to be returned and mess with pluralization when going through this new API. If it's a B/C break to add new methods then honestly it's time to shut down shop.
The one potential problem would be using a 3.6 or earlier language pack on 3.7
yes, that is why it looks like it will not be B/C... as we can't expect all existing language packs to be updated. Not only older ones which are not maintained, but also those which are maintained and will not be updated in time.
Imho, this should go into 4.0. Except evidently if you do not care about losing quite a few languages in 3.7
@infograf768 The problem will only arise for code which uses it, like this specific extension-updater code. The existing back-end pluralization will work without a problem.
A workaround could be to maintain the old client side implementation and remove it, after all language packs have been updated.
The issue isn't as much about B/C. The issue is more that this will not work for all languages.
It works for the "simple" ones like english and german.
Look at all the various plural forms you need to take care of:
http://www.unicode.org/cldr/charts/29/supplemental/language_plural_rules.html
Look for example at russian:
"one": 1, 21, 31, 41, 51, 61, 71, 81, 101, 1001, …
"few": 2~4, 22~24, 32~34, 42~44, 52~54, 62, 102, 1002, …
"many": 0, 5~19, 100, 1000, 10000, 100000, 1000000, …
"other": 0.0~1.5, 10.0, 100.0, 1000.0, 10000.0, 100000.0, 1000000.0, …
And that is even still one of the more straight forward ones
I think there is no way you can calculate that based on a simple array.
If anything, the language either has to provide a JS file which does the same as the current getPluralSuffixes method which can be included as well or we add some AJAX callback API which gets us the correct suffix for a given count and language. I think the latter would be perfect but not that simple probably.
I know, and I didn't say I'll do everything with a simple array. I said, it was not tested with more complex ones. The idea was to have a starting point for a possible solution.
However, you're right in that we need something to cover more complex situation.
For those cases like for your example, the russian language, the array could have an eval assignment for PHP and JS, like this:
return array(
array('=', 0, '0'),
array('eval',
array(
'php' => "array(($count%10==1 && $count%100!=11 ? '1' : ($count%10>=2 && $count%10<=4 && ($count%100<10 || $count%100>=20) ? '2' : 'MORE')));, '1')",
'js' => "the equivalent JS code"
)
),
);
I know eval is generally discouraged, but in this case it does not eval any unchecked user data and it provides a viable solution.
Also, if someone can put something malicious in the eval'ed string in this case, it can also be in the non-eval'ed string (the normal code).
To provide and maintain different files of more or less the same content for PHP and JS is in my opinion double work and a lot of testing and checking if strings are there, are the same, etc. It is not as straightforward as what I proposed.
The AJAX callback API is a thing we could maybe discuss.
However, why doing extra calls just for language strings?
There are already enough calls on a simple site, not to mention on more complex ones. And not all the things are possible to optimize (for example, merging JS and CSS), without problems.
Keep in mind that localise.php files has to be done by translators, not coders.
Imho, that gets to complex this way.
Yes, but the code for pluralization is already there, so some "translator" has already "coded" that, and the javascript part would also have to be written either way (except in your AJAX callback proposal).
This way it only has to be done once in the localise.php file and then there is a standard and easy way to access the strings via the JText.plural() API on the client side.
It just does not work when no change in the language pack.
Just tested with fr-FR pack and this PR
Warning: call_user_func() expects parameter 1 to be a valid callback, no array or string given in /Applications/MAMP/htdocs/testwindows/trunkgitnew/libraries/joomla/language/language.php on line 480
which is
return call_user_func($this->allPluralSuffixesCallback);
It works though when for example, I unpublish 2 articles:
When using any language, I get these errors in Control Panel
Notice: Undefined index: PLG_QUICKICON_EXTENSIONUPDATE_UPTODATE in /Applications/MAMP/htdocs/testwindows/trunkgitnew/libraries/joomla/language/text.php on line 315
Warning: in_array() expects parameter 2 to be array, null given in /Applications/MAMP/htdocs/testwindows/trunkgitnew/libraries/joomla/language/text.php on line 315
Notice: Undefined index: PLG_QUICKICON_EXTENSIONUPDATE_UPDATEFOUND_1 in /Applications/MAMP/htdocs/testwindows/trunkgitnew/libraries/joomla/language/text.php on line 308
Warning: in_array() expects parameter 2 to be array, null given in /Applications/MAMP/htdocs/testwindows/trunkgitnew/libraries/joomla/language/text.php on line 308
For Russian language I also get
Warning: Invalid argument supplied for foreach() in /Applications/MAMP/htdocs/testwindows/trunkgitnew/libraries/joomla/language/text.php on line 304
ru-RU.localise.php contains
public static function getPluralSuffixes($count)
{
if ($count == 0) {
$return = array('0');
} else {
$return = array(($count%10==1 && $count%100!=11 ? '1' : ($count%10>=2 && $count%10<=4 && ($count%100<10 || $count%100>=20) ? '2' : 'MORE')));
}
return $return;
}
We had for Scottish Gaelic
public static function getPluralSuffixes($count) {
if ($count == 0 || $count > 19) {
$return = array('0');
}
elseif($count == 1 || $count == 11) {
$return = array('1');
}
elseif($count == 2 || $count == 12) {
$return = array('2');
}
elseif(($count > 2 && $count < 12) || ($count > 12 && $count < 19)) {
$return = array('FEW');
}
return $return;
}
Yes, but the code for pluralization is already there, so some "translator" has already "coded" that,
Of course there is. And I guess they had support when they had to do it.
However the existing code is quite simple to understand what it should do. Take a number and return the matching suffix. In your proposal it gets a lot more complicate.
Btw, your code comments suggest that eval doesn't work, but I haven't looked exactly how the JS part works.
Another thought: Why do we even try to do the string on client side? We already know the count on the server side when we return the JSON string. We could just add the translated message to that JSON response imho. Then we can use the regular JText::plural method.
Because the string is generated server side before the AJAX call to get the count in the current structure. So it isn't aware of the count and you can't easily build a lookup with the current structure since you can have different pluralization forms in different languages, so you'd need all of the pluralized forms in the JS array to handle it correctly with minimal impact anywhere else.
Because the string is generated server side before the AJAX call to get the count in the current structure.
I get that, but what prevents us from changing it?
We don't have to have the string present in the JS code on pageload since we don't need it before the AJAX response. It could as well be part of the response from the AJAX call itself.
The AJAX response currently is an array with the possible update informations. It's a bit unfortunate that it is not put into a "subelement" of the response as it doesn't allow to add new things easily, but technically we don't even have to take care of B/C because it is not library code we're touching here. So we could put the update array into a subarray "updates" of the response array and the message as a separate array entry "message".
From what I see that method is only used by the two quickicons.
@Bakual @infograf768 Thanks for your comments, so far. Will look at the errors, and also think on this in the next days. @Bakual the eval() was a quick and dirty idea, not a real working implementation at the moment. Will analyze everything you guys threw at me
BTW I wish everybody a nice Christmas/Holiday time.
OK to test now. (BTW for some strange reason, I don't get any more update notifications at all in the Extensions - update screen of my test-installation. Even if I install really old extensions. )
Tested:
To be B/C (with 3.6.x French or Russian pack for example), this PR should NOT delete the existing strings:
-PLG_QUICKICON_EXTENSIONUPDATE_UPDATEFOUND="Updates are available! <span class='label label-important'>%s</span>"
-PLG_QUICKICON_EXTENSIONUPDATE_UPDATEFOUND_BUTTON="View Updates"
-PLG_QUICKICON_EXTENSIONUPDATE_UPDATEFOUND_MESSAGE="<span class='label label-important'>%s</span> Extension Update(s) are available:"
If this is merged, I suggest to add a comment before these strings something like
; The following three strings are deprecated and will be removed with 4.0.
as we do not have a 3.x Scottish pack for now, could not test with that one.
@infograf768 You're right, just forgot to re-include them after my changes. Will do that and your deprecation-comment suggestion.
Thanks.
Please do not add any empty line before the comment.
OK
Ready to test and review!
Personally I don't see the point of this. For the original issue, we can use what we already have in core, generate the plural string on server side and add it to the AJAX response. That specific method could be rewritten to use the JSON document type (&format=json
) anyway.
Adding an array with all possible plural strings to JS looks wrong to me. Using an eval
in more complex cases to create to those strings (both in PHP and JS!) looks even more wrong to me.
For the PHP part it's a step back for sure, it makes the code more complex and less understandable.
@Bakual the main reason is, being independent from Joomla, besides the main request.
Imagine a perfectly valid scenario, where some extension looks up data from on or more sources other than Joomla, using AJAX.
The nearest example to this PR, would be an implementation of the extension-updater, that checks for extension updates directly from the browser. (Which actually isn't a bad idea, as it would speed up things, due to parallel requests, and would always be up-to-date).
To answer the concerns you addressed:
If you use eval
then you doing something wrong. Or it must be a really edge case.
For JavaScript check the Function constructor . Do not use eval
.
The array with possible plural strings is only created for the specific strings needed. Not all strings. So this is not that big an issue. And the call is much nicer than the current one.
I'm of course aware of that, but for a single string you want to output you need to "translate" each possible plural form. That is what I meant.
The nearest example to this PR, would be an implementation of the extension-updater, that checks for extension updates directly from the browser. (Which actually isn't a bad idea, as it would speed up things, due to parallel requests, and would always be up-to-date).
Yeah, it would initially speed things up a bit due to parallel requests, and the next page refresh it will be much, much slower due to missing database "caching". Also looking at the current PHP logic to determine a valid update that would be a horrible idea to do in JS. So no, that is not a good example at all. And even then, I would personally prefer to short callback and let PHP calculate the resulting string, with doing that many requests to external sources, it wouldn't matter having one more (even a really small one). But that's me.
The PHP side really doesn't need any changing anyway. There is no issue with the current method and you never need all plural strings in PHP.
We only would need a JS equivalent, but not with using eval imho and without the multiple options to define the comparision array. Having a simple and an extended variant of the same method looks to complex to me. Just give the JS some code to process. Maybe even something that can be taken (maybe slightly adjusted) from http://www.unicode.org/cldr/charts/29/supplemental/language_plural_rules.html or another site.
As for this PR specifically, it doesn't really pose any security threat.
Eval always is a possible security threat. There almost certainly will be a case where some other security issue will allow to misuse the eval here.
Yeah, it would initially speed things up a bit due to parallel requests, and the next page refresh it will be much, much slower due to missing database "caching". Also looking at the current PHP logic to determine a valid update that would be a horrible idea to do in JS. So no, that is not a good example at all. And even then, I would personally prefer to short callback and let PHP calculate the resulting string, with doing that many requests to external sources, it wouldn't matter having one more (even a really small one). But that's me.
I did not look at the current PHP logic, and I didn't imagine it to be that complex. Also, it was only an example ;) - and while thinking about it, I kinda liked the idea. But I'll take a look at the PHP logic.
BTW, one could "cache" that info in the browser, so no calls at all would need to be made (This could also be done in the current implementation).
The PHP side really doesn't need any changing anyway. There is no issue with the current method and you never need all plural strings in PHP.
You're right, and I have already changed that. Need to do some more changes and tests to check out the new implementation, before pushing.
Eval always is a possible security threat. There almost certainly will be a case where some other security issue will allow to misuse the eval here.
Actually, for this PR (as I wrote) eval()
would no more be a threat than any code that can be put in the getPluralSuffixes()
method. But since it's not necessary anyways, it doesn't matter anymore.
...five months have past and this nice addition still sits around in the PR pool. The latest state of it, was that it was functioning fine, after all the reviewers' comments where addressed.
Unless someone comes up with a better, more practical working solution to the problem at hand, while at the same time being flexible enough (see my comment here: #13217 (comment)), I would propose the community to re-evaluate this one.
Milestone |
Added: |
Milestone |
Added: |
Milestone |
Removed: |
Milestone |
Removed: |
Category | Administration Language & Strings JavaScript Front End Plugins Libraries | ⇒ | Administration Front End JavaScript Libraries Plugins |
and this nice addition still sits around in the PR pool
@frankmayer i understand your comment. But there are not 2 successfully Tests to get a change for merge.
I am closing this as it is aimed at J3.
Any new feature should be proposed for J4.
Thanks for your efforts.
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2019-07-19 08:10:56 |
Closed_By | ⇒ | infograf768 |
Category | Administration JavaScript Front End Plugins Libraries | ⇒ | Administration Language & Strings Libraries JavaScript Front End Plugins |
shouldn't it be re-evaluated for J4 than be closed?
It can't be re-evaluated as the code is specific to J3 and in any case can't even be tested in j3 because dating back to 2017.
If anyone interested to pursue, including evidently @frankmayer , a totally new proposal has to be proposed for J4.
shouldn't it be re-evaluated for J4 than be closed?
shouldn't it be re-evaluated for J4 than be closed?
If anyone wants to take a stab at this for J4 please feel free to build upon my efforts. Unfortunately, atm, I do not have the time to pursue this. But it would be nice to see those efforts bearing good fruits.
If you want to have proper plural forms for that text, you need to use JText::plural(). Your PR will only work properly for languages that work like english where there is a form for "ONE" and another one for "MORE". Some languages however have a special form eg for 2, 12, 22. This is taken care by JText::plural().
See https://docs.joomla.org/International_Enhancements_for_Version_1.6