@lavipr Looks good at a first look. But new schema update SQL scripts (name e.g. 4.0.0-2019-07-02.sql) below folder administrator/components/com_admin/sql/updates are missing for the database changes in joomla.sql. Will you add them later? Or do you need some help or advise with that?

@lavipr Looks good at a first look. But new schema update SQL scripts (name e.g. 4.0.0-2019-07-02.sql) below folder administrator/components/com_admin/sql/updates are missing for the database changes in joomla.sql. Will you add them later? Or do you need some help or advise with that?

@richard67 Thank you. Indeed I need help or advice. That would be very helpful. I wasn't aware of that.

@lavipr I can make a PR for the schema updates against your branch, which you then can merge so it goes into this PR. Is that ok for you?

@lavipr I can make a PR for the schema updates against your branch, which you then can merge so it goes into this PR. Is that ok for you?

@richard67 That would be great! Thank you.

This looks like a lot of work - thanks

I will try to find some time tomorrow to do a real test

@brianteeman Thank you for your code review and your test.
@SharkyKZ Thank you too.

@lavipr Before I make any schema update: Why did you use data type text for the master_date column? I would have expected date for MySQL and timestamp without time zone for postgresql if it really is a date. If you save there the string representation of a date, then it needs to check your code it that string representation will always be of same format, regardless of current language.

@lavipr In addition to my comment above about data type of master_date column, I see that you use string data type in your PHP functions for that master date value. And in PHP code you build SQL queries with master date comparisons in the where clauses. Such comparisons for smaller or larger (<, >) and so on work with string representations of dates and time only when format is always so that alphabetical order is always equal to numerical order, like it is e.g. for format YYYY-MM-DD hh:mm:ss with hh= 24 hour format.

@lavipr I strongly recommend to use date and time data types for dates and time in PHP and also in SQL and not string data types. Or do you have a really good reason why you did it differently?

@lavipr Before I make any schema update: Why did you use data type text for the master_date column? I would have expected date for MySQL and timestamp without time zone for postgresql if it really is a date. If you save there the string representation of a date, then it needs to check your code it that string representation will always be of same format, regardless of current language.

@richard67 You are right. I'll change it. Thanks for your explanation.

@lavipr Ok, I'll wait with the schema updates. Ping me here when ready.

We currently don't support uppercase letters in layouts (IDK if there's a reason for this). So administrator/components/com_contenthistory/tmpl/compare/compareMaster.php needs to be renamed.

Eager to test, but atm just a question: is it necessary to define the master/reference language in the language filter plugin, instead of com_associations Options?

I really like the idea of a master language and this whole PR is an awesome piece of code. Thanks for that!

But now the "but": I would really like to see this going further. Not one global master language but unlimited local master languages. Every item decides "automatically" their own master language (probably via parameter). Because I think on big news portals different languages could be the master language depending on the article (one original article is French, so French would be master here, other is English etc.).

So tl;tr: Don't stop here, but go deeper on item level for a more flexible approach!

Eager to test, but atm just a question: is it necessary to define the master/reference language in the language filter plugin, instead of com_associations Options?

@infograf768 The idea was that because the master language depends on the fact that item_associations is set, this can be also done there.

@lavipr I would prefer as it is anyway impossible to use com_associations if items associations is not set.

Not one global master language but unlimited local master languages

Beware as this is what blocked the gsoc project as it is extremely complex and confusing...

@lavipr Ok, I'll wait with the schema updates. Ping me here when ready.

Could you please have a look to my database changes: 5c321cb ? Thanks!

@lavipr Data type in database looks ok now. I will make the PR for the schema updates in a minute or two. But still in PHP you use string data type. How it should be done you can look up in other PHP code dealing with date/timestamp columns, like e.g. columns published_up in several db tables.

@lavipr Data type in database looks ok now. I will make the PR for the schema updates in a minute or two. But still in PHP you use string data type. How it should be done you can look up in other PHP code dealing with date/timestamp columns, like e.g. columns published_up in several db tables.

@richard67 I also added update sqls. I will look for dealing with date/timestamps. Thanks!

@lavipr Hmm, just right now I have finished the update sqls, see lavipr#6

@lavipr Hmm, just right now I have finished the update sqls, see lavipr#6

Ah I forgot to add the new files to git... So I merged your PR. Thank you very much.

Welcome. I always try to help when desired. Now you have a lot to do with this prepared statement stuff from Harald, but I am sure you will manage that, too.

thank you for this big enhancement..and if you need any help on prepared statements, don't hesitate to ping me

Ah, and I forgot the PHP changes for datetime data type. Still a bit work, but on a good way.

Thank you!

thank you for this big enhancement..and if you need any help on prepared statements, don't hesitate to ping me

@alikon In fact, I would need help with the prepared statements. I'm trying to use them, but I always get an error when using ParameterType::Integer: 'Undefined class constant 'Integer'. I don't understand what the problem is.
I did import use Joomla\Database\ParameterType;
And used it like this for example:
$query = $db->getQuery(true)
->select($db->quoteName('master_date'))
->from($db->quoteName('#__associations'))
->where($db->quoteName('id') . ' = :id')
->where($db->quoteName('context') . ' = :context')
->bind(':id', $id, ParameterType::Integer)
->bind(':context', $context);
$db->setQuery($query);

@lavipr You have to use ParameterType::INTEGER, i.e. the INTEGER in capital letters..
See an example PR for the change to prepared statements here: #25386.

@lavipr You have to use ParameterType::INTEGER, i.e. the INTEGER in capital letters..
See an example PR for the change to prepared statements here: #25386.

Oh my bad... I think I've seen enough code today :D Thanks.

@lavipr No, not your bad, Harald's bad ;-) He gave you the example in the wrong way.

TEST:
Lets take as example Articles > Categories
4 languages installed: en-GB, fr-FR, de-DE, fa-IR
I will use here the term "Language" but it means "Content Language"

Site default language is en-GB.
Global Master set to site language i.e. en-GB

Associations are set for this category:

The Category (en-gb) has never been modified therefore No Version history.

Now lets load com_associations.

This is what I get when the language filter is set to the site language-> English (en-GB) Master:

Clicking on the lang associations icon -but not yet on the "EN" icon- or on the item itself works as before this patch.
Editing can be done using the sidebyside for both target and reference and we can switch to any existing language.

We have an important difference though with former behavior as "EN" is displayed and clicking on it will obtain the same result as clicking on the original/reference/item (Target is empty until language selected).
This means the UI has changed and now displays all existing lang associations including the one which is filtered by.
I can live with this as the side by side is working fine.

Now lets switch the language in the filter, for example to French.
Now we only get in the Associations column both the "EN" and "FR". German and Persian are no more present.

The behavior is totally different.

1. Clicking on the Title of the item "Catégorie (fr-fr)" displays the usual side by side where the Select Target Language only presents the Global Master choice.

2. If we click on any of the lang icons (EN or FR), the side by side is no more presenting the selected item (here "Catégorie (fr-fr)" as reference, but as target and all existing languages are still presented as possible in the Select Target Language.

Now, let's modify the Category (en-gb) item to get a new version. Added some text in the description.

If the modification is done while being in an already displayed side by side, nothing special happens.
If it has been done in the side by side and Reference saved or in the original item in index.php?option=com_categories&view=categories&extension=com_content then we get

1. In the Categories Manager

When NOT filtering by language

We already see the issue as we get the same result as in com_ associations : only the master and language of the item which is not tagged to the Global Master language are displayed.

Ths is even more prominent when filtering by a language which is NOT the Global Master one:

Basically, at looking at the manager we have no idea if the item is already associated to the other languages used by the site. This should be corrected imho.

Also the icons behave differently in the sense that the EN (Global here) is displaying the usual sidebyside with the Global while the red icon is opening the new update side by side.

2. In com_associations
   Similar behavior depending on the settings of the Language filter field.

BUG
Modifying the Content Category set to the Global Master language provokes modifications for the articles belonging to this category but also for articles belonging to any associated category, by creating new version history for the articles.

We get

If we thereafter click on the red icon for an article not set to Global Master language, i.e.

We get, when we are lucky

or sometimes (This also happens in com_associations)

js error:
TypeError: b.querySelector(...) is null sidebysideupdate.min.js:1:884
which is sidebysideupdate.js:52:16 in the uncompressed js.

Looking at the modifed date of these articles: it is empty.

NOTE: Independant from this PR as it was also the case before: Checkin is broken for com_associations.

Hope this helps.

Thanks @infograf768 for testing! I'll answer you later.

There is a possibility that the js error be related to the multiple tests I did on the same item, thus introducing an unwanted cache pb.

I would like to try to explain my concept in more detail:

By introducing the Master language, a relationship is established between the Master and his children. This also means that there shouldn't be a direct relationship from child to child.
For the behaviour this will then consequently work: A Master Element is shown all its associated children. If a child is displayed in the list, only the relationship to his master is displayed, since there is no longer a direct relationship to the other child languages.

The purpose of using a master language is as follows: In a company, for example, articles are usually written in one language. And, by default, "translations" are created from that language. Until now, it was practically irrelevant which order one used. Let's take the three languages English, French, German. So it was possible to associate English->French->German with each other.
Now the difference with a master language: This should be associated with English->French and English->German. (so of course all three are still linked in the association table, but the individual relationships are determined by the master language (child<->master)).

For the representation and the opening of the sidebyside view this is done: In the best case, the master is displayed as a reference (which works from the association column if a master is present). It does not work in two situations: There may not be a master item in the associations: This will open the child ( named child because it is not in the master language) as a reference and the user is practically forced to associate it with a master element. And it doesn't work if you click directly on the item in the list (maybe the link should be changed?) and also there he can only see the master (like in the list views).

If you want to know all the languages being associated, you have to filter by the master language.
I think otherwise the master language has little sense, apart from the out of date or up-to-date information.

I hope I could make my concept a little more understandable.

Apart from that the bug needs of course to be fixed.

Thanks for the detailed explanation. All makes sense to mr

Data types still have to be fixed. Currently this PR uses strings for datetimes in all the PHP code.
I don't want to make pressure, I only want to mention it here again so other reviewers and potential testers take notice of that.

@lavipr
Thanks. Yep I had understood the concept. The UI is not obvious and we may want to clarify it in a way or another. Let's first solve the bugs indeed.
Note: I still get a weird behavior with the js. Works once and then breaks again (Firefox).

Note:
Using Version History is essential here. Therefore modifying a Master menu item parameters is not considered as a modification for the associated menu items.

It is a special case: shall we/can we deal with it?
Maybe by comparing dates (needs adding a Modified date in the #_menus + in #_associations) and showing a message in the side by side? Although we would not know what was modified.
We could also totally ignore it as modifying menu items is reserved to users with specific access to com_menus and they should know the changes to do or not in the associated menu items .

No preconceived idea here.

Without wanting to be over-correct, perhaps we should not use the term "master"

https://en.wikipedia.org/wiki/Master/slave_(technology)#Terminology_concerns

I suggested in former occasions the term "Reference Language" as field label.
In classes and generally in methods and variables, it could be GlobalRefLang

Note:
Versioning should be enabled for each component for this to totally work as intended.

I disabled it for articles: the red icon is still displayed. Clicking on it: the changes are not highlighted as it is the sidebyside manager which is used (Not the Update one) [+ the same js error... grrr].
That is fine I guess.

Looks related to the date entered in the master_date column in the #_associations table.
Maybe we should use that aspect of the code for Menu Items by also entering a date in the master_date column? We would not get details of the changes, but at least a sign something has been modified.

Thanks for your input! I will start to improve the code step by step with regard to the comments and fix the bugs.

I would like to talk about the correct naming of the 'global master language' or the respective 'master' of an association. In order to avoid more confusion, the old term will be used here for the moment.

I'm not really happy about naming it as a global reference language and then consequently the master as a reference, because the term reference is already in use and for users this always means the left side of the edit view to which they are oriented.
In my concept the 'master item' should always appear as a reference, but there are cases where it is not possible. For example, there is no 'master item' for an association yet. Then an existing entry in another language will be the reference and a new target in the global master language will be created.

Therefore I would like to suggest for the global language and the item in the global language either 'global source language' or 'global original language' and 'source item' or 'original (item)'.

I would be pleased about your opinions.

Does "default language" and "parent" work?

Thanks!
Indeed "parent" would work, at least in the database. Would it be clear enough in the tooltip something like 'This iten is out of date with the parent'? I guess so.
I think 'default' might be too unspecific.
For example, in the list view the info 'Master' is displayed next to the set global master language, so that a user can immediately see which one is set. If there were now for example 'English Default', this could possibly mean anything.
Maybe an additional info, something like default association language? Where 'default' could also suggest that this is not a global setting?

What's the difference between global master language and default association language

Most of the time we have the same content for each language but it also happens that you need a different content per language (which doesn't mean that articles won't be associated - same subject but different content), as you may need to communicate differently to a French audience or to an American one. So it's not because the default/master language (I don't know the difference) is updated that it means the other language needs to be updated too.
Also, you don't necessarily update first the default language, but another one (it happens to my clients often: main language is set up to US but they update 1st the FR version).
What will happen in these cases?
Will this feature be optional?

@Sandra97, yes this feature is optional.

thx lavi for the master rewrite.

@infograf768

BUG
Modifying the Content Category set to the Global Master language provokes modifications for the articles belonging to this category but also for articles belonging to any associated category, by creating new version history for the articles.

I can't reproduce the error. I changed a category to the global master language (now the default association language), the children of this association change to 'out-of-date'. The articles belonging to this category do not change their association status.

js error:
TypeError: b.querySelector(...) is null sidebysideupdate.min.js:1:884
which is sidebysideupdate.js:52:16 in the uncompressed js.

The cause of the js error was the opening of a checked out item in the edit view. Meanwhile the check-in problem seems to have improved significantly. I can't reproduce this weird behaviour anymore (described at the end of #25403 (comment)).

I' d be very happy if you could test it again. :)

@lavipr Now when I proceed according to your testing instructions, i.e. new install, then install several languages, then install multilingual sample data and then install blog sample data, the installation of blog sample data stops after step 2 (menus), and I get many many PHP notices in my PHP error log, which are sequences of following 4 notices:

PHP Notice: Undefined property: Joomla\CMS\Table\Module::$element in /test-1/plugins/system/languagefilter/languagefilter.php on line 987
PHP Notice: Undefined property: Joomla\CMS\Table\Module::$enabled in /test-1/plugins/system/languagefilter/languagefilter.php on line 988
PHP Notice: Undefined property: stdClass::$item_associations in /test-1/plugins/system/languagefilter/languagefilter.php on line 989
PHP Notice: Undefined property: Joomla\CMS\Table\Module::$element in /test-1/plugins/system/languagefilter/languagefilter.php on line 1031

On a clean 4.0-dev without your PR this does not happen.

@SharkyKZ Could you advise regarding my previous comment about PHP notices? I will check, too, but you are better in PHP than I am.

Edit: For 3rd notice about undefined property: stdClass::$item_associations I have already the solution.

@richard67 Move context check before using $table here https://github.com/joomla/joomla-cms/pull/25403/files#diff-9a4ee24a09bcef172864692ed05135caR991

@SharkyKZ Before where? Your link shows the complete diff for all changed files, no certain line highlighted.

@SharkyKZ E.g. like that?

  // Get the parameters which are to be saved.
  $tableElement    = $table->element;

  if ($context != 'com_plugins.plugin' && $tableElement != 'languagefilter')
  {
  	return true;
  }

  $params          = json_decode($table->params);
  $pluginStatus    = $table->enabled;
  $itemAssocStatus = $params->item_associations ?? false;

@SharkyKZ or maybe even like that?

  if ($context !== 'com_plugins.plugin')
  {
  	return true;
  }

  // Get the parameters which are to be saved.
  $tableElement = $table->element;

  if ($tableElement !== 'languagefilter')
  {
  	return true;
  }

  $params          = json_decode($table->params);
  $pluginStatus    = $table->enabled;
  $itemAssocStatus = $params->item_associations ?? false;

@lavipr Do you already know how to fix the other PHP notices? If not: Shall I continue to help, or do you want to find out alone?

I' d be very happy if you could test it again. :)

@lavipr
I will take time to test thoroughly again during this next week.

@lavipr See lavipr#7 for fix of remaining notices which I've mentioned above.

@lavipr Now I get 2 times following 2 PHP notices when initially setting Set a Default Association Language? to Yes in the language filter plugin and saving the plugin:

PHP Notice: Only variables should be passed by reference in /plugins/system/languagefilter/languagefilter.php on line 1202
PHP Notice: Only variables should be passed by reference in /plugins/system/languagefilter/languagefilter.php on line 1238

@lavipr Now I get 2 times following 2 PHP notices when initially setting Set a Default Association Language? to Yes in the language filter plugin and saving the plugin:

PHP Notice: Only variables should be passed by reference in /plugins/system/languagefilter/languagefilter.php on line 1202
PHP Notice: Only variables should be passed by reference in /plugins/system/languagefilter/languagefilter.php on line 1238

thanks. I will fix that.

@lavipr Oh, I've already made a PR.

@lavipr Oh, I've already made a PR.

thank you!

@lavipr Now after having enabled the Default Assiciation Language in the plugin, I've modified an article in the master language, then went to the new side by side edit view and clicked the "Update Target and Close" button. The I got:

PHP error log shows nothing, so no idea where is the mistake.

Do you think you can find out yourself? If not, we could ping SharkyKZ, am sure he will find it.

@richard67 hm now I can't reproduce your error. Which browser are you using?

@lavipr Firefox 68.0.2 (64-Bit).

Weird. Can't reproduce the error here either.

But I notice the problem with the check-in again more frequently...

I click the icon marked with red frame in the following screnshot in the multilingual associations list view for the master article, then I get a js error TypeError: b.querySelector(...) is null in file sidebysideupdate.min.js:

Will switch on debug so the unminified js is used, then I can tell more.

When after that I save the target as described before, then I get that controller name thing.

sidebysideupdate.js:52:16:
target.querySelector('#jform_language').setAttribute('disabled', '');
It seems the query selector does not find what it looks for.

I click the icon marked with red frame in the following screnshot in the multilingual associations list view for the master article, then I get a js error TypeError: b.querySelector(...) is null in file sidebysideupdate.min.js:

Will switch on debug so the unminified js is used, then I can tell more.

Ok, yes that is the problem when the target item is checked out. Then the element cannot be loaded as a reference, so the js cannot find the usual elements because the actual element is not loaded.

Ah, so you can reproduce it?

But I am not sure if that js error is then later the reason for the controller name thing when saving the target.

No, I couldn't. Only once the unloaded element. But after saving the thing with the controller name didn't show up. I'm trying to get the problem again that the target can't open the checked out item. But it doesn't show up as often as it used to.

@lavipr Are you testing on Windows or on Linux? I have my Joomla installation on Linux. Maybe the controller name issue comes from some inconsistency in upper or lower case of some source file name and the corresponding class name, and maybe on Windows where file names are case-insensitive that problem does not happen?

@lavipr Are you testing on Windows or on Linux? I have my Joomla installation on Linux. Maybe the controller name issue comes from some inconsistency in upper or lower case of some source file name and the corresponding class name, and maybe on Windows where file names are case-insensitive that problem does not happen?

Indeed, I'm testing on Windows....

@SharkyKZ Any idea regarding #25403 (comment) and #25403 (comment)?

@richard67 Yes, that's the issue. Namespaced classes are case sensitive on case sensitive filesystems.

@SharkyKZ I don't find the mistake.

Controller name here needs to have correct casing.

https://github.com/joomla/joomla-cms/pull/25403/files#diff-c5890c27f645fcb19937caa120a99a4dR248
https://github.com/joomla/joomla-cms/pull/25403/files#diff-005714eb814c240817bbd105353cf0acR19

Ahh, I see. @lavipr Can you correct that?

Controller name here needs to have correct casing.

https://github.com/joomla/joomla-cms/pull/25403/files#diff-c5890c27f645fcb19937caa120a99a4dR248
https://github.com/joomla/joomla-cms/pull/25403/files#diff-005714eb814c240817bbd105353cf0acR19

@SharkyKZ Thanks a lot.

@SharkyKZ and @richard67 thanks!
I hope it works now.

I have to figure out how it works with checkin/out. As @infograf768 mentioned (#25403 (comment)), checkin is broken in associations.

@lavipr Now the controller name issue is solved. But it seems we have other issues still. I will continue to test tomorrow. Have a good night.

@lavipr It seems you also have to correct case-sensitive names of models, e.g. here:

https://github.com/joomla/joomla-cms/pull/25403/files#diff-842fb355510d470fa074b046ef1e190fR37.

For this example I get this error:

There might be other places where names of models have to be corrected.

Could you check and correct?

@lavipr Now correct, but then appears new PHP notice. Fixed in lavipr#9.

@lavipr Now everything works regarding starting conditions. But as soon as I've edited an article in master language (EN) and then click to ther icon e.g. for French I get following:

After I click close (without save), the english article is checked out.

There is no PHP error or warning or notice, only the JS error that the selector has not found any item mentioned in previous discussions.

Thanks for testing @richard67 . As I said, there is a problem with check-in and check-out. If you change the language before editing, you will see for this target that it is checked out. I need to figure out how it works with check-in/check-out, as that causes the problem.

Shall this be closed?

Shall this be closed?

I would like to see this finished and in Joomla, hopefully @lavipr finish this PR

This can be closed and picked up in another PR if someone is interested as topic starter is no longer available.

I am moving this to 4.1, we are in beta and this is a new feature and far from ready to merge.

Sorry that it took so long to respond. This has a big potential it would be a nice addition for Joomla. Unfortunately it never came out of the draft state and has currently a lot of conflicts, so I'm going to close this. If you are willing to drive it further, please fix the conflicts and rebase it to the 4.2-dev branch so we can help you get it into a testable state. Thanks for your help making Joomla better.

As mentioned, it's a feature which is useful and if possible should make it into the core. So to not lose it between thousands of closed PRs, it should stay as a draft for a future rework.

This pull request has automatically rebased to 4.2-dev.

This pull request has been automatically rebased to 5.1-dev.

Hi guys,
working on this would be good to pay attention at #19807