This pull request requires also #16493

After whole code will be migrated, then this approach should be changed, that each extension will be checked against J! upgrade compatibility with a single AJAX request, to make sure that we will not end with a timeout when checking multiple extensions.

If I understand this right, we had something similar in J2.5 to show if an extension will be compatible with J3: #5015

I'm not getting the point of that feature here. Is'nt that what the update server should take care of. So you can propose a different update definition of a different joomla version. So no need for extra coding?

It is possible that it is a similar code to PR #5015. The difference is that it is displayed in Joomla Update component and now updates are checked against the update server and I do not know how it was done in PR #5015.

This feature is a step to make a view in Joomla Update component to show the compatibility of 3rd party extensions against Joomla version which you are going to update to or upgrade.

This is a task which a got from @wilsonge on JAB17.

I do not know how it was done in PR #5015.

In the 2.5 extension manager, it has shown in a tooltip next to the version information if there is an update available for the same extension in J3. The information was stored in the "system_data" column of the #__extension table.

@piotrmocko can you give Example for each of the three Extensions neede for Test?

I have a package containing of a component, a module and a plugin that work from 3.2 and up. In the package I also have a fields plugin that is installed as part of the package but only works on 3.7 and higher (it is not required for the correct working of the component)

My update server has the following: 3.[234567] That is for the whole package.

in the fields plugin I have the following entry:

What I understand is that this information (version="xyz" would be used (future) to do just what you propose: not install that plugin / component / module on joomla sites that have a lower version number.
Currently that is not implemented and no matter what you fill in to version in the plugin's manifest file, it will install on all versions of Joomla.

So just my two cents: if you want to be more in control of when to install what you probably need a combination of checking the update server AND the manifest version information.

@franz-wohlkoenig I will give you tomorrow some extensions to test.

@Ruud68 this feature is to give a Joomla administrator information if his extensions will be compatible with the version of Joomla which he want to install.

@wilsonge did JAB17 team manage to migrate the code? I can help you with that if there is anything more to do.

@franz-wohlkoenig I have updated the description how to test this PR

@piotrmocko so there are 2 Test on 3.7.3-dev (with and -out Update-Server) and one on 3.8.0.

You can test it even on Joomla 3.5 if the code did not change to much. Those arguments which I have suggested are only telling the method to which Joomla you would like to update. I advise to test it on Joomla 3.7. But remember that there is also required PR #16493

For Tester: #16493 is merged, test on staging or latest nightly Build.

@wilsonge What is correct now to check for core extensions? Item ID 9999 like done in this PR here? Or 999 like done in PR #16474 ?
I ask you because this PR here seems to be based on your input.

See https://github.com/joomla/joomla-cms/blob/staging/installation/sql/mysql/joomla.sql#L481
First extension ID installed regulary is 10000

Sidenote: that only works if the core extensions are installed using the Joomla Update component. If it's done using extension discovery after an unsupported manual upgrade, then core extensions have IDs higher than 10000.

Hmm, @Bakual ' s sidenode above makes me think it is not really save to use the ID number to check if an extension is core or not. But what else way do we have beside a hard-coded list which would have to be maintained every time an extension is added?

to check if an extension is core or not.

No it is not ;) I guess we need to hardcode a array (the best would be a global place) with the core extensions or implement a database flag core vs 3rd party.

@zero-24 Well, I agree. But that should be not scope of this PR, I think. Which method (hardcoded array or db column in the extensions table being set by core installation sql scripts) should be discussed in PLT, I think, because it is an important decision. Should I make a "discussion" issue for that (or RFC or however to call it) in the issue tracker?

@zero-24 Meanwhile I think the db column thing would be easier to maintain because no edit of list necessary, only have to set the right value of that column in the sql scripts for installation. Disadvantage: If some bad 3rd party extensions developer sets that value, too, in his/her sql script, it will cause trouble.

@richard67 I think we can open a RFC (no code required yet) issue describing the problem and the solution and I can pass that to the maintainer chat on glip if needed.

But i think this here and the other one needs to be on hold until we have a clean solution.

Can you do the RFC Issue? @richard67

Yes. Am on the way.

@zero-24 @Bakual I created the RFC issue #16579 to discuss the core or not core determination thing.

Thanks

@zero-24 I have already applied changes according to your review

@piotrmocko applied PR and Plugin installed got:

So i didn't get a Screen like in your first Comment.

My PR is only a part of what you see on the screen shot.

The remaining part of code should have been made by other volunteers on JAB17
and @wilsonge was leading that team so he knows what is the progress.

@piotrmocko can you please update your Testing Instructions with this Informations so willing Testers know that ist not to Test now?

Regarding the question on how to detect if an extension is core or not which I discussed in my comments above, this PR here should not wait for a solution.

Let it continue to use the 9999.

Any later PR implementing a safe check will also have to look for such things and replace them by using the new way which it implements.

@piotrmocko I have created a PR against your branch which replaces the unsafe test for extension_id > 9999 to exclude core extensions by using the new JExtensionHelper class which has been merged in staging today:

piotrmocko#1

I have not tested my change yet. I will do soon and let you know about the result.

Then you can accept it, which means you merge it into your branch patch-29.

@richard67 I have merged your PR into mine

Good. I will test yours tonight when back from work. Do you have a code snippet we can use for test?

I have updated the test instruction with some code.

I have to test tomorrow or on weekend, today I am too tired now.

@piotrmocko Sorry, just tested and found a silly mistake which I have made in my previous PR for you. I have forgotten to change variable name at some place. You will see that quickly when checking my new PR piotrmocko#2 for you.

I tested meanwhile with the correction of my silly mistake (see my previous post). The test code produces now following warning:

getExtensions('3.7.2') Warning: Creating default object from empty value in /path_to_joomla/libraries/joomla/updater/update.php on line 267.

I see it 2 times, one immediately after the other at the beginning of the test output.

I think this comes now from your original code and not my corrections.

I will later try to find out more about that, but maybe you can check, too.

The warning comes from calls "$update->loadFromXML($location);" with $location = "https://update.joomla.org/language/translationlist_3.xml".

Can it be that language packs in general need a special treatment for getting the update xml location from the translationlist xml?
The exclusion of core extensions from the list only excludes the core language pack (EN-GB) but not additional languages being installed on a multilingual site (which I used for testing).

@richard67 Lang packs update server is addef tu package but returns collecdtion type xml infirst step and extesion type in second step.

@wojsmol That's what I meant. So it seems @piotrmocko has to add extra handling for language packs to do the second step. As before I can try to help because I am interested in that feature, but I do not have much time.

Maybe the simple solution for the problem I found is to exclude language packs from the check, like it is already done with core extensions?

This could make sense because language packs are always coming up with new versions after the new Joomla version has been released, they do not prepare in advance versions compatible with future Jooma versions like other extensions do.

@piotrmocko Has @wilsonge said anything about that when giving you this task? Shall language packs be checked or not? For the reason I mentioned above they should not be checked in my opinion. Do you agree?

@piotrmocko If they shall not be checked, simply add "->where($db->qn('type') . ' <> ' . $db->q(language))" after the "->where($db->qn('protected') . ' = 0')".

But you should also merge my PR with the correction of my silly copy and paste mistake, otherwise nothing works and you get an SQL error.

The more I think about it, the more I think that language packs should be excluded from the check, like core extensions already are. I make another PR to @piotrmocko 's branch to help with that.

@piotrmocko Let me know if my PRs are not welcome. I do not want to takeover your PR, it is still your PR and you did most of the work. I only want to help with making it work because I am interested in the feature it helps to implement, and so I try to help as much as I can, but if your task was to do it without any help let me know and I will stop.

Hmm, I just see language packs are not so easy to exclude, because they have also a package (type = 'package') extension.

@piotrmocko I've made a new PR to your branch, which excludes language extensions and their parent package from the check and includes also my previous correction. Please check piotrmocko#3. I have already tested your PR with my corrections, and it seems to be ok.

I have tested this item ✅ successfully on 2300d97

Details about test result and how I tested I will post in following comment.
@piotrmocko Please modify your testing insctructions and the description of your PR so it is clear that core extensions and (core and 3rd party) language extensions and language packs are excluded from the check for compatibility.

@richard67 done

I inserted following code snippet between lines "$model = $this->getModel();" and "$this->loadHelper('select');" in file "administrator/components/com_joomlaupdate/views/default/view.html.php" for testing:

echo "getExtensions('3.7.4')<br /><br />";
$testExtensionsLists = $model->getExtensions('3.7.4');
echo "Compatible:<br />";
foreach ($testExtensionsLists['compatible'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br />  Not compatible:<br />";
foreach ($testExtensionsLists['not_compatible'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br />NA:<br />";
foreach ($testExtensionsLists['na'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br /><br />getExtensions('3.8.0')<br /><br />";
$testExtensionsLists = $model->getExtensions('3.8.0');
echo "Compatible:<br />";
foreach ($testExtensionsLists['compatible'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br />  Not compatible:<br />";
foreach ($testExtensionsLists['not_compatible'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br />NA:<br />";
foreach ($testExtensionsLists['na'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br /><br />getExtensions('4.0.0')<br /><br />";
$testExtensionsLists = $model->getExtensions('4.0.0');
echo "Compatible:<br />";
foreach ($testExtensionsLists['compatible'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br />  Not compatible:<br />";
foreach ($testExtensionsLists['not_compatible'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}
echo "<br />NA:<br />";
foreach ($testExtensionsLists['na'] as $testExtension)
{
echo $testExtension->extension_id . ' ' . $testExtension->type . ' ' . $testExtension->name . '<br />';
}

The result was:

getExtensions('3.7.4')

Compatible:
10055 plugin plg_installer_webinstaller
10060 component COM_OSMAP
10062 plugin plg_system_ossystem
10109 library Alledia Framework (Don't uninstall this if you're using an Alledia/Joomlashack extension)
10110 component com_patchtester
10111 plugin PLG_PWEB_FBARTICLEIMAGES

Not compatible:

NA:
10044 template protostar-clone
10045 component com_phocaguestbook
10079 plugin PLG_OSMAP_JOOMLA
10086 file Komponente Phoca Guestbook - Deutsche Sprachdateien


getExtensions('3.8.0')

Compatible:
10055 plugin plg_installer_webinstaller
10060 component COM_OSMAP
10062 plugin plg_system_ossystem
10109 library Alledia Framework (Don't uninstall this if you're using an Alledia/Joomlashack extension)
10110 component com_patchtester

Not compatible:
10111 plugin PLG_PWEB_FBARTICLEIMAGES

NA:
10044 template protostar-clone
10045 component com_phocaguestbook
10079 plugin PLG_OSMAP_JOOMLA
10086 file Komponente Phoca Guestbook - Deutsche Sprachdateien


getExtensions('4.0.0')

Compatible:
10109 library Alledia Framework (Don't uninstall this if you're using an Alledia/Joomlashack extension)

Not compatible:
10055 plugin plg_installer_webinstaller
10060 component COM_OSMAP
10062 plugin plg_system_ossystem
10110 component com_patchtester
10111 plugin PLG_PWEB_FBARTICLEIMAGES

NA:
10044 template protostar-clone
10045 component com_phocaguestbook
10079 plugin PLG_OSMAP_JOOMLA
10086 file Komponente Phoca Guestbook - Deutsche Sprachdateien

Why the plugin PLG_OSMAP_JOOMLA is listed among the not available ("NA") is not clear to me yet. But it does not have a parent package so that cannot be the reason.

So or so the result is correct for me. The PLG_OSMAP_JOOMLA in my test is updated with the component, and it has no parent package, so it is ok that it is "NA".

More testers please.

@richard67 You assumption from #16494 (comment) is ok in staging if this is 3.7.4 but in this mey be not true in 3.8-dev or 3.9-dev where language packs may not require changes as these branches are more focused on gaining stability and preparing for as easy as possible update to 4.0.

@wojsmol I don't understand. You mean with 3.8-dev and 3.9-dev language packs will provide updates for future versions? I do not see that, so I still think it is OK to exclude them.

This cen be true, but for details I request comment from @mbabker or @infograf768

I have not tested this item.

Changing back to not tested until it is clarified if language packs shall be excluded or not.

@richard67 I try contacting @infograf768 on Skype. Weiting on invitation aproval.

@wojsmol Have you contacted @infograf768 ? What did he say? Does the functionality introduced with this PR here have to check language packs, too? And for which use case?

@wilsonge What did you have in mind regarding language packs when this task was set up at JAB?

I ask because if language packs shall be checked, the recent change in this PR for filtering them out has to be reverted (without reverting the corrections of my own mistakes when I helped here a bit).

@richard67 No invitation aproval soo far.

@richard67 JM is repleaing on github so give him more time.

@piotrmocko @richard67 I discuss language packs matter with @infograf768 on skype. Summing up the discussion, special treatment for language packages will be required. Let's wait for his comments after reading this PR thoroughly. Currently he has internet problems and can not access joomla.org and github.com.

Folks, I think indeed that core language packs (and possibly other specific extension language packs like weblinks) must keep their own update way. As you know, we have a cron job to prepare the update xmls automatically.

The reason is simple.

In any specific MAJOR J version (3.x, 4.x) some language packs may not be updated to last J version (because the Translation Team is late or has disapeared and is not replaced).

But, although such lang packs are not up to date, they are still used.
Example for 3.x : Albanian, Armenian, Dari, Hebrew, Uyghur, for the ones which are quite old.

Most of the time, users are specially concerned by the frontend and it is the frontend part of our packs which changes the less.
When they really need something absent or wrong used in the frontend, they create an override.

afaik, we will have a new repo for 4.x lang packs with totally new packs, but the situation may not change regarding that aspect.

@infograf768 Do I understand you right that this PR here is ok like it is, i.e. not showing core extensions and also not showing any language packs and their extensions (regardless if core or not)?

As you and @wojsmol hopefully understood, this PR here is for showing compatibility with a selected version, which will be the update offered for the core. E.g. if you are on 3.7.3 and now will be offered the update to 3.7.4, you will be able to see (when the complete functionality which is descibed in the description of this PR is ready) in the Joomla Update component which extensions are compatible to this 3.7.4 and which are not and which we don't know about.

So from my point of view it indeed does not make sense to handle language packs here - but @wojsmol did not want to believe me.

@infograf768 Please confirm if I did understand you right.

don't know if pr is ok. would need precise test instructions while using staging, but in any case, users should be able to decide if they want to update or not depending on the infos they would get through this patch.

I have tested this item ✅ successfully on 2300d97

Alter test result again after previous discussion.
My successful test is sill valid.
Why jenkins and appveyor tests fail I don't know.
But for new PRs, jenkins is not used, so maybe it is related to that and not to this PR.

I leave language packs up for JM. Basically the idea here is to reproduce the interface https://github.com/nicksavov/joomla-cms/tree/2.5.x-with-pre-upgrade-check which was about the ability to see what's outdated and what's not for a major version upgrade (but we'd also be displaying it for all upgrades, major, minor patches). The issues there was that it based itself on getting the compatibility info from the extensions manifest which was a bad idea because it meant releasing extensions + templates for every CMS release. If you get it from the update manifest you don't force people to release a new extension just to update the manifest file compatibility.

As to whether language files are displayed in the list or not I'm unsure. Many language files aren't updated every patch release, sometimes every minor and sometimes less frequently than that. And having an out of date language pack is often not an update blocker (because as JM rightly points out the frontend is normally not so affected).

I dont know why languages are even being discussed here. The PR is called Allow to check 3rd party extensions compatibility before J! upgrade #16494. Languages are not 3rd party, they are never ALL 100% up to date and they wont break an upgrade

That was exactly my opinion. It was @wojsmol who forced that discussion.

OK we're all good then :)

Oops I accidenlty merged it to staging, so reverted it back and applied to 3.9-dev with 5d919c3

Hello, we at Technische Hochschule Mittelhessen, Gießen, Germany are currently developing a PR to support all features shown in the Mockup-screenshot based on 3.9-dev branch.

We will send in a PR early next week.

@SamuelSchepp I am confused by your comment as this has already been merged into 3.9

This was just the model's required. Their going to work on the full interface that was originally envisaged by Nick Savov + Andrew Eddie

ah - thanks for the explanation