@sanus70 can you please send the required changes against 3.10-dev so we make sure the issues are fixed for the existing 3.10 installs and the changes we done there can after that merged up into 4.0-dev.

Is it possible to provide an example / dummy extension so this change can be tested? I would also suggest to add a test case with existing extesnsions that use this feature so we do not break them by fixing some other cases.

Heard back from Nic...

Hello Phil,

I think we should but not for the reason listed.

When having an extension with commercial and free versions it makes sense to have two separate update sources. The reason is that the packages will be different, therefore their checksums will be different.

The update XML file should contain the checksum of the package which will be downloaded. The package should be identical for all users. I suppose this guy’s using the obsolete ZIP file tainting plugin in Akeeba Release System or a similar method to modify each ZIP file. This is problematic for many reasons, it doesn’t actually help with policing illegal downloads and is just an overall bad idea. It made sense 10 years ago but definitely not anymore. However, there ARE use cases where it would make sense. For example, if someone is distributing connectors to a SaaS it would make sense to have a customised update for each client. For example, you could issue a patch update whenever their certificate used to connect with the service is updated and include it in their customised package, essentially implementing a zero reconfiguration update for them.

The Download ID makes sense to be sent at the manifest URL for a different reason: someone might choose to implement a “secret” update server which only lists extensions and their updates to authenticated users. This makes sense for custom extensions delivered to specific clients as opposed to mass distributed extensions. This was not a concern in the past since these extensions wouldn’t have an update server at all; the developer in charge of developing these extensions would do the updates or deliver the updates to the person installing them. Since Joomla 3.10 and later REQUIRE an update server to not complain about compatibility issues it makes sense to have an update server for bespoke extensions but not have it publicly listed. So the Download ID in the URL would make sense.

The downside is that this could essentially work as a cache buster for CDNs. So, it shouldn’t be used indiscriminately on all updates or by default. There should be an option in the XML manifest of the extension, probably a third attribute of the dlid key e.g. onfetch=“1” to turn it on and have it turned off by default. This way the majority of developers who have optimised their infrastructure for the way Joomla works don’t have to reinvent the wheel and those, like Christian, who want the Download ID in the update server URL can do so with a minimal change to their extension XML manifest.

You have my permission to include my full reply in a public discussion.

Best regards,

Nicholas K. Dionysopoulos
PHP software engineer
https://www.dionysopoulos.me

In general, it seems for me that there is just a logical error at the current implementation of automatic downloads.
There is no problem if the update is not protected by the extra_query key.
In case the key is used, the update will currently run with the warning "No checksum found on the update server.", since the installer looks for a given checksum. This warning is unprofessioal for end users, so a SHA should be given.
But if the request while updating (without key) differs from the request searching for updates (with key), the server sends different manifests (free version / pro version) and different SHAs.

Just my 2 cents.

By the way, I can not see any problems regarding CDN busting, since the key is provided for specific extensions only. The Joomla team increased the update site page for the J4 version to edit these key!
So there is also no need to introduce another atribute at the extension manifest.

By the way, I can not see any problems regarding CDN busting,

A query string appended to the end of an update site will cause CDN busting and mean the XML Update Site Response cannot be cached by the CDN.

This might not be an issue for minor extensions, but with extensions with 10000s of uses, this is a huge issue!

The Joomla team increased the update site page for the J4 version to edit these key!

Correct, but the download id, up to now, has only ever been appended to the non-cdn-cached zip file url.

But if the request while updating (without key) differs from the request searching for updates (with key), the server sends different manifests (free version / pro version) and different SHAs.

Free and Pro versions are two different products and therefore should have two different update sites, and two different XML's and two different SHA hashes. Problem solved.

Having checked some of the 18433 Update Site Urls I have access to, I can see that MOST developers have already worked around the whole download id process anyway. MOST developers are already hard coding the license/download id/key into the Update Site's XML url.

Some examples (with fake replaced keys)

https://www.jevents.net/updates/00003G647366D3-0000-0000-0000-4C0D94B32741-d000000000000QubmV0/pkg_rsvppro-update-3.4.48.xml

https://www.acyba.com/index.php?option=com_updateme&ctrl=updatexml&component=acymailing&version=5.10.18&level=Enterprise&li=00000000000000000000000000000000%3D

https://idealextensions.com/?extension=com_contactenhanced&pid=4&dm=0000000000000000&output=updatexml

http://updates.rockettheme.com/joomla16/updates.xml?site_id=0000000000000000&access_token=0000000000000000&ignore=update.xml

And from memory there was a reason the url had to end in xml and not some other extension... cant remember why now.

Free and Pro versions are two different products and therefore should have two different update sites, and two different XML's and two different SHA hashes. Problem solved.

Yes this is right.
Please explain me how this will work in you opinion for a paid download. You have to ensure that the downloader is allowed to do so.

Please explain me how this will work in you opinion for a paid download. You have to ensure that the downloader is allowed to do so.

The same way everyone does. Its got nothing to do with my opinion, its got everything to do with the way it was written to work.

The XML file contains the hash of the professional version of the product.

This zip file is the same file, to anyone who has paid to access it.

The download id is appended to the zip file url you specify in the XML to the zip file.

The thing I think you dont understand:
When someone tries to access the zip file without the download id, you present a 403 Forbidden, when a download id is provided, you intercept that, check it your end against your database of paying customers, and if ok = deliver the zip file as normal.

Full Working Example - Akeeba Backup Professional.

The update url is https://cdn.akeebabackup.com/updates/pkgakeebapro.xml which is CDN Hosted.

In that XML you can clearly see the b39e509f9c846a068a29ef5e65a62b53d82552b622827f7f3f85dbaaab9cf883 SHA hash for Akeeba Backup for Joomla! 8.0.10

You can also see the download url of

https://www.akeeba.com/download/akeeba-backup/8-0-10/pkg_akeeba-8-0-10-pro-zip.zip

However if you go to that url you will get a 403 Forbidden.

However if you go to that url with a ?dlid=a_valid_download_id appended to the zip file url like:

https://www.akeeba.com/download/akeeba-backup/8-0-10/pkg_akeeba-8-0-10-pro-zip.zip?dlid=0000000000000000

then you are given the ZIP file download.

Actually the URL published by Nic contains the &jcompat=my.zip appended too, this and the &ignore=update.xml type ending of the update urls on some extension developers update urls I believe is to overcome issues in older Joomla versions that expected the urls to be static with the correct extensions.

https://www.akeeba.com/download/akeeba-backup/8-0-10/pkg_akeeba-8-0-10-pro-zip.zip?dlid=0000000000000000&jcompat=my.zip

Equally the FREE version of Akeeba Backup for Joomla has a different XML Update site CDN hosted at https://cdn.akeebabackup.com/updates/pkgakeebacore.xml which contains the SHA hashes and zip files defined for the free version.

Thanks, got it!
So the evaluation of the appended key for the xml request is useless, since this has to point to the pro version by default.
So it does not makes sense to append the key for this request as long as Joomla will handle the update servers in this way.
Thanks for your patience.

For a better understanding, maybe the code should then be changed so the key will be only appended at the zip request.

So the evaluation of the appended key for the xml request is useless, since this has to point to the pro version by default.

Yes, the XML file you return from he URL you publish as your Update Site has to have the SHA hash of the zip file advertised in the XML. Both should relate to the same product - the pro version in your case.

The evaluation of the download id is done at the point you serve (or deny access to) the zip file contents, when Joomla attempts to download the zip file with or without a download id appended as ?dlid=0000 - you should evaluate that 0000 and decide if thats a valid key for your product and if so serve the zip content else provide a 403 response.

In this way, anyone visiting a Pro XML url like https://cdn.akeebabackup.com/updates/pkgakeebapro.xml still doesnt have access to the product and there is no chance of https://cdn.akeebabackup.com/updates/pkgakeebapro.xml?dlid=0000 ever being cached anywhere (cause google is king)

So I close this PR...
Thanks