@alex7r please add the version tag to the collection xml as this is an required field as explained in the docs. (https://docs.joomla.org/Deploying_an_Update_Server#Collection)

@zero-24 It do not help solving the issue, please note:

• I do not want to list specific "latest" version of the extension as I have different levels of stability;
• the issue is about unlocking functionality of parsing "extra_update_sites" from xml (which is supposed to be in place already).

I do not want to list specific "latest" version of the extension as I have different levels of stability;

Ok so you are requesting a new feature for the collection.xml right? The other way around would be an extension.xml update server there this should be possible already.

the issue is about unlocking functionality of parsing "extra_update_sites" from xml (which is supposed to be in place already).

I'm sorry i can't follow you here, can you please be more specific what is supposed to work how?

Quote from docs:

detailsurl – The URL of the XML file which contains that extension's individual update definitions

So no, there is no "feature request". I just want current functionality to be properly defined and tied instead of lose ends. (As I stated above: there are almost done possibilities (one described in docs, the other is buried in code)).

Please see lines (I'll provide them as a backtrace order):

• There is possibility to iterate through extra_sites
  https://github.com/joomla/joomla-cms/blob/staging/libraries/src/Updater/Updater.php#L291
• Which are returned by findUpdate
  https://github.com/joomla/joomla-cms/blob/staging/libraries/src/Updater/Adapter/CollectionAdapter.php#L262
• And populated here
  https://github.com/joomla/joomla-cms/blob/staging/libraries/src/Updater/Adapter/CollectionAdapter.php#L117

However there is a multiple bugs in:

• iteration through additional sites (actually they are just skipped anytime);
• additional sites treated as "collection" type [hardcoded], and we would want to allow "extension" type according to "docs teaser" by "detailsurl";
• actually docs do not say anything about "category", and described "detailsurl" functionality is not working at all.

Well please let me explain the current working behavior and than lets talk about the ref attribute you found.

the current working behavior of collection.xml

The collection.xml files are intended todo the following (see the docs)

The "collection" server type allows developers to define an extension's manifest to pull updates from a collection. This type of server can be used if the developer wants to define all of their extension's updates in a single file (not recommended) or if their extension has multiple sub-extensions which are not distributed or updated at the same time (such as a package extension type). The below example is the "collection" definition used by the updater when processing core Joomla! updates:

For that reason the following attributes are required to make this feature work (also from the docs):

All definitions must be defined between tags in your collection manifest. The tag has two optional parameters; name and description. For each extension that this collection references, a separate tag is required. The tag has the following parameters, all of which are required for updates to properly process:

name – The name of the extension
element – The untranslated extension name i.e. mod_custom
type – The extension type (component, module, plugin, etc.)
version – The latest version of the extension
detailsurl – The URL of the XML file which contains that extension's individual update definitions

How the feature (collection URL + details URL) work is the following:

1. extension dev creates an collection URL (example)
2. extension dev adds the collection URL with the parameter in the extension xml <server type="collection">https://example.com/list.xml</server>
3. Joomla uses that URL to check for updates
4. It first tries to match the highest version it can find respecting the optional targetplatform attribute etc.
5. Than your detailsurl comes into action and get asked about the details of that version.
6. the update is found and a message is showed.

Your questions

iteration through additional sites (actually they are just skipped anytime);

I was not aware of that feature at all, as the "REF" thing does not seam to be documented at all.

additional sites treated as "collection" type [hardcoded], and we would want to allow "extension" type according to "docs teaser" by "detailsurl";

detailsurl != ref in this case. The details URL allows extension xml files and is working good in that matter. For ref I don't know what the original intent was as it is not documented.

actually docs do not say anything about "category", [...]

Another thing that is not in the docs agree.

[...] and described "detailsurl" functionality is not working at all.

I can asure you that detailsurl is working as the Joomla Core use it for its updating but as mention in the doc it requires the version tag to be included in the collection file too to work correctly. The better point is what is this magic ref tag about.

The ref attribute

hmm it seams that this ref attribute was there already more than 11years ago and it seams that there is no more history about this? Maybe I 'm missing something?

Maybe @mbabker @wilsonge @infograf768 remember what this REF should have been used for, i have not got the idea of that tag yet? By the inline comments there it seams it was already expected to only get collection Update URLs and for that reason it was hardcoded to that.

Next Steps

For the next steps I would like request you to provide the following:

• More detail about your requirements
• More details about your current problems with stability_minimum.
• What are the issues in the current system you want to overcome with that way of using the collection file

Please keep out the current behavior of the ref attribute or extra sites etc. Please explain what you want to happen, why you need that behavior and what issue that would fix.

So we can take a look into a way fixing that issues that does not seam to be adressed by the current system yet.

Thanks :)

As with everything else in Joomla as it pertains to the code, your best bet is to reverse engineer the API to figure out the expected behaviors and not rely on documentation (as the documentation is either not written, written for a non-technical perspective, or partially written by someone who has only documented things they understand without having reverse engineered the code). This is why I have said in the past there should be feature specifications for the behaviors in core that can form the foundation of proper documentation.

I have nothing else to offer here.

As I've recently came across article describing Joomla community problems and issues of communication may be interpreted as rude or offensive would like to put a disclaimer here just in case:

If sentences seems like offensive - it is not. The overall motive is to be heard and to highlight problem with facts that are result of hours/days of debugging and looking for way around before coming to the repository with "issue" or "PR" as those proved itself to be long and not very friendly process in the past.

@zero-24 Feel like I need to ask to verify if something wrong on my side, or...
Did you test the code or just reading how it seems to be working?
Because not even one of my test resulted in detailsurl being fetched at least. (Just stored to DB)

@mbabker

to reverse engineer the API

was done before coming here, and everything described above is the results of this investigation (and not relying on docs, but highlighting where they do not match).

@zero-24 So let's start from the beginning with description of the problem:
Currently I use collection type update xml.
I would like to be able to provide different versions to my users based on their configured (in Joomla Updater component) stability level.
I can not do this in collection xml because it does not support this capability.
The extension xml type does support this.
Working this out I've came across detailsurl which should allow to list all extensions is collection xml and force extra fetch of extension xml for each item in the collection.

Additional comments:

• during all my tests detailsurl is never fetched (with version or without)
• there is ref in <category>
• ref sets collection type for extra_sites
• extra sites in ref are useless as they are never fetched due to https://github.com/joomla/joomla-cms/blob/staging/libraries/src/Updater/Updater.php#L297 and 'updatesiteid'=>$this->_updatesiteid

So as the result:

• docs are wrong
• code have a few loose ends that can be used in achieving my goal, but none of them does work

@zero-24 Feel like I need to ask to verify if something wrong on my side, or...
Did you test the code or just reading how it seems to be working?
Because not even one of my test resulted in detailsurl being fetched at least. (Just stored to DB)

Yes it absolutely works as intended and documented. I have just tested it with one of my own plugin that is using collection update server and you can test it yourself by updating the Joomla core that is using the same system.
To be sure i'm talking about the intended and documented behavior including the required fields etc.

I can not do this in collection xml because it does not support this capability.

Yes.

The extension xml type does support this.

Correct..

during all my tests detailsurl is never fetched (with version or without)

Well to be sure when you use collection.xml the detailsurl is only called in case there is an update and you choose to install it, that is part of the reason that the version and other infos are required in the collection.xml itself.

This is the codepath that call this URL: https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_installer/models/update.php#L347

This mention code path is called when you hit "install this update" for this extension. This code path also includes handling of the minimum stability tag.

docs are wrong

I don't agree on this one as they work exactly as described in the docs but i agree that collection does not support minimum stability yet.

code have a few loose ends

I agree :)

that can be used in achieving my goal, but none of them does work

Yes they don't work because they are not intended to be used in this way so they don't work that way :D

Conclusions

So basically we are back to this

I can not do this in collection xml because it does not support this capability.

I agree on this when using collection.xml Joomla does not support minimum stability yet in an consistent way.

The current problems

1. Problem one is that only the data in the collection.xml file are checked for the initial check the, detailsurl is ignored.
2. Problem two is based on the first one and is that it requires the collection to set an version tag. Without this the update will never show up, as mention in the docs, but this is a problem in the use case you are describing.
3. Problem three is that when installing the update the minimum stability is respected which might result into installing a different version than showed in the UI and set in the collection.xml

Possible solutions

Possible fix for the first and third problem

A possible solution would be to combine in that case (collection.xml) both adapters and also follow the detailsurl including checking for all other attributes.

The code would than do something like this:

• check collection.xml what version is relevant for me?
• ok found the version
• lets check the details whether we can acutally install that update.

At the time we install the update we should also check whether we would actually install a different version than mention in the collection.xml and block that. This should only happen in case someone does not correctly setup the collection and extension xml yet or between the inital check and the download limiting options have been changed like minimum stability or php version etc..

Possible fix for the second problem

This would be the new feature i mention as using a collection without version but with minimum stability etc that is not supported by Joomla yet.

But is collection that the best way to use? Lets look back where the collection update server is intended to be used for:

The "collection" server type allows developers to define an extension's manifest to pull updates from a collection. This type of server can be used if the developer wants to define all of their extension's updates in a single file (not recommended) or if their extension has multiple sub-extensions which are not distributed or updated at the same time (such as a package extension type).

Based on this we have two implemented use cases for the collection update server.

1. define all extension's in a single file (not recommended)
2. multiple sub-extensions which are not distributed or updated at the same time

And here starts the problem as the use case here is something completely different.

Be able to provide different versions based on the configured stability level.

As this is something completely different than the current collection.xml features I would suggest to not use them for this kind of problem as this would only make the collection adapter more complicated. But we might can implement a new adapter that can handle that.

The extension xml type does support this.

For that reason I would like to understand why you don't use the extension.xml and what is the usecase of using collection and extension.xml together in your case?

As in the end without version and the other meta infos in the collection the collection would be just a link list that provides different detailurls based on mimimum stability or php version and just says something like "You are using 3.x and your minimum stability is RC look at this details URL", "You are using 3.x and your minimum stability is Stable look at this details URL"

But I don't got yet what would be the benifit of such thing over the extension xml that does that today already and needs to be created anyway? Yes it would make that thing a bit more abstract and as you can use different xml files the individual xml files could be a bit shorter and more focused, but I'm not sure whether file size is a problem at all in this case.

Other things

I have nothing else to offer here.

Thanks @mbabker for your input, i fully agree on your assessment :) My question came from the hope you might know what this ref was intended to do in the first place.

Maybe @Hackwar knows what this ref is about? Or maybe where to look / who to ask?

Do we have the commit history from before the git times anywhere so we could look there who and why this got introduced? Or is this data lost at some time?

Do we have the commit history from before the git times anywhere so we could look there who and why this got introduced? Or is this data lost at some time?

It's all there, minus the release tags since nobody thought about mapping SVN tags to git tags. Start from https://github.com/joomla/joomla-cms/tree/1.7.3 and traverse backwards, then hope either the commit messages or any changelog entries semi-reflect the change that actually happened.

Ok that it what I have found too so we only have the inital commit here: 6a3da60#diff-bdfc86903b98d8e68b328ed19f8ce0f0R53

That already include that ref thing.

hmm can we get any more details from:
git-svn-id: http://joomlacode.org/svn/joomla/development/trunk@11194 6f6e1ebd-4c2b-0410-823f-f34bde69bce9
The mention page requires me to login or is this dead now? I had also no luck on finding something relevant in the issue tracker archive. Maybe I'm looking wrong?

That's the SVN commit number the git commit was created from, you aren't going to find anything more useful there. And unless you find a JoomlaCode issue number in all of that, you're not going to find anything on the tracker archive (and it looks like there is no other reference than that commit, so you're kind of out of luck for following links).

Maybe a search based on the date and the author ?

Maybe a search based on the date and the author ?

I had no luck with that too. :(

And unless you find a JoomlaCode issue number in all of that, you're not going to find anything on the tracker archive

Ok thanks for confirming :)

Closed as having Pull Request #25928

So the first and third issue can be fixed by #25928 the actual feature to use a collection like update server without version has not implemented yet. Re-opening for that reason.

@alex7r can you please get feedback on this issue so we can start working on a solution to this problem or would #25928 be ok to make your use case working?

@zero-24 your call on this one

Well my proposal and comments was not followed up. So I will close this issue here. Feel free to create a new one once feedback is aviable. If anything it would be a new feature to be implemented into J4.x