?
avatar mbabker
mbabker
9 Nov 2017

It should be possible to define multiple <downloadurl> tags for an element's <downloads> in an update server definition. This would allow possibilities such as download mirrors, especially in the case of certain countries blocking access to the resources a download originates from.

Votes

# of Users Experiencing Issue
13/13
Average Importance Score
5.00

avatar mbabker mbabker - open - 9 Nov 2017
avatar joomla-cms-bot joomla-cms-bot - change - 9 Nov 2017
Title
Support for multiple <downloadurl> tags in update stream
Support for multiple tags in update stream
Labels Added: ?
avatar joomla-cms-bot joomla-cms-bot - labeled - 9 Nov 2017
avatar b2z
b2z - comment - 9 Nov 2017

Thanks! That is the top issue for Russian community at the moment. Joomla update server is blocked on the territory of the Russian Federation by the decision of the state Agency (particularly s3-us-west-2.amazonaws.com).


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar effrit
effrit - comment - 9 Nov 2017

Thanks! This is really big problem: we cant access to update server from Russia so now we only can downloads updates and releases from gitHub manually. Its not what newcomers expect from modern CMS so we must do something to avoid security and reputation risks.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar Septdir
Septdir - comment - 9 Nov 2017

+1 I can't update sites hosted on Russian servers...

avatar zikkuratvk
zikkuratvk - comment - 9 Nov 2017

How will the old sites be updated? If this site is not available for automatic update now.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar pavluk
pavluk - comment - 9 Nov 2017

Recently switched to the Russian VPS the same problem sites are not updated.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar CB9TOIIIA
CB9TOIIIA - comment - 9 Nov 2017

Help Russian community :)


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar zikkuratvk
zikkuratvk - comment - 9 Nov 2017

Maybe it's better to allow the installation of an alternative server with localization?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar SergKom
SergKom - comment - 9 Nov 2017

This problem significantly hinders the use of the system. Not everyone has realized what happened. People who need it soon will be a lot. Thanks for the Jooomla-community support.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar dmitrystas
dmitrystas - comment - 9 Nov 2017

+1. Alternative mirrors are very important cos there's no access to the update server from Russian Federation now


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar starinajes
starinajes - comment - 9 Nov 2017

+1. Alternative mirrors from Russia


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar n-robert
n-robert - comment - 9 Nov 2017

I dont think multiple tags are necessary: in this case we would have to modify JUpdate to parse and store them, then we would have to add some check (which is bad) to find working URL to pass to JoomlaupdateModelDefault.
Instead we сould just add to tag multiple URLs separated by, for example, semicolon. Then we should modify JoomlaupdateModelDefault::download existing check for actual download URL.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar zero-24
zero-24 - comment - 9 Nov 2017

Please check this one out: #18545 it is not directly allowing multiple <downloadurl> tags as this is not possible without a B/C break but it proposes a fix by implementing a fallback URL tag.

avatar caprom
caprom - comment - 10 Nov 2017

I don't confirm the issue being from Russia. I have no problem with Joomla updates from a Russian IP, neither from Joomla admin panel, nor when directly downloading update packages from s3-us-west-2.amazonaws.com. If necessary I can provide all information for testing.

avatar franz-wohlkoenig franz-wohlkoenig - change - 10 Nov 2017
Category com_joomlaupdate
avatar franz-wohlkoenig franz-wohlkoenig - change - 10 Nov 2017
Status New Discussion
avatar joomla-cms-bot joomla-cms-bot - edited - 10 Nov 2017
avatar franz-wohlkoenig franz-wohlkoenig - change - 10 Nov 2017
Status Discussion Closed
Closed_Date 0000-00-00 00:00:00 2017-11-10 06:02:12
Closed_By franz-wohlkoenig
avatar joomla-cms-bot joomla-cms-bot - change - 10 Nov 2017
Closed_By franz-wohlkoenig joomla-cms-bot
avatar joomla-cms-bot joomla-cms-bot - close - 10 Nov 2017
avatar joomla-cms-bot
joomla-cms-bot - comment - 10 Nov 2017
avatar franz-wohlkoenig
franz-wohlkoenig - comment - 10 Nov 2017

closed as having Pull Request #18545

avatar mbabker mbabker - change - 10 Nov 2017
Title
Support for multiple tags in update stream
Support for multiple downloadurl tags in update stream
avatar mbabker mbabker - edited - 10 Nov 2017
avatar mbabker mbabker - change - 10 Nov 2017
Status Closed New
Closed_Date 2017-11-10 06:02:12
Closed_By joomla-cms-bot
avatar mbabker mbabker - reopen - 10 Nov 2017
avatar mbabker
mbabker - comment - 10 Nov 2017

Let's keep this open. The PR is at best a workaround, even if it means we have to make a B/C break in the API it should be possible to add more download URLs here without coming up with a new name for each tag.

avatar Septdir
Septdir - comment - 10 Nov 2017

Maybe like that #18547 ?

avatar franz-wohlkoenig franz-wohlkoenig - change - 11 Nov 2017
Status New Discussion
avatar n-robert
n-robert - comment - 11 Nov 2017

Or this #18553?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar alikon
alikon - comment - 11 Nov 2017

happy to see more than 1 PR coming out ?
hope this happen more often

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 11 Nov 2017

If all PR was coming out please don't forget to close the Issue.

avatar Septdir
Septdir - comment - 11 Nov 2017

@franz-wohlkoenig PR solves only part of the problem with the inability to download the update. It is also necessary that, in the most updated Joomla updates, alternative download links are available.

@alikon More than one PR is caused by the fact that the problem with updates in Russia is very acute, although I personally liked to contribute, even if my PR is not accepted.

avatar n-robert
n-robert - comment - 11 Nov 2017

Agreed with @Septdir: we can create a bunch of PR, but without alternative mirrors everything is useless.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar mbabker
mbabker - comment - 11 Nov 2017

Well, we're in a chicken and egg problem. Even if we wanted to use GitHub as a mirror (as right now it's the only other resource we have that can handle the traffic scale without another dedicated platform), we can't because core has no way to be told "if this location fails try another one". So this is at least helping with giving file mirroring capabilities, then internally we can work to ensure we have appropriate mirrors set up (maybe just having our downloads site and GitHub are enough, but both are on AWS S3 architecture so we're screwed if someone just flat out blocks the entirety of S3 instead of specific regions/buckets).

So it's a step forward and personally I'm glad to see multiple PRs for the idea because it gives us a chance to look at different ways to implement the approach instead of accepting the first idea that works.

avatar n-robert
n-robert - comment - 11 Nov 2017

@mbabker Thanx! Glad that you didn't remain indifferent to our problem.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar effrit
effrit - comment - 11 Nov 2017

@mbabker, i agree what GitHub is good mirror in current situation, but problem is time lag - new stable releases didn't appear here in same time as on main downloads site, so for us it will not solve the problem. Especially in terms of critical updates. Is it possible to put releases on GitHub on same time?

avatar Septdir
Septdir - comment - 11 Nov 2017

@effrit If we are talking about updating joomla, I think that the second mirror should not be tied to a third-party resource. And should download directly from ip joomla.org, because github can also block.
In most cases, the main mirror will be available, so there should not be any problems.
IMHO.

avatar mbabker
mbabker - comment - 11 Nov 2017

The not showing up on GitHub thing was all me, I just ran into a time crunch Tuesday morning made worse by a combination of non-Joomla related stress issues so I didn't get to publish that when I wanted to.

As for a mirror on our domain I'll do some research into a reasonable solution for that. It probably will be using our update.joomla.org platform as it's already a dedicated system in hour hosting architecture and is on a CDN so can help with not putting too much stress on that server.

avatar mbabker
mbabker - comment - 11 Nov 2017

So I've been thinking things through a little bit, and I think we actually should take the opportunity to deprecate <downloadurl> and replace it with a functionally similar <download> tag. I also think we can use the new tag and define standard attributes and values and actually use that data in updates (like right now type="full" is basically ignored), that would actually help with an open issue of not being able to serve our smaller patch packages for core updates right now.

I know this is a bit of a change in direction but thinking about it using a new tag designed to be a collection element versus a single item lets us also address other architectural shortcomings needing attention without being majorly disruptive to extensions (well until the old tag goes away).

avatar effrit
effrit - comment - 11 Nov 2017

@mbabker, seems like we have a plan now. Maybe it will be right thing to implement #18547 PR and make release for testing?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar effrit
effrit - comment - 11 Nov 2017

@mbabker, seems like we have a plan now. Maybe it will be right thing to implement #18547 PR and make release for testing?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar Septdir
Septdir - comment - 11 Nov 2017

@mbabker
I believe that the entire current system of updates, or rather the xml file, does not fit well. I would have completely redesigned it (I thought about it for a long time when I wrote a component of updates for my project), I would also share kernel updates from updating extensions. Well and still probably 10 offers on this this account. If you want to talk about this, we can contact you and discuss everything personally.
But firstly, in the current situation, we need a quick solution to the problem.
Secondly, any changes in the update mechanism are threatened with refusals of updates of extensions. And other consequences.
IMHO

avatar caprom
caprom - comment - 13 Nov 2017

@mbabker

"...we're screwed if someone just flat out blocks the entirety of S3 instead of specific regions/buckets)."

Actually what is blocked is something like https://s3-us-west-2.amazonaws.com/poker888...or...whatever

Some ISPs in Russia have no technical means to block a single https page. That's why they block a whole domain. This story dates back to early 2016. I mean it's a matter to change ISP since a percentage of unlucky people is very low in Russia.

avatar mbabker
mbabker - comment - 13 Nov 2017

Well I found one of the sites that you can check these things on and found it was only that region. Which is good for now. But like I said, it'd be major trouble (for a lot of people) if instead of blocking a specific AWS region/subdomain the entire AWS domain were blocked.

avatar effrit
effrit - comment - 13 Nov 2017

@mbabker, this is really problem now. the official site is
http://blocklist.rkn.gov.ru/
and it show s3-us-west-2.amazonaws.com as blocked, so internet and hosting providers just tell us 'its law, we cant do anything about it'.
so we wait for solution!
please dont listen caprom, he told peoples what there is no problem because he have not it.
and his 'minority' came on our Russian forum every day!

avatar Theatral
Theatral - comment - 14 Nov 2017

I'm experiencing the same problem, many sites are not updated. The decision can be in the near future?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/18543.

avatar Septdir
Septdir - comment - 28 Nov 2017

Any news?

avatar zero-24
zero-24 - comment - 28 Nov 2017

For a shortterm solution it looks like the OSM decided the following:

Will be migrating AWS storage of downloads to another region to get around the restrictions in place by some countries

https://volunteers.joomla.org/board-of-directors/reports/693-osm-inc-full-board-meeting-november-09-2017

But i have no more details like this nor what the current status is about the migration.

avatar wilsonge wilsonge - change - 2 Dec 2017
Status Discussion Closed
Closed_Date 0000-00-00 00:00:00 2017-12-02 17:12:38
Closed_By wilsonge
avatar wilsonge wilsonge - close - 2 Dec 2017

Add a Comment

Login with GitHub to post a comment