? Success

User tests: Successful: Unsuccessful:

avatar milux
milux
7 Nov 2016

Pull Request for Issue #12816

Summary of Changes

Changed method "download()" of class "JoomlaupdateModelDefault", found in administrator/components/com_joomlaupdate/models/default.php.
The method does now fetch the HTTP headers of the update download location $updateInfo['object']->downloadurl->_data until the actual file (on AWS) is reached.
It further removes the path and the query string from the URL, resulting in the actual filename of the update package, which should work fine for all platforms.

Testing Instructions

Some people should check whether I messed up something for some specific platform setup.

Documentation Changes Required

none

avatar milux milux - open - 7 Nov 2016
avatar milux milux - change - 7 Nov 2016
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 7 Nov 2016
Category Administration Components
avatar wojsmol
wojsmol - comment - 7 Nov 2016

@milux In a moment I'll be testing this PR.
cc @trzepiz @ot2sen @noras666 @brianteeman Please test this.

avatar wojsmol wojsmol - test_item - 7 Nov 2016 - Tested successfully
avatar wojsmol
wojsmol - comment - 7 Nov 2016

I have tested this item โœ… successfully on 613a6c1


OS Windows 8.1

#Fields: datetime   priority clientip   category    message
2016-11-07T16:11:17+00:00   INFO ::1    update  Update started by user Super User (410). Old version is 3.6.2.
2016-11-07T16:11:20+00:00   INFO ::1    update  Downloading update file from http://joomla-official-downloads.s3.amazonaws.com/joomladownloads/joomla3/Joomla_3.6.4-Stable-Update_Package.zip?[AWS patams].
2016-11-07T16:11:25+00:00   INFO ::1    update  File Joomla_3.6.4-Stable-Update_Package.zip successfully downloaded.
2016-11-07T16:11:27+00:00   INFO ::1    update  Starting installation of new version.
2016-11-07T16:11:49+00:00   INFO ::1    update  Finalising installation.
2016-11-07T16:11:49+00:00   INFO ::1    update  Ran query from file 3.6.3-2016-08-15. Query text: ALTER TABLE `#__newsfeeds` MODIFY `link` VARCHAR(2048) NOT NULL;.
2016-11-07T16:11:49+00:00   INFO ::1    update  Ran query from file 3.6.3-2016-08-16. Query text: INSERT INTO `#__postinstall_messages` (`extension_id`, `title_key`, `description.
2016-11-07T16:11:50+00:00   INFO ::1    update  Deleting removed files and folders.
2016-11-07T16:11:56+00:00   INFO ::1    update  Cleaning up after installation.
2016-11-07T16:11:56+00:00   INFO ::1    update  Update to version 3.6.4 is complete.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar ot2sen ot2sen - test_item - 7 Nov 2016 - Tested successfully
avatar ot2sen
ot2sen - comment - 7 Nov 2016

I have tested this item โœ… successfully on 613a6c1

Applied the patch using patchtester to the 3.6.2 that couldnยดt be updated yesterday.
After applying patch and running the update it completes and is now at 3.6.4


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar jeckodevelopment jeckodevelopment - change - 7 Nov 2016
Status Pending Ready to Commit
avatar jeckodevelopment
jeckodevelopment - comment - 7 Nov 2016

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar milux milux - change - 7 Nov 2016
Milestone Added:
avatar milux milux - change - 7 Nov 2016
Labels Added: ?
avatar zero-24
zero-24 - comment - 7 Nov 2016

Hmm i'm not sure here. @mbabker can you take a look here?

avatar joomla-cms-bot joomla-cms-bot - change - 7 Nov 2016
Milestone Removed:
avatar jeckodevelopment jeckodevelopment - change - 7 Nov 2016
Status Ready to Commit Needs Review
Labels
avatar joomla-cms-bot joomla-cms-bot - change - 7 Nov 2016
Labels
avatar jeckodevelopment
jeckodevelopment - comment - 7 Nov 2016

Waiting for review


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar milux
milux - comment - 7 Nov 2016

@zero-24 The change could be problematic when the last observed location does not represent a file ending on ".zip". Is that possible? In that case I might check for proper file extension and append it manually if necessary...

avatar mbabker
mbabker - comment - 7 Nov 2016

Well there are two options:

1) Just remove the query string segment and leave it as the URL fragment without a file extension (probably bad news)
2) Follow the headers to get the correct file name with file extension.

I don't think there's an issue here honestly, unless someone's got a scenario I'm not aware of to look at.

avatar zero-24
zero-24 - comment - 7 Nov 2016

Thanks @mbabker ๐Ÿ‘

avatar milux
milux - comment - 7 Nov 2016

@mbabker
1) Already tried that one... doesn't work, Joomla doesn't extract the file without proper file extension.
2) Exactly what I did, however if the file from AWS doesn't end on ".zip", the process will fail

There are actually two more options:
3) Just remove the query string, check if the basename ends on ".zip", append if necessary
4) Resolve until destination and do the same thing as in 3)

So, what should it be? ;)

avatar mbabker
mbabker - comment - 7 Nov 2016

Leave it as is. I wouldn't hardcode a .zip suffix check in the off chance the file extension ever changes (like if we tried serving the smaller .tar.* files instead of the ZIP package).

avatar ggppdk
ggppdk - comment - 7 Nov 2016

Leave it as is. I wouldn't hardcode a .zip suffix check in the off chance the file extension ever changes (like if we tried serving the smaller .tar.* files instead of the ZIP package).

administrator/components/com_joomlaupdate/restore.php
method: getUnarchiver()

it only supports: JPA, JPS, ZIP

avatar milux
milux - comment - 7 Nov 2016

... with JPA and JPS being Akeeba-specific formats afaik, but how about a 5th solution:
5) We take the file extension from the original URL and append it to the basepath if it is not there yet. ๐Ÿ˜œ

avatar milux milux - change - 7 Nov 2016
Milestone Added:
avatar mbabker
mbabker - comment - 7 Nov 2016

This is fine as is. Really we'd only have further problems if someone uploaded packages to AWS in the future without a file extension, in which case it'd probably be a broken file anyway. Plus our main interface to AWS is through the downloads site and is a Joomla extension, so the packages are going through validation on upload there. Let's roll with it.

@joomla/cms-maintainers When this is merged, I'd suggest tagging a standalone update for the update component since it looks like anyone hosting a production site on a Windows platform will have issues upgrading with packages pulled from AWS thanks to the URL query strings in both the XML update file and the resulting URL after following all redirects.

avatar milux milux - change - 7 Nov 2016
Milestone Removed:
avatar ggppdk
ggppdk - comment - 7 Nov 2016

There is a rare problem (maybe not possible problem) that might occur with this PR

if the redirect/final URL does not have a query string (no character ?) then:

// Remove URL and query string
$urlBasename = basename($packageURL);
$basename    = substr($urlBasename, 0, strpos($urlBasename, '?'));

$basename will be empty, because substr will return an empty string, because strpos will give FALSE as length parameter

This should be safer :

// Remove URL and query string
$basename = basename($packageURL);
if (strpos($basename, '?'))
{
    $basename = substr($basename , 0, strpos($basename, '?'));
}
avatar ggppdk
ggppdk - comment - 7 Nov 2016

Please see my above comment

about adding zip as extension

  • only in the case that extension is not detected in the redirected URL, is a good sanity check,

Also we can try to grab the extension for the original update URL too,
Besides that is the purpose of &jcompat=my.zip , right ?

Is it preferable if no extension is detected that upgrade will fail ?
Plus currently as mentioned above, the getUnarchiver() only supports ZIP and joomla will not distrubute packages with JPA or JPS format

avatar milux
milux - comment - 7 Nov 2016

@ggppdk Thanks for your suggestion, integrated that.
Because of what @mbabker suggested, I don't think that we need to hassle around with the extension any more.
Is this ready for RTC then?

avatar jeckodevelopment
jeckodevelopment - comment - 7 Nov 2016

Needs tests again and it seems that Travis is not happy

avatar milux
milux - comment - 7 Nov 2016

Exactly... I just wonder how I made it unhappy? ๐Ÿค”

avatar milux
milux - comment - 7 Nov 2016

@jeckodevelopment Maybe now? ๐Ÿ˜‰

avatar jeckodevelopment
jeckodevelopment - comment - 7 Nov 2016

test test test ๐Ÿ˜„
thanks!

avatar wojsmol wojsmol - test_item - 7 Nov 2016 - Tested successfully
avatar wojsmol
wojsmol - comment - 7 Nov 2016

I have tested this item โœ… successfully on 4c66273


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar milux milux - test_item - 7 Nov 2016 - Tested successfully
avatar milux
milux - comment - 7 Nov 2016

I have tested this item โœ… successfully on 4c66273


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar milux
milux - comment - 7 Nov 2016

@jeckodevelopment Done, for the sake of it. (There wasn't really a change, and yes, I actually tested it even though and didn't just press the submit test button. ๐Ÿ˜‰ )

avatar jeckodevelopment jeckodevelopment - change - 8 Nov 2016
Status Needs Review Ready to Commit
avatar jeckodevelopment
jeckodevelopment - comment - 8 Nov 2016

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12817.

avatar jeckodevelopment jeckodevelopment - change - 8 Nov 2016
Milestone Added:
avatar jeckodevelopment jeckodevelopment - change - 8 Nov 2016
Labels Added: ?
avatar rdeutz rdeutz - reference | 7aa5e65 - 8 Nov 16
avatar rdeutz rdeutz - merge - 8 Nov 2016
avatar rdeutz rdeutz - close - 8 Nov 2016
avatar rdeutz rdeutz - change - 8 Nov 2016
Status Ready to Commit Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2016-11-08 09:04:47
Closed_By rdeutz
avatar rdeutz rdeutz - close - 8 Nov 2016
avatar rdeutz rdeutz - merge - 8 Nov 2016
avatar nvyush nvyush - reference | c583f65 - 9 Nov 16
avatar wojsmol
wojsmol - comment - 9 Nov 2016

Is it planned a new version of the com_joomlaupdate component with this fix before the release of Joomla 3.7.0?

avatar zero-24
zero-24 - comment - 9 Nov 2016

@wilsonge @rdeutz any plans on this here?

avatar wilsonge
wilsonge - comment - 9 Nov 2016

No. We only distribute com_joomlaupdate in exceptional circumstances - basically when we make break the updater in a release. All other modifications ship as per the usual release cycle of the CMS

avatar wojsmol
wojsmol - comment - 9 Nov 2016

@wilsonge Without this fix com_joomlaupdate does not work under Windows. See #12817 (comment) and joomla/joomla-websites#742

avatar mbabker
mbabker - comment - 9 Nov 2016

I would suggest someone publish a page with guidance for users on a Windows platform as they cannot direct download and upgrade if the decision is to not release an interim update for the component.

avatar trzepiz
trzepiz - comment - 9 Nov 2016

@mbabker This is a big global problem for all Windows users and Windows servers. Should I tell everyone in Poland that "One click update" does not work on Windows?.

avatar mbabker
mbabker - comment - 9 Nov 2016

It's fixed in the code for 3.7. I can't do anything about any other decisions.

avatar brianteeman
brianteeman - comment - 9 Nov 2016

How many people run Joomla on a windows server?

avatar mbabker
mbabker - comment - 9 Nov 2016

13% of sites sending stats in the last 60 days are Windows. How many of those are local dev versus production, we don't track, but that's definitely a higher use rate than other configurations we support.

avatar PhilETaylor
PhilETaylor - comment - 27 Nov 2016

13% ? wow... would be interested to find out how many were IIS and how many were XAMP - maybe the stats should log server type too ?

Add a Comment

Login with GitHub to post a comment