? Pending

User tests: Successful: Unsuccessful:

avatar regularlabs
regularlabs
13 Nov 2018

On some servers headers of downloaded files have lowercase keys because some weird server settings.
So you get headers like:

[content-disposition] => attachment; filename="foobar-v1.2.3.zip"

instead of:

[Content-Disposition] => attachment; filename="foobar-v1.2.3.zip"

Because of the case difference, the Installer cannot find the correct filename in the 'Content-Disposition', as key matching is case sensitive.

Summary of Changes

This PR will convert all keys to lowercase and base the 2 code parts trying to do stuff with the header info ('Location' and 'Content-Disposition') on the lowercase keys.

Testing Instructions

This is hard to test, as the casing of the keys in the headers is server dependant.
I have no idea what server setting causes it yet.
So only way to test this would be to override the headers manually or something.

Or just read the code and see that it doesn't do anything crazy and doesn't break anything.

avatar regularlabs regularlabs - open - 13 Nov 2018
avatar regularlabs regularlabs - change - 13 Nov 2018
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 13 Nov 2018
Category Libraries
avatar regularlabs regularlabs - change - 13 Nov 2018
The description was changed
avatar regularlabs regularlabs - edited - 13 Nov 2018
avatar regularlabs
regularlabs - comment - 13 Nov 2018
avatar regularlabs
regularlabs - comment - 13 Nov 2018

PPS: There are more places in the Joomla code where headers are checked, which are case sensitive.
Maybe good idea to go through all of them too.

avatar regularlabs
regularlabs - comment - 13 Nov 2018

PS: there also seem to be different servers using different capital formats:
$headers['Content-Type'] vs $headers['Content-type'] vs $headers['content-type'].

So converting all the keys to lowercase first, before checking, is the way to go.

It would save a step if Joomla\CMS\Http would already return the header keys as lowercase.
But that would cause B\C issues with everything still relying on the camelcase keys.

avatar mbabker
mbabker - comment - 13 Nov 2018

It would save a step if Joomla\CMS\Http would already return the header keys as lowercase.
But that would cause B\C issues with everything still relying on the camelcase keys.

Truthfully, I wouldn't try making too many changes to the HTTP API at this point, especially considering changes coming with 4.0.

Since Joomla\CMS\Http\Response was created as a rather simple DTO, putting an API around it to normalize things could be rather painful as far as B/C goes.

At the Framework level, in the next major we've made Joomla\Http\Response extend Zend\Diactoros\Response as part of making our package a bit more compliant with the various PSR standards. The Diactoros Response object does do a bit of internal standardization when using the PSR-7 API for reading header values (note the equivalent to our existing $response->headers, $response->getHeaders(), doesn't have this standardization). And in 4.0 Joomla\CMS\Http is inheriting from Joomla\Http so that bit of refactoring does also apply to the CMS.

I wouldn't necessarily be opposed to seeing PRs backporting some of the PSR-7 based API into the Response object to add the standardization logic (so you could do $response->getHeader('Content-Type') without the case sensitivity issues and without the B/C concern of changing how $response->headers is seeded), but I personally wouldn't invest too much time into changing things as most of this issue is already addressed, just not on the current stable release line.

avatar regularlabs
regularlabs - comment - 13 Nov 2018

Ok, totally agree :)

avatar regularlabs regularlabs - change - 14 Nov 2018
Labels Added: ?
avatar zero-24 zero-24 - change - 14 Nov 2018
Status Pending Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2018-11-14 21:56:41
Closed_By zero-24
avatar zero-24 zero-24 - close - 14 Nov 2018
avatar zero-24 zero-24 - merge - 14 Nov 2018
avatar zero-24
zero-24 - comment - 14 Nov 2018

Merging thanks.

avatar regularlabs
regularlabs - comment - 15 Nov 2018

?

avatar PhilETaylor
PhilETaylor - comment - 27 Nov 2018

This resolves a major issue with updating extensions in Joomla 3.9.0 - people get this:

screenshot 2018-11-27 at 11 20 55

On debugging all the headers in the $response->headers array ARE ALREADY LOWER CASE for some reason, even when the response headers were not (I checked)

So maybe the upstream change in the HTTP Package @mbabker changed this?

avatar mbabker
mbabker - comment - 27 Nov 2018

3.x does not use the Framework's HTTP package so no "upstream change" can impact it.

In 4.x the headers are standardized in the Joomla\Http\Response object because it is inheriting from Zend\Diactoros\Response which performs that standardization (as noted before).

avatar PhilETaylor
PhilETaylor - comment - 27 Nov 2018

Ive debugged this to the CurlTransport... which in getResponse, the headers returned are all in lowercase, even though the server is giving Correct-Case....

So I think this is a curl issue?

https://tools.ietf.org/html/rfc7230#section-3.2 states:

Each header field consists of a case-insensitive field name followed
by a colon (":"), optional leading whitespace, the field value, and
optional trailing whitespace.

So this PR is actually a good thing to handle all cases.

avatar mbabker
mbabker - comment - 27 Nov 2018

Even if there is something in curl doing it, the main point is that our HTTP adapters aren't doing any form of standardization on their own (3.x does none, 4.x inherits it because our Response object is basically replaced).

So, yeah, this PR is fine for what it is. It just needs to remain clear there is no standardization or manipulation happening in our HTTP API, any coincidental changes are purely because of the native PHP functions each transport wraps around.

avatar PhilETaylor
PhilETaylor - comment - 27 Nov 2018

yup - but what this PR does is standardize the comparison of apples and pears and compares apples with apples :)

And more importantly, allows extensions to be installed again.

This is a massive issue, that loads of people are complaining about this week :(

avatar regularlabs
regularlabs - comment - 27 Nov 2018

I don't think pears come in to play here.
All this PR does is that it sees apples and Apples and APPLES all as apples.
So it doesn't matter how loud you say apples, they are still apples.

avatar PhilETaylor
PhilETaylor - comment - 27 Nov 2018

Yup - and it works and fixes the issues people are having, and now released in 3.9.1 so people can stop complaining to us all :)

avatar regularlabs
regularlabs - comment - 27 Nov 2018

yep :)

I have had the same buttload of complaints about this issue too.

avatar robwent
robwent - comment - 12 Jun 2019

I just found this after having the same error message when trying to update akeeba and Joomla 3.9.6 => 3.9.8

When I checked the merged patch it was already on the site.

It turned out that the domain was up to its storage quota and upping the available space got rid of the error and everything update as it should.

I could see that the packages to update were downloaded to the tmp folder so probably a borderline case when there was enough space to download but not enough to unxip.

Might help someone else who lands on this issue.

Add a Comment

Login with GitHub to post a comment