If you merge this while the issue with the AmazonS3 certificate isn't fixed, then Joomla Updates will fail due to invalid root certs on some servers.
One major difference between JHttp and the com_joomlaupdate downloader is that JHttp does peer validation while the latter doesn't do it.

@Bakual, I am aware of the peer validation but I had no problem to do a Joomla update or download a language pack from AWS using the staging branch. This is also good to identify where the issues are and we can fix them.

Yes, it works on some (most) servers, but it fails on some. It depends on the server setup and it can be easily tested with com_weblinks. Update Weblinks from 3.4.0 to 3.4.1 using the extension updater. If that works, then your server is unaffected. But if that update fails, then the Joomla Update will fail as well with this PR. And this is not something you want to happen.
Thus the AmazonS3 issue needs to be fixed before this PR can be merged.

@Bakual I have added the Weblinks test to the test instructions.

[quote]If that works, then your server is unaffected.[/quote]I thought it was based on the cacert.pem Joomla supplies rather than what the server has. It works different?

Servers also have their own cacert.perm installed often - so I guess it at least partly depends on that.

The amazons3 issue cannot be fixed while we ship the current cacert.pem
which is not compatible with amazons3 - as explained ad nauseum already

On 10 December 2015 at 22:26, George Wilson notifications@github.com
wrote:

Servers also have their own cacert.perm installed often - so I guess it at
least partly depends on that.

—
Reply to this email directly or view it on GitHub
#8645 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

Brian is right. We either have to change our cacert.pem to include the root cert for AmazonS3 or wait till they fix it themself (which doesn't seem to be anytime soon).
If we change the cacert.pem, we need to have a process how it can be updated in future.
If someone needs a playground to test this issue, just tell me. I have such an affected server.

@Bakual I would like to have access. Thanks.

If your stuff doesn't work Roland the next step is to try with the cacert.perm that comes from centos nic is shipping with FOF (https://github.com/akeeba/fof/blob/development/fof/Download/Adapter/cacert.pem)

Sent credentials to Roland.
The cacert.pem from Nic is working. I tried that. But we need to get a way to recreate this periodically if we go this way. The one we use now (curl.haxx.se/ca/cacert.pem) is considered the "official" one and is available for public download. I don't know how the one from Centos can be retrieved.

Just Google it. It's well documented how to create a valid cacert.pem even
on the haxx web site. It's just a case of compiling with the correcr
options which for some reason haxx decided to override.
On 11 Dec 2015 5:59 pm, "Thomas Hunziker" notifications@github.com wrote:

Sent credentials to Roland.
The cacert.pem from Nic is working. I tried that. But we need to get a way
to recreate this periodically if we go this way. The one we use now (
curl.haxx.se/ca/cacert.pem) is considered the "official" one and is
available for public download. I don't know how the one from Centos can be
retrieved.

—
Reply to this email directly or view it on GitHub
#8645 (comment).

They increased the security requirements. Anything that used 1024-bit key's got removed from their lists. (see http://curl.haxx.se/mail/archive-2014-10/0066.html#start)

Just Google it. It's well documented how to create a valid cacert.pem even
on the haxx web site.

Yeah, but you will get a different one on every distro / browser. And we certainly don't want another cacert.pem source each time we update it. That's why I ask for a way to get predictable results.
With haxx.se this was very simple as they offer the file for download and they are kind of an official source.
With Centos I'm not aware of such a download location. We would need to recreate it from a Centos (which version?) installation each time. It greatly reduces the number of people who could do that both because of access to Centos and knowhow of creating the file.
Tobias tried to just add the (unsafe) root cert from AmazonS3 to our cacert.pem, but for some reason you can't just add a cert to the file.

No you do NOT need to get it from centos. As I said the instructions to
create it are easy to find. If I can find it I'm sure you can. (On phone or
would paste the link)
On 11 Dec 2015 6:34 pm, "Thomas Hunziker" notifications@github.com wrote:

Just Google it. It's well documented how to create a valid cacert.pem even
on the haxx web site.

Yeah, but you will get a different one on every distro / browser. And we
certainly don't want another cacert.pem source each time we update it.
That's why I ask for a way to get predictable results.
With haxx.se this was very simple as they offer the file for download and
they are kind of an official source.
With Centos I'm not aware of such a download location. We would need to
recreate it from a Centos (which version?) installation each time. It
greatly reduces the number of people who could do that both because of
access to Centos and knowhow of creating the file.
Tobias tried to just add the (unsafe) root cert from AmazonS3 to our
cacert.pem, but for some reason you can't just add a cert to the file.

—
Reply to this email directly or view it on GitHub
#8645 (comment).

It isn't about creating them. It's indeed possible to do that as long as you have CLI access to a server.
As far as I understand the system, the root CAs "installed" on a system vary by the server enviroment. So if you create a cacert.pem on your server it will most likely not be the same as when I create one on my server.
There is no "official" list of root CAs that have to be installed on each system. Most server OS take the list of root CAs which are bundled by browsers, namely the ones from Mozilla. Which is also the one haxx.se offers for download. And Mozilla (not haxx.se) removed the certs using the unsecure RSA-1024 encryption around a year ago.
So apparently some servers like Centos didn't remove those unsecure root CAs from their distro and that's why they still work. Those bundles installed on the server seem to be checked additionally to the cacert.pem we provide.

Just did some tests again on my server and apparently it works now. Also https://www.sslshopper.com/ssl-checker.html#hostname=github-cloud.s3.amazonaws.com shows the cert now as fine.
Thus it looks like AmazonS3 finally solved the issue on their end.

Thus our current cacert.pem will work and this PR should be fine.

Sorry for the confusion then. I wasn't aware they have changed their certs now. Last I checked was about two or three weeks ago.

True, it seems finally Amazon S3 uses a new certificate that uses "DigiCert High Assurance EV Root CA" as root CA (also seems to be cross-signed with "Baltimore Cybertrust Root" CA).

Both root CA ("DigiCert High Assurance EV Root CA" and "Baltimore Cybertrust Root") are in curl.haxx.se root CA certs file: http://curl.haxx.se/ca/cacert.pem

We still need to test this PR on more shared hosts. Nic's code has a lot more fallbacks than we have currently in JHttp. So we need to be careful we don't break updates on poor quality hosts

I have tested this item successfully on 6324947

I have test the first test with Joomla Update it and it worked.
Thanks

This PR has received new commits.

CC: @iplocker

This PR has received new commits.

CC: @iplocker

Both tests worked for me on a chrooted server (thats fails WP update btw.).

I have not tested this item.

Tested ok (not that I ever had a problem before on this server)

However I did get a failure when testing under ftp mode where it said "invalid credentials" but still went ahead and updated (did it fall back to using non-ftp mode) - will try to work out how to replicate that

This PR has received new commits.

CC: @brianteeman, @iplocker

both updated
works from my side

@brianteeman I tested this with the FTP layer enabled but didn't get any error and the update worked as expected. If you are able to replicate it, I would like to hear it.

Yeah no idea what happened. Tried five times and couldn't replicate it.
On 27 Jan 2016 10:20 am, "RolandD" notifications@github.com wrote:

@brianteeman https://github.com/brianteeman I tested this with the FTP
layer enabled but didn't get any error and the update worked as expected.
If you are able to replicate it, I would like to hear it.

—
Reply to this email directly or view it on GitHub
#8645 (comment).

I have tested this item successfully on c4f86b5

I have tested - following your instructions - on two live hosts and it worked. Thanks roland-d!

I have encouraged a lot of people to test this, making it easier to understand how to test is the main obstacle. I didn't think I would know how to test, these instructions were great.

Obviously I'm a complete noob, but I think it is worth adding the following to the instructions for other noobs.

After install patch tester.

Go to Components > Patch Tester.

Click on Fetch Data

Find the patch. (I tried to search by number but it didn't work so I sorted descending and scrolled through to find it)

Click on the "apply the patch" button.

After you have finished testing go to this link: https://issues.joomla.org/tracker/joomla-cms/8645 and click on the blue "test this" button. Fill in the form.

I tested this successfully.

I have tested this item successfully on c4f86b5

Tested, Joomla! updated. Weblinks Updated. Siteground server (Go Geek)

I have tested this item successfully on c4f86b5

Installed on local WAMP isnatllation.

PHP 5.4.3
Apache 2.2.22
MYSQL 5.5.16

No issues encountered.

I have tested this item successfully on c4f86b5

Tested on: Namesco Hosting
MySQL: 5.5.5-10.1.10-MariaDB
PHP: 5.6.17
WebServer: Unkown

No issues.

I have tested this item unsuccessfully on c4f86b5

Tested on: UKFast Hosting
PHP: 5.4.45
MySQL: 5.5.37
WebServer: Unkown
Failed to install Patch. Step 3 of testing instructions.
Uninstalled patch tester and reinstalled, same issue.

Think this issues for this particular hosting has already been raised by JoomlaLondon.

If patchtester isn't working you can try manually applying it to the files

@Bodge-IT check if you are having have the issue described in this PR #8676

I have tested this item successfully on c4f86b5

Tested on Siteground Hosting
PHP: 5.3.29
MYSQL: 5.5.44-37.3-log
Apache: Unknown

No Issues.

After much testing i guess we can move here to RTC? Or do we need more tests here?

RTC

RTC Jissues bot. Listen to me!

OK Plan is to merge this in combination with the com_joomlaupdate changes in @zero-24 's PR

@roland-d could you have a look at the merge conflicts, thanks

@wilsonge you mean #9612 with the in combination statement?

This PR has received new commits.

CC: @Bodge-IT, @brianteeman, @iplocker, @uglyeoin, @waader

@rdeutz Merge conflicts fixed.

@rdeutz yes I mean #9612 :)

ok merged in combination :-)