No Code Attached Yet
avatar ThomasGarcia05
ThomasGarcia05
27 Jan 2022

Hello,

it seems to me that there is an issue when updating Joomla under a proxy

The proxy is well designated in the config panel, but the update don't run.

In the log we get the following error :
[04-Jan-2021 18:06:06] WARNING: child 9550 said into stderr: "NOTICE: PHP message: PHP Warning: get_headers(https://downloads.joomla.org/cms/joo..._Package.zip): failed to open stream: Connection timed out in /applis/xxx/joomla/administrator/components/com_joomlaupdate/models/default.php on line 285"

After a check in /joomla/administrator/components/com_joomlaupdate/models/default.php on line 285"

It seems that the get_headers don't follow the PROXY parameter of the config panel

In the download function whe made the following change :

PHP code:
/**

  • Downloads the update package to the site.
  • @return boolean|string False on failure, basename of the file in any other case.
  • @SInCE 2.5.4
    */
    public function download()
    {
    $context = array(
    'http' => array(
    'proxy' => 'UrlProxy:Port',
    'request_fulluri' => true,
    ),
    );

$updateInfo = $this->getUpdateInformation();
$packageURL = trim($updateInfo['object']->downloadurl->_data);
$sources = $updateInfo['object']->get('downloadSources', array());
//$headers = get_headers($packageURL, 1);
$headers = get_headers($packageURL, 1, $context);

// Follow the Location headers until the actual download URL is known
while (isset($headers['Location']))
{
$packageURL = $headers['Location'];
//$headers = get_headers($packageURL, 1);
$headers = get_headers($packageURL, 1, $context);
}

Whith this change the automatic update run correctly.

Is it a change that will occur in the next version ? Did i made a mistake ?

Regards
(Thomas Garcia, with the support of Eric Bertin-Maghit)

avatar ThomasGarcia05 ThomasGarcia05 - open - 27 Jan 2022
avatar ThomasGarcia05 ThomasGarcia05 - change - 27 Jan 2022
Labels Removed: ?
avatar joomla-cms-bot joomla-cms-bot - change - 27 Jan 2022
Labels Added: No Code Attached Yet
avatar joomla-cms-bot joomla-cms-bot - labeled - 27 Jan 2022
avatar richard67
richard67 - comment - 27 Jan 2022

@ThomasGarcia05 Could you provide some information about from which to which Joomla version you are updating? Thanks in advance.

avatar zero-24
zero-24 - comment - 27 Jan 2022

Hmm on a quick look it looks like we have to add something like this when there is a proxy setting:
https://stackoverflow.com/questions/17111112/using-get-headers-with-a-proxy

Or can we do what get_headers does via our HTTP class somehow that would be a better solution I guess as IIRC there we already handle all that proxy stuff.

avatar zero-24
zero-24 - comment - 27 Jan 2022

Does anyone know how to easlily "simulate" an proxy so that we could setup an test enviorment? Or does someone have a proxy setup that we could use to test on?

avatar brianteeman
brianteeman - comment - 27 Jan 2022

You can use Charles - @HLeithner was able to help me set it up previously

avatar richard67
richard67 - comment - 27 Jan 2022

@ThomasGarcia05 How are the Proxy Settings set in the "Server" tab of Global Configuration?

avatar ThomasGarcia05
ThomasGarcia05 - comment - 27 Jan 2022

The last time it was to 3.10.5 from the version just before.
For the Proxy Settings (my translation from french, sorry for mistakes)
Behind a load balancer : no
Enable outbound proxy : yes
Outbound proxy host : http://proxy.ac-aix-marseille.fr
Outbound proxy port : 8080

Other fields without answer


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/36869.

avatar HLeithner
HLeithner - comment - 27 Jan 2022

you can't use a "normal" proxy or better only the proxy without manually disable the certificate validation for joomla update.

We force https and a valid certificate, if your proxy has a root certificate which can be used by joomla you should be possible to use the proxy. else it will only be possible with a core hack

actually think about it it's a feature that can be removed since hopefully 99% of the internet uses trusted tls connections. Or at least the gui can be removed and only be set in the configuration.php also with a custom root certificate.

So to make it short I would say it's an expected behavior if you don't use a man in the middle attack (like legal law enforcement) in your proxy.

Bringing my thoughts to an end we should set CURLOPT_HTTPPROXYTUNNEL for HTTPS connections in curl, then the proxy doesn't "proxy" anymore instead it makes a direct TCP connection and certification validation would work again. Of course only if the proxy supports the CONNECT header for this type of proxing.

to try this out could you please add the following line to the file libraries/src/Http/Transport/CurlTransport.php at line 180 after $options[CURLOPT_PROXY] = $config->get('proxy_host') . ':' . $config->get('proxy_port');

$options[CURLOPT_HTTPPROXYTUNNEL] = 1;

if this works for you someone should create a PR for this adding the parameter if the requested url starts with https

avatar zero-24
zero-24 - comment - 27 Jan 2022

The get_header calls are there since 3.7.0 without anyone reporting any issues. #12817

My question would by why did we need that thing anyway? Doesnt JHttp usually follow the location anyway?

@ThomasGarcia05 please test this new method and report back whether that workes for you:

	/**
	 * Downloads the update package to the site.
	 *
	 * @return  boolean|string  False on failure, basename of the file in any other case.
	 *
	 * @since   2.5.4
	 */
	public function download()
	{
		$http       = new JHttp;
		$updateInfo = $this->getUpdateInformation();
		$packageURL = trim($updateInfo['object']->downloadurl->_data);
		$sources    = $updateInfo['object']->get('downloadSources', array());

		try
		{
			$head = $http->head($packageURL);
		}
		catch (RuntimeException $e)
		{
			// Passing false here -> download failed message
			$response['basename'] = false;

			return $response;
		}

		// Follow the Location headers until the actual download URL is known
		while (isset($head->headers['location']))
		{
			$packageURL = $head->headers['location'];

			try
			{
				$head = $http->head($packageURL);
			}
			catch (RuntimeException $e)
			{
				// Passing false here -> download failed message
				$response['basename'] = false;

				return $response;
			}
		}

		// Remove protocol, path and query string from URL
		$basename = basename($packageURL);

		if (strpos($basename, '?') !== false)
		{
			$basename = substr($basename, 0, strpos($basename, '?'));
		}

		// Find the path to the temp directory and the local package.
		$config   = JFactory::getConfig();
		$tempdir  = (string) InputFilter::getInstance(array(), array(), 1, 1)->clean($config->get('tmp_path'), 'path');
		$target   = $tempdir . '/' . $basename;
		$response = array();

		// Do we have a cached file?
		$exists = JFile::exists($target);

		if (!$exists)
		{
			// Not there, let's fetch it.
			$mirror = 0;

			while (!($download = $this->downloadPackage($packageURL, $target)) && isset($sources[$mirror]))
			{
				$name       = $sources[$mirror];
				$packageURL = trim($name->url);
				$mirror++;
			}

			$response['basename'] = $download;
		}
		else
		{
			// Is it a 0-byte file? If so, re-download please.
			$filesize = @filesize($target);

			if (empty($filesize))
			{
				$mirror = 0;

				while (!($download = $this->downloadPackage($packageURL, $target)) && isset($sources[$mirror]))
				{
					$name       = $sources[$mirror];
					$packageURL = trim($name->url);
					$mirror++;
				}

				$response['basename'] = $download;
			}

			// Yes, it's there, skip downloading.
			$response['basename'] = $basename;
		}

		$response['check'] = $this->isChecksumValid($target, $updateInfo['object']);

		return $response;
	}

Looks like we would also have to test that change on windows as on that ticket it looks like its intended to fix an issue with xampp on windows. On my end it works still it looks a bit pointless to me as we always follow the redirects? Dont we? I must miss something.

avatar ThomasGarcia05
ThomasGarcia05 - comment - 28 Jan 2022

Hello,

this last method work for us. But after the update we come back to the original file.
Is it possible to put it in the future version of Joomla ?

Thank you.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/36869.

avatar zero-24
zero-24 - comment - 28 Jan 2022

Hello,

this last method work for us. But after the update we come back to the original file. Is it possible to put it in the future version of Joomla ?

Thank you.

This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/36869.

Sure thats the plan but i have not been able to setup the PR yet. Thanks for the test will prepare a PR over the weekend.

avatar zero-24 zero-24 - change - 29 Jan 2022
Status New Closed
Closed_Date 0000-00-00 00:00:00 2022-01-29 11:45:13
Closed_By zero-24
avatar zero-24 zero-24 - close - 29 Jan 2022
avatar zero-24
zero-24 - comment - 29 Jan 2022

An PR has bee created here please test there again as i have improved the code to more closly follow what we did with the get_headers call: #36886

Will close here as there is a PR.

avatar richard67
richard67 - comment - 30 Jan 2022

@ThomasGarcia05 Please test #36886 . Thanks in advance.

Add a Comment

Login with GitHub to post a comment