Can you see anything in the php / apache error logs? What is happening when you set error reporting to Maximum / development?

Can you see anything in the php / apache error logs? What is happening when you set error reporting to Maximum / development?

[Sun Jan 12 11:33:43.180120 2020] [php7:warn] [pid 6126] [client ::1:45752] PHP Warning: POST Content-Length of 22972015 bytes exceeds the limit of 16777216 bytes in Unknown on line 0, referer: http://localhost/beta1/administrator/index.php?option=com_joomlaupdate

It would be nice to display a message to alert the administrator about the "post_max_size".

The "post_max_size" is missing:

@twister65 You are trying to update a 4.0-beta1-dev nightly build to a later 4.0-beta1-dev nightly build. This is not supported because some existing 4.0 sql update files had to be modified. Updating is currently supported from 3.9 or 3.10 to 4.0-dev and will later be supported when 4.0 has beta status (i.e. beta 1 has been released) between the particular releases, i.e. from beta 1 to beta 2 and so on and to rc 1 and so on. I think I remember we have discussed that already in past.

The name of the current development release, "...beta1-dev" might be confusing because it reads a bit as if 4.0 has already beta status, but this is not the case.

@zero-24 You should know that, too. If in doubt, discuss with @wilsonge .

This might or might not be a reason for this issue. E.g. it may be that if the pre-update nightly build is a bit old, then it may be that code and data structures don't fit together when running the update. But if it is not so old it may be that all is ok.

Just wanted to point that out so you know that the scenario used in this issue is not officially supported.

Suggestion.
Insert before

@richard67 , I tested by changing "post_max_size" in my php.ini file. And it works very well.

The name of the current development release, "...beta1-dev" might be confusing because it reads a bit as if 4.0 has already beta status, but this is not the case.

This is discussed here: #27403 ;)

You are right that this might not be supported but it seams that in this case it has todo with post_max_size.

@twister65

Suggestion.
Insert before

Can you please explain what you want to happen with that hidden value as it seams not checked later one? Maybe we should better add a check for the post_max_size in the upload method you mention?

The "post_max_size" is missing:

joomla-cms/administrator/components/com_joomlaupdate/Model/UpdateModel.php

Lines 1005 to 1006 in c104d5e

	// Is the max upload size too small in php.ini?
	if ($userfile['error'] && ($userfile['error'] == UPLOAD_ERR_INI_SIZE))

Seems to be so.

As far as I remember from other works we might have such checks at several places. I'll have a look how it is done elswhere, maybe it is just missing at this place mentioned by @twister65 .

The MAX_FILE_SIZE hidden field (measured in bytes) must precede the file input field, and its value is the maximum filesize accepted by PHP. This form element should always be used as it saves users the trouble of waiting for a big file being transferred only to find that it was too large and the transfer failed.
https://www.php.net/manual/en/features.file-upload.post-method.php

The variable $maxSize must be converted into bytes.

Also, see UPLOAD_ERR_FORM_SIZE in https://www.php.net/manual/en/features.file-upload.errors.php

With your agreement, I can make a PR for this.

@twister65 As far as I can read up to now you are right. PR is welcome of course.

@twister65 P.S.: If testing instructions in your PR will be based on the issue description above here then please mention the starting version before the update, i.e. which nightly build it was, and if necessary provide a download. Or describe it so that it is clear that the update might not succeed later depending on the starting point version and that it shall be only tested if it correctly starts without the error mentioned in this issue. We have to avoid that people expect this update to succeed in any case, which not work if they use an old alpha version or a too old nightly build.

Sorry, my suggestion does not work. If anyone else has a solution, they are welcome.

admittedly I haven't followed all of this thread but we already have the max_size here

question is if that is the upload_max_filesize or the post_max_size or the minimum of both.

Read the code - its well documented (for a change)
libraries\src\Filesystem\FilesystemHelper.php

@brianteeman What you refer to is function "fileUploadMaxSize", which is used by the package installer "plugins\installer\packageinstaller\tmpl\default.php".

But the Joomla update component to which this issue refers to seems to use function "getMaxUploadSize", which is defined in "libraries\src\Utility\Utility.php" at this place "administrator\components\com_joomlaupdate\tmpl\joomlaupdate\default_upload.php". This should return the minimum of both values in bytes and so should be ok.

@twister65 I just see in the screenshot in your issue description that the max size of 16 MB was shown correctly. So I understand right that the issue now is that nothing happened after click on the button, i.e. no error message was shown?

Yes, we should have an alert message saying that the post_max_size is too small, like the upload_max_filesize.

This either requires to know the file size before uploading or to have an appropriate error code. I'm not sure if I can help with that.

What confuses me is that you did not see at least an error message:

https://github.com/joomla/joomla-cms/blob/4.0-dev/administrator/components/com_joomlaupdate/Model/UpdateModel.php#L1014-L1018

Because it is a php error, and the controller does not switch.

[Sun Jan 12 11:33:43.180120 2020] [php7:warn] [pid 6126] [client ::1:45752] PHP Warning: POST Content-Length of 22972015 bytes exceeds the limit of 16777216 bytes in Unknown on line 0, referer:

the controller does not switch.

Sorry, I don't understand.

It crashes before the controller has control.

Well I'm investigating, but it can be that it becomes too complicated for me. So any help would be welcome. I hope someone reads.

I just see with the extension installer it is the same when trying to upload a zip file larger than the max. size (= minimum of post_max_size and upload_max_filesize). The same "... in Unknown on line 0" in the PHP log.

In J3 it's the same.

@richard67 the filesize can be queried pre-upload.

// Check file size
if ($_FILES["fileToUpload"]["size"] > 500000) {
    echo "Sorry, your file is too large.";
    $uploadOk = 0;
}

so, if we get the php values first, compare those to the file size we can inform pre-update.
https://www.w3schools.com/php/php_file_upload.asp

Not sure if I can implement something, but if so, I won't have time before next weekend.

So if someone is faster: Feel welcome.

@richard67
it Looks like It should be something like this.. I lost the issue #.
HOWEVER, imo, there is no need for 2 functions that due the same thing and this will need to be checked for how media uploads are handled as well.
I think if It was me, I'd add a function in Utility to go "getfileSize()" so its reusable.

				<td>
					<input class="input_box" id="install_package" name="install_package" type="file" size="57" /><br>
					<?php $maxSize = JHtml::_('number.bytes', JUtility::getMaxUploadSize()); ?>
					<?php if( $_FILES["install_package"]["size"] > $maxSize){
						echo JText::sprintf('JGLOBAL_MAXIMUM_UPLOAD_SIZE_LIMIT_EXCEEDED', '&#x200E;' . $maxSize);
						// PUT EXIT CODE HERE
					}
					?>
					<?php echo JText::sprintf('JGLOBAL_MAXIMUM_UPLOAD_SIZE_LIMIT', '&#x200E;' . $maxSize); ?>
				</td>

This is around line 100 at
\administrator\components\com_joomlaupdate\views\default\tmpl\default_upload.php

Will see what I can do but am not sure how fast I can.

Btw., if our update zip has already some 21.9 MB, maybe we should raise the minimum requirement for post_max_size and upload_max_filesize to 32 MB?

As far as I remember we currently show a warning in backend only when one of them is below 8 (or was it 16?).

@N6REJ Problem with your code is that $maxSize is a formatted string, e.g. '16.00 MB', but it needs the pure number of bytes as integer. I guess that's the HTMLHelper (or in your code for J3: JHtml) call doing this. I already know what to do, just wanted to let you and other readers know.

@N6REJ Beside that it's the wrong place for that code. The code is executed when the view is loaded, not when the upload button is selected. The $_FILES["install_package"] isn't known yet at this time.

@zero-24 @HLeithner @wilsonge The issue here is that in both J3 and J4 and in both the "Upload & Install" functionality of the extension installer and the "Upload & Update" functionality of the Joomla Update component you can try to upload a big file which is larger than either post_max_size or upload_max_filesize. This fails then silently, and the message in PHP log doesn't tell where.

The method which is described here https://www.php.net/manual/en/features.file-upload.post-method.php to avoid that doesn't work, I've tested that. The reason is maybe because we use JS to submitt the form, i.e. our button is a button and not an <input type="submit" ...>.

Do you have any idea what we could do?

What we could do is we could check the file size on JS side in the Joomla.submitbuttonUpload() function. Currently we only check if the file name is empty or not at that place.

What I'd like to ask you guys (and other qualified readers of course, too): Is there anything speaking against checking the size of a file to be uploaded on JS side? E.g. any security aspects?

If nothing speaks against it, I could try to implement that check in JS.

Update: I just see that it is not possible in JS to access local files, for security reasons. That makes sense of course.

We can use the Bootstrap form validation method:
https://getbootstrap.com/docs/4.0/components/forms/#validation

The Bootstrap class to validate is form-control-file.

We can use the Bootstrap form validation method:
https://getbootstrap.com/docs/4.0/components/forms/#validation

The Bootstrap class to validate is form-control-file.

Who sais that this will work when a file upload aborts due to post_max_size or upload_max_filesize being exceeded?

Please before you post any stuff found in Google, read that yourself first and check if it relevant before posting.

P.S.: And how shall Bootstrap determine the actual size of the file to be uploaded? With Javascript? That's not possible as far as I could find out.

Anyway, I'll check and test and see if it helps something. But I don't have much hope.

@richard67 See this discussion for possible insights. #18554 (comment)

@Quy Thanks. So if the comment there tells that PHP can't catch the upload_max_size violation, and if @twister65 joined that discussion, then I don't understand why he has opened this issue here. Some kind of amnesia?

I've just tested with J4 what happens when post_max_size is big enough but upload_max_size is violated (can be easily tested with current 4.0 nightly update package and setting upload_max_size to 16M).

In Joomla Update Component, Upload & Update:

In Extensions Installer, Upload Package File:

So it seems in the Joomla Update Component we have an issue with language strings. Will check and make new issue or PR if necessary.

Will also check if the 403 in the Joomla Update Component is related to some mistake in my testing environment or if there is an issue, too.

The extensions installer seems to be fine.

The other way round, if upload_max_size is big enough but post_max_size is violated, the behavior is like descibed at the top in the issue description for the Joomla Update Component, and it is the same for the Extension Installer.

@brianteeman Sorry for disturbing, but maybe you know: Is it right that the Joomla Update Component wants to use language strings of the Extension Installer for the error messages shown at the top of my comment above?

Here the places I've found where com_joomlaupdate uses language strings from com_installer:

./administrator/components/com_joomlaupdate/Model/UpdateModel.php:980:                  throw new \RuntimeException(Text::_('COM_INSTALLER_MSG_INSTALL_WARNINSTALLFILE'), 500);
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:986:                  throw new \RuntimeException(Text::_('COM_INSTALLER_MSG_INSTALL_WARNINSTALLZLIB'), 500);
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:992:                  throw new \RuntimeException(Text::_('COM_INSTALLER_MSG_INSTALL_NO_FILE_SELECTED'), 500);
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:999:                          Text::_('COM_INSTALLER_MSG_INSTALL_WARNINSTALLUPLOADERROR') . '<br>' .
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:1000:                         Text::_('COM_INSTALLER_MSG_WARNINGS_PHPUPLOADNOTSET'),
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:1009:                         Text::_('COM_INSTALLER_MSG_INSTALL_WARNINSTALLUPLOADERROR') . '<br>' . Text::_('COM_INSTALLER_MSG_WARNINGS_SMALLUPLOADSIZE'),
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:1017:                 throw new \RuntimeException(Text::_('COM_INSTALLER_MSG_INSTALL_WARNINSTALLUPLOADERROR'), 500);
./administrator/components/com_joomlaupdate/Model/UpdateModel.php:1037:                 throw new \RuntimeException(Text::_('COM_INSTALLER_MSG_INSTALL_WARNINSTALLUPLOADERROR'), 500);
./administrator/components/com_joomlaupdate/tmpl/joomlaupdate/default.php:20:Text::script('COM_INSTALLER_MSG_INSTALL_PLEASE_SELECT_A_PACKAGE', true);
./administrator/components/com_joomlaupdate/tmpl/joomlaupdate/default_upload.php:19:Text::script('COM_INSTALLER_MSG_INSTALL_PLEASE_SELECT_A_PACKAGE', true);
./administrator/components/com_joomlaupdate/tmpl/joomlaupdate/default_upload.php:36:            <h4 class="alert-heading"><?php echo Text::_('COM_INSTALLER_MSG_WARNINGFURTHERINFO'); ?></h4>
./administrator/components/com_joomlaupdate/tmpl/joomlaupdate/default_upload.php:37:            <p class="mb-0"><?php echo Text::_('COM_INSTALLER_MSG_WARNINGFURTHERINFODESC'); ?></p>

Should we add those texts to administrator/language/en-GB/com_joomlaupdate.ini with suitable names of course and change places found with the grep above to use those new texts?

Update: I've found a few more, e.g. COM_INSTALLER_INSTALL_BUTTON.

look at the code in j3. what is used there?

@brianteeman Same texts used in J3, and behavior is the same, too, as described in my comment above.

@brianteeman Question is what's the right way to fix it. I hope you have an idea.

Basically this shouldn't be to hard to check in Javascript:

c/p from Stackexchange https://stackoverflow.com/questions/3717793/javascript-file-upload-size-validation

<form action="upload" enctype="multipart/form-data" method="post">

    Upload image:
    <input id="image-file" type="file" name="file" />
    <input type="submit" value="Upload" />

    <script type="text/javascript">
        $('#image-file').bind('change', function() {
            alert('This file size is: ' + this.files[0].size/1024/1024 + "MB");
        });
    </script>

</form>

API: https://developer.mozilla.org/en-US/docs/Web/API/File/Using_files_from_web_applications

@HLeithner If it requires HTML5 we can't use it for J3, we can use it only for J4, is this right?

@twister65 @brianteeman @HLeithner and other readers: Assuming we implement a check on JS side using what Harald has linked above, which of the 2 following possible solutions would you prefer?

1. A warning is shown in the form as soon as the user has selected a file which is too big, i.e. the file size check is done on every value change of the file field. In addition we could always show the size of the actually selected file in the form.
2. Nothing happens until the user selects the "Upload" button. Then the file size is checked and if too big an appropriate error message is shown, and the upload is not started.

Please give feedback what you prefer.

And if anyone of you has an idea what is the better way to fix the missing language strings, please let me know: Better add new strings with same text to the language file of com_joomlaupdate, or make com_joomlaupdate correctly load the language strings from com_installer's language file?

@twister65 Are you working on it already? If so, let me know and I wait for your result.

@twister65 I'm working on it, too. Have already working solution, just making it nice.

@twister65 If you want you can have a first look on my PR #27570 . It is still work in progress because I have to add description and testing instructions and so on, and a few things can be improved for sure, but it works for me. It does both: Display file size when file is selected and warning if too big, and display alert message when trying to upload anyway. If that PR is ok I'll do the same for the Extension Installer Upload & Install.

Please test PR #27570 .

If that PR is not a valid approach we can reopen this issue.

@twister65 Please test PR #27570 for the Joomla Update Component and PR #27665 for the Extensions installer, if you can (i.e. if you have npm). Thanks in advance.