? ? ? Pending

User tests: Successful: Unsuccessful:

avatar richard67
richard67
1 Jul 2020

Closes #29764 .

Solves also server side part of issue #29763 . Client side part of that issue has been solved with PR #29877 .

Summary of Changes

This Pull Request (PR) adds a MIME type check for files uploaded with the "Upload & Update" function of the Joomla Update Component.

Currently the "Upload & Update" function supports only zip files, and so only MIME type application/zip will be allowed here. No idea what the other update packages (tar.gz, tar.bz2) are good for. There is no update from folder option or update channel for which they could be used.

In opposite to the MIME type checks in the media manager, this check here executes the 2nd check using the finfo_... function not only when the mime_content_type function is not available, it also executes the 2nd check when the 1st check failed with an exception. This makes more safe against false alarms e.g. due to problems on that server with the magic file used by mime_content_type.

Requests for Comments

RFC 1: Should I add a configuration option to the Joomla Update Component for switching off this new MIME type check on hosts where there is some problem with MIME type detection, and extend the language strings shown in case of error or wrong MIME type by corresponding hint that this might be a false alarm on some hosts and can be switched off in the options?

RFC 2: Should the same check also added for files downloaded by the "Live Update" function of the Joomla Update Component?

RFC 3: Shall such check(s) also be added to the Extensions Installer?

RFC 4: Currently many exceptions raised in the function modified by this PRs use language strings from com_installer, except of the new MIME type check added by this PR, which already uses com_joomlaupdate strings. The com_installer strings are currently not translated. This has to be fixed with another PR. How shall it be fixed: Load the com_installer language file, or copy the strings to the com_joomlaupdate language file and rename them from COM_INSTALLER_MSG_... to COM_JOOMLAUPDATE_MSG_...? And is it a know issue? I didn't find one, but maybe I've missed it.
=> For fixing the untranslated language strings see PR #29899 .

Testing Instructions

  1. On a clean current staging or recent 3.9 nightly build or a 3.9.19, login to backend and go to "Components -> Joomla Update".

  2. Go to tab "Upload & Update" and select a file for upload which is not a zip file, e.g. one of the "tar.gz" or "tar.bz2" update packages.
    snap-1

  3. Use the "Upload & Install" button to start the upload.
    Result: You see the login screen for the next update step:
    snap-2

  4. Enter the super admin's credentials and use the "Install" button to continue.
    Result: You get an error alert at the beginning of the upload:
    snap-3

  5. Use the "OK" button to close the alert.
    Result: You are stuck at the upload start without any progress, see issue #29764 .

  6. Use the "Back" button of your broswer to get rid of that page.

  7. Apply the patch of this PR.

  8. Apply the patch of PR #29881 . That PR is not required for this one here but makes testing easier.

  9. Repeat steps 2 and 3.
    Result: See section "Expected result AFTER applying this Pull Request" below.

Actual result BEFORE applying this Pull Request

See testing instructions above. At the end you stick with:
snap-4

See also issue #29764 .

Expected result AFTER applying this Pull Request

If both PRs this one here and the PR #29881 have been applied:
snap-7

If only this PR has been applied:
snap-5

snap-6

Documentation Changes Required

None as far as I know.

avatar richard67 richard67 - open - 1 Jul 2020
avatar richard67 richard67 - change - 1 Jul 2020
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 1 Jul 2020
Category Administration com_joomlaupdate Language & Strings
avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67
richard67 - comment - 1 Jul 2020

@brianteeman As I value your wide knowledge on UX: Could you check the 3 "RFC" items in the Summary of Changes of this PR and let me know your opinion? Thanks in advance.

avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar brianteeman
brianteeman - comment - 1 Jul 2020

No idea what the other update packages (tar.gz, tar.bz2) are good for.

They are just alternate archive formats

avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67
richard67 - comment - 1 Jul 2020

@brianteeman I know what these file types are, please don't treat me as if I was an idiot.

But as the Joomla Update Component cannot unpack them, they are useless, that's what I meant.

Update: And in opposite to the extensions installer we don't have an "Update from folder" here where they could be used.

avatar richard67
richard67 - comment - 1 Jul 2020

Meanwhile I have 4 "RFC" items in the description. Feedback is welcome.

avatar zero-24
zero-24 - comment - 1 Jul 2020

RFC 1: Should I add a configuration option to the Joomla Update Component for switching off this new MIME type check on hosts where there is some problem with MIME type detection, and extend the language strings shown in case of error or wrong MIME type by corresponding hint that this might be a false alarm on some hosts and can be switched off in the options?

I personally would say yes so we don't break updates for broken hosts.

RFC 2: Should the same check also added for files downloaded by the "Live Update" function of the Joomla Update Component?

Feel free to do that there should not be an issue as this is something we control ourself.

RFC 3: Shall such check(s) also be added to the Extensions Installer?

I would say yes with a dedicated option to switch it off. We should check whether the other file formats are actually supported there.

RFC 4: Currently all these exceptions raised in the function modified by this PRs use language strings from com_installer. These strings are currently not translated. How shall I fix it: Load the com_installer language file, or copy the strings to the com_joomlaupdate language file and rename them from COM_INSTALLER_MSG_... to COM_JOOMLAUPDATE_MSG_...?

Please use dedicated strings from com_joomlaupdate as this component should not depend on com_installer.

avatar richard67
richard67 - comment - 1 Jul 2020

@brianteeman Sorry, was not offended. Regarding those other file types, I think about implementing support for them so they can be unpacked, the tar part of it is the arts, not the gz or bz2, those are easy ;-)

avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67 richard67 - change - 1 Jul 2020
Title
[RFC] Add MIME type check for upload files to com_joomlaupdate
[RFC] [WiP] Add MIME type check for upload files to com_joomlaupdate
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67
richard67 - comment - 1 Jul 2020

Option to switch the MIME type check off (see "RFC 1" in the description) will be added later to this PR when I get more feedback that it's a good idea. I personally think so, but maybe many others disagree.

avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar brianteeman
brianteeman - comment - 1 Jul 2020

The error message is very geeky. Does it really need to say anything more than "wrong file"

avatar richard67
richard67 - comment - 1 Jul 2020

@brianteeman Maybe "The file type is not suitable for a Joomla update package." and "The file type could not be determined."?

And what do you think about "RFC 1" to add an option to switch off that check?

avatar richard67 richard67 - change - 1 Jul 2020
Labels Added: ? ? ?
avatar richard67
richard67 - comment - 1 Jul 2020

@brianteeman I've updated the language stings, also the names. Are they better now? And what do you think about "RFC 1" to add an option to switch off that check?

avatar richard67 richard67 - change - 1 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 1 Jul 2020
avatar richard67
richard67 - comment - 1 Jul 2020

@HLeithner What do you think: Should I add an option to switch the new check off, like it can be seen with the diff shown by the followint link? richard67/joomla-cms@staging-add-upload-file-mime-type-check-to-joomlaupdate...richard67:staging-add-upload-file-mime-type-check-to-joomlaupdate-2

avatar HLeithner
HLeithner - comment - 2 Jul 2020

I'm not sure if this over engineering is really needed, wouldn't a simple <input type="file" accept=".zip"> be enough? I mean if someone by pass this restriction our second step will fail with an error anyway.

avatar richard67 richard67 - change - 2 Jul 2020
The description was changed
avatar richard67 richard67 - edited - 2 Jul 2020
avatar richard67
richard67 - comment - 2 Jul 2020

For fixing the untranslated language strings mentioned with "RFC 4" see PR #29899 .

avatar richard67
richard67 - comment - 2 Jul 2020

I'm not sure if this over engineering is really needed, wouldn't a simple <input type="file" accept=".zip"> be enough? I mean if someone by pass this restriction our second step will fail with an error anyway.

@HLeithner Not really, there is not a useful error shown, ses issue #29764 and also here the actual result without the PR. Or which second step do you mean?

avatar HLeithner
HLeithner - comment - 2 Jul 2020

Actually if you set the accept attribute you can't select the wrong file type.

avatar richard67
richard67 - comment - 2 Jul 2020

Actually if you set the accept attribute you can't select the wrong file type.

@HLeithner Again wrong. Read the description of this PR and check with Firefox: You get a pre-selected filter which you can de-select, see the screenshots in the description.

avatar richard67
richard67 - comment - 2 Jul 2020

@HLeithner Sorry, I made a mistake in my previous comment: You should read the description of PR #29877 , not of this one here. It clearly tells that it is up to the browsers what they do with the "accept" attribute, and most browsers allow to bypass it.

avatar HLeithner
HLeithner - comment - 2 Jul 2020

Yes I already noticed this, my simple message is please don't break the updater because of a cosmetic thing.

avatar brianteeman
brianteeman - comment - 2 Jul 2020

all of this seems like a LOT of work to address a fairly non-existent problem. Definitely more important things to work on

avatar richard67
richard67 - comment - 2 Jul 2020

@HLeithner It is not a cosmetic thing only, and for not breaking it I suggested to add the option to switch the new file type check off, and I asked you to check it. Is that not sufficient to make sure it doesn't break anything?

@brianteeman It seems to me you did read only half of the description and the referred issues.

avatar brianteeman
brianteeman - comment - 2 Jul 2020

Nope I have followed it all

avatar HLeithner
HLeithner - comment - 2 Jul 2020

Looking at getMimeType() you have a very good chance to break broken hosting setups, if you just want to get a error message popup then please do it at this dialog and not introducing a new option for our failure to do the check 100% failsafe.

avatar richard67
richard67 - comment - 2 Jul 2020

I have no problem to close this PR if it is not wanted for staging. I just was asked to make that for 3.10-dev, and I thought it doesn't make sense to have it there and in 4 only. And I think if it is rejected for staging, it should also not be done in 3.10 or 4.

If someone makes a decision I am happy with it.

I only don't want to be torn apart between different opinions.

avatar zero-24
zero-24 - comment - 2 Jul 2020

our failure to do the check 100% failsafe.

is there a 100% failsafe solution? The only thing i can think of would be not aborting but adding an warning when the mine is not matched I'm not sure whether that makes sense at the end.

avatar HLeithner
HLeithner - comment - 2 Jul 2020

If you use the exact same solution like in the update process that fails then it's at least the same problem but now it's checks the type by 2 functions that may fail or doesn't exists at all and would return false.

To make it short I don't want to have this in j3.9 and as brian said there are much more important issues in j4 to fix.

avatar richard67
richard67 - comment - 2 Jul 2020

To make it short I don't want to have this in j3.9

@HLeithner I am absolutely ok with that and happy there is a decision.

@zero-24 I hope you will understand that if you want something like this PR in 3.10, I don't want to make the PR, especially not if it turns out that later it will also not be wanted in J4, because that would be very inconsistent at the end. I am ok with it if you want to make such PR, and you may use my code. I'll wait a bit with deleting the branch but will close this PR for now.

I'm not disappointed or so, all is fine. I wanna fix things and not build personal monuments.

avatar richard67 richard67 - change - 2 Jul 2020
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2020-07-02 11:32:25
Closed_By richard67
avatar richard67 richard67 - close - 2 Jul 2020
avatar zero-24
zero-24 - comment - 2 Jul 2020

:(

avatar richard67
richard67 - comment - 2 Jul 2020

@zero-24 If George wants that for J4, too, someone could make a PR in for 3.10, but if George doesn't want it it would not make sense to do that in my opinion.

Add a Comment

Login with GitHub to post a comment