No Code Attached Yet
avatar coolcat-creations
coolcat-creations
25 Sep 2019

Steps to reproduce the issue

I have severall .STEP Files to upload into Joomla.
I added stp STP step and STEP to the allowed extension list
I added text/plain to the mime/types

  • Uploading the STEP files one by one works
  • Uploading a ZIP of one STEP File works

- Uploading a ZIP containing all Step files does not work: Error Message: You have tried to upload file(s) that are not safe.

- Uploading a ZIP containing 4 JPG images with 29 MB in total does not work Error Message: You have tried to upload file(s) that are not safe.

  • Uploading a ZIP containing 4 small files works
  • Uploading an unzipped File with 30 MB works

The Error is thrown in administrator/components/com_media/controllers/file.php in Line 67 COM_MEDIA_ERROR_WARNFILENOTSAFE - The error seems wrong at this place because he comment above is telling me that the server did not get any file (?!)

Expected result

The ZIP file should be uploaded

Actual result

I get an error "You have tried to upload file(s) that are not safe."

System information (as much as possible)

PHP Version 7.2.22-1+ubuntu16.04.1+deb.sury.org+1
Joomla 3.9.12 (but it did not work with 3.9.11 either)
Package Size was only 30 MB

max_execution_time | 240
max_input_time | 120
post_max_size | 600M
fileinfo support enabled

Time from click on Upload to error message: 55 Seconds (measured with timer)

Additional comments

avatar coolcat-creations coolcat-creations - open - 25 Sep 2019
avatar joomla-cms-bot joomla-cms-bot - change - 25 Sep 2019
Labels Added: ?
avatar joomla-cms-bot joomla-cms-bot - labeled - 25 Sep 2019
avatar HLeithner HLeithner - change - 25 Sep 2019
Title
[3.9.12] Mediamanager: uploading bigger zip archives / unsafe file Message
[3.9] Mediamanager: uploading bigger zip archives / unsafe file Message
avatar HLeithner HLeithner - edited - 25 Sep 2019
avatar brianteeman
brianteeman - comment - 25 Sep 2019

Is it possible for you to share the actual file that you are trying to upload.

I suspect that its a false positive similar to #15563 but without the exact file you are trying to upload its hard to be certain.

avatar coolcat-creations
coolcat-creations - comment - 25 Sep 2019

I think it's quite unimportant which file (stp, step or other files) because I took also a ZIP with some JPEGS inside and it did not work either. And the test throwing the error is looking for if there is a file and not what the file is like (I think the error message is wrong and the error is caused by something else)
Here is a ZIP Package which fails to be uploaded in mediamanager: https://coolcat-creations.com/imagetest.zip

Also failed on another Server and another Website

avatar brianteeman
brianteeman - comment - 25 Sep 2019

What settings have you set in the media manager to allow zip files

avatar coolcat-creations
coolcat-creations - comment - 25 Sep 2019

Upload of Zip Files works without Problems. But not with that one... (When the contents are larger)
Settings are: image/jpeg,image/gif,image/png,image/bmp,application/x-shockwave-flash,application/msword,application/excel,application/pdf,application/powerpoint,text/plain,application/x-zip, application/zip

avatar SharkyKZ
SharkyKZ - comment - 25 Sep 2019

This is coming from input filter. The linked zip file contains .py in its data which sets off this filter rule:

'forbidden_extensions' => array(
'php', 'phps', 'pht', 'phtml', 'php3', 'php4', 'php5', 'php6', 'php7', 'phar', 'inc', 'pl', 'cgi', 'fcgi', 'java', 'jar', 'py',
),

@zero-24 @SniperSister

avatar coolcat-creations
coolcat-creations - comment - 25 Sep 2019

@SharkyKZ What is .py? Because I use cleanarchiver - there should be only the jpgs inside? Thank you.

avatar Quy
Quy - comment - 25 Sep 2019

Please read the initial post of #15563 that explains the issue you are experiencing.

avatar brianteeman
brianteeman - comment - 25 Sep 2019

confirmed - there are 14 instances of .py and six of .pl in the data when read as a stream

avatar coolcat-creations
coolcat-creations - comment - 25 Sep 2019

duhhh... thank you both for explaining! So there is no way to fix this issue? maybe something like Apples "upload anyway, I am sure" as a superadmin ?

avatar brianteeman
brianteeman - comment - 25 Sep 2019

The equivalent is to use fto

avatar coolcat-creations
coolcat-creations - comment - 25 Sep 2019

Thats unfortunately not a solution because there is no FTP allowed and most users are not allowed to use the SSH connection... :-(

avatar brianteeman
brianteeman - comment - 25 Sep 2019

There is no other option

avatar SharkyKZ
SharkyKZ - comment - 26 Sep 2019

So there is no way to fix this issue? maybe something like Apples "upload anyway, I am sure" as a superadmin ?

Technically this is possible. Super admins can upload unfiltered files in other places anyways. But I'd wait for security experts to comment on this topic.

avatar SharkyKZ
SharkyKZ - comment - 9 Sep 2020

@joomla/security any comments?

avatar rozniak
rozniak - comment - 9 Sep 2020

Sorry for the duplicate report - I didn't get any hits when I searched for the issue originally (would've saved me a lot of time too...) 😩

I'm not intimately familiar with Joomla!, I came across this when troubleshooting a particular ZIP on a client site - I kinda want to ask (excuse me if it's a stupid question) - why exactly is ".py" being searched for via string inside a ZIP file? Seems like a weird way of filtering out dodgy files?

avatar brianteeman
brianteeman - comment - 10 Sep 2020

.py indicates a file in the python programming language

avatar SharkyKZ
SharkyKZ - comment - 10 Sep 2020

Yes, but why check for it in raw data instead of opening the archive and inspecting filenames?

avatar bembelimen
bembelimen - comment - 10 Sep 2020

The "file scanner" has several issues and one of them is the (in my opinion) wrong handling of ZIP files. Probably for 4.0 there is an external library somewhere to do the job?

avatar rozniak
rozniak - comment - 10 Sep 2020

I am more than happy to look into a proper solution to these false-positives, I really would just like to know what the goal of this particular bit of code is, so that we can find a better way of achieving it without this issue.

I understand it is scanning through these particular file extensions (.zip, .bz2 etc.) for instances of other file extensions (.py, php etc.), but what I do not know is why this is done (what is the security vulnerability) nor why in this particular way (as mentioned by @SharkyKZ , surely if it's looking at archive files in particular... it should inspect them like archives and not searching for string-in-string?)

avatar bembelimen
bembelimen - comment - 10 Sep 2020

The goal is very easy: prevent the upload of an archive which contains one of the forbitten files in it.

Perhaps the list should be aligned with this list: https://github.com/joomla/joomla-cms/blob/staging/libraries/src/Helper/MediaHelper.php#L178-L181

avatar rozniak
rozniak - comment - 10 Sep 2020

Sure, but why is that important to do on a ZIP? What is it preventing?

avatar bembelimen
bembelimen - comment - 10 Sep 2020

Not sure if it's still possible, but there are self-extracting executable available.

avatar rozniak
rozniak - comment - 10 Sep 2020

If that's the case then I am not sure that scanning file extensions within the ZIP would be enough solve that problem?

avatar thomaslanger
thomaslanger - comment - 10 Dec 2020

I have the same problem with a ZIP-File containing <?
I don't use the Mediamanager. It is not a Mediamanager only bug/problem.
I disable file checking on ZIP-Files in my component code with

$safeFileOptions['php_ext_content_extensions'] = array('rar', 'tar', 'gz', 'tgz', 'bz2', 'tbz', 'jpa');
if (JFile::upload($src, $dest, false, false, $safeFileOptions))
{
// your code here
}

In this case of false positiv results, the current file checking make no sens.

avatar PhilETaylor
PhilETaylor - comment - 16 Dec 2020

Yes, but why check for it in raw data instead of opening the archive and inspecting filenames?

Sure, but why is that important to do on a ZIP?

Because - Security.

The institutional memory for the security issue this fixed, seems to have been lost from the project :-(

EDIT: from my very very bad memory I think the security issue was something like, you could upload hack.php by calling it hack.zip on servers where mime type sniffing failed, hack.zip would be a plain text php file just named with zip extension. I forget how you could then go on to exploit the site from that situation... was it not Nicolas that worked on this, I can't remember.

avatar brianteeman
brianteeman - comment - 26 Aug 2022

Thank you for raising this issue.

Joomla 3 is now in security only mode with no further bug fixes or new features.

As this issue doesn't relate to Joomla 4 it will now been closed.

If we are mistaken and this does apply to Joomla 4 please open a new issue (and reference this one if you wish) with updated details for testing in Joomla 4.
cc @zero-24

avatar Quy Quy - change - 26 Aug 2022
Status New Closed
Closed_Date 0000-00-00 00:00:00 2022-08-26 22:54:39
Closed_By Quy
Labels Added: No Code Attached Yet
Removed: ?
avatar Quy Quy - close - 26 Aug 2022

Add a Comment

Login with GitHub to post a comment