@roland-d you know you're gonna make few people unhappy, right?

Thanks for this.

I think there may be some extra steps to remove for the installation template but it is hard to check on my phone. I can check in the morning

The testing instructions only describe to check if options have gone. They don’t tell to check that things work as before. But this has to be tested, especially by people with shared hosting.

@richard67 You are correct, I am going to update. I realized the same thing when I went to sleep :P

You know that you remove an important security feature?

@HLeithner Care to explain?

Sure,

normally you don't want that your software can manipulate it self. Yes we want it in some situations, f. ex. updates. The reset of the time we maybe want to write to some directories (image uploads, temp dir, cache dir, log dir). All of them doesn't have to execute code.

So if you do you server configuration right you don't want/have write access by CMS to any directory on the server, best case is it doesn't have write access at all.

In combination with restricting access to PHP files in write able directories makes it a bit hard to exploit a file upload exploit.
Something like this:

<Directory images>
     <FilesMatch "(?i)\.(php|phtml)$">
            Order Deny,Allow
            Deny from All
    </FilesMatch>
</Directory>

@nikosdion as this touches com_joomlaupdate can you check the changes please

(note for testers - you can not use com_patchtester to test the changes in the /installation files)

doesnt this section need updating

@roland-d there is still a lot of stuff to remove from the installer or are you intending that Joomla could still be installed with ftp

I disagree 100% with this PR. While philosophically I agree and in theory the idea behind it is solid, it is in disconnect with real world usage of Joomla (sadly) and what we've done to help users understand the security implications.

What we see is plenty of clients stuck on hosting which uses mod_php on Apache, itself running as www-data:www-data. This is still the default configuration of Ubuntu even in 18.04 LTS. As a result the use of the FTP layer is a one-way street for these users.

Of course it has very serious security implications. However, this is not a problem you can solve with technology alone, definitely not by simply removing this feature. It's primarily an education problem, first and foremost with (people who think they are) web hosts. They need to learn to use FastCGI or FPM. Then it's a matter of educating users to avoid hosts where the FTP layer is necessary.

I don't see us solving anything by removing the FTP layer from Joomla!. The people who need it will either stick with an old version of Joomla! or they will choose a different CMS which works on their host. Or both. They do not understand the security implications and we have done a really bad job educating them on that the last 15 years. If there is something we need to change it's educating users. Look at the results after we started warning users about old PHP versions: they are migrating to newer versions of PHP. Users are not suicidal idiots, they just don't know better unless those in the know (us) educate them.

What I would very much prefer is an approach where we educate users and ease them into removing this feature in the future.

For versions 3.x I would recommend adding a post-installation message if the FTP layer is installed informing the users about the dangers of the FTP layer.

In version 4.x we could double down on that, printing a BIG FAT RED WARNING below the header of the backend page with the same information.
people
The copy could read something along those lines:

WARNING! YOU ARE USING JOOMLA'S FTP LAYER
By necessity, this feature stores your FTP credentials in your site's configuration. This could have serious security implications. The FTP layer feature will be removed from Joomla! in version 5.0.
Try disabling this feature. If your site does not work properly after that please contact your host and ask them to use FastCGI or PHP-FPM to run PHP on their server instead of mod_php. This is a faster and more secure alternative.

I would even put the FTP layer controls in an area with a red background and a big warning above it "THESE OPTIONS MAY HAVE AN ADVERSE EFFECT ON YOUR SITE'S SECURITY".

Then in version 5.0 we can remove the FTP layer completely. Of course you could accelerate the timeline and remove the FTP layer in version 4.1. That's a decision that needs to be taken by the production department, not us random contributors on the issue tracker.

It goes without saying that the documentation would need to be updated to dissuade users from enabling the FTP layer at all.

@nikosdion do you have any stats with akeeba/admintools usage with ftp?
The "education" process you suggest can definitely be done as well

Since the use of FTP layer is in the ANGIE application (restoration script) we do not collect that information for privacy reasons. We know empirically that the stream of FTP-related issues hasn't decreased significantly in the last 5 years. I try to educate my users but I'm basically facing the argument "Joomla! comes with an FTP layer and they don't say it's insecure so you are obviously a lazy developer who doesn't want to support it".

Well it was worth asking ;)

I guess security issues would be reduced if the user can not save their ftp username/password. Currently if they choose not to then they will be asked for it each time. So perhaps we should just remove the ability to save them and then each time they will need to enter them AND we can display an "education" message at the same time

That would be the ideal way to handle this. I'd even go one step further to show a warning before people fill in their credentials if their site is not HTTPS. Credentials sent over plain HTTP can be trivially intercepted and are a major security risk. Credentials sent over HTTPS are reasonably secure. I mean that they are only stored in the session which temporarily places them in the database or the server memory (depending on the storage backend) and wiped after the operation completes.

We can also move the warning to 3.9 and 3.10

If you want to use ftp layer, you can stay on that release scheme

I like the eduaction path, doesn't matter if we've it 5 year longer in our repo

@brianteeman Thank you for the review. I have also cleaned up the installation/ folder now. I believe I have cleaned up now what needs to be cleaned up. If not, please tell me.

@mbabker Thank you for the review as well. I wasn't sure about the Factory change, so I have reverted that now.

@HLeithner

normally you don't want that your software can manipulate it self.

What do you think the FTP layer is doing? Giving you just that option. Sending data plain-text across the wire is a security issue if you ask me.

As pointed out by @nikosdion ultimately this is a decision of the Production Team on how to proceed with this.

For the time being I will keep this PR updated, I can't guarantee to do so for 5 years 👅

@roland-d first glance is that its ok - will check further later this weekend

One question remains - what will happen if a user on J3 with ftp enabled upgrades?

@brianteeman I think we should add this to the pre-update checker as a check. You would still be able to update to Joomla 4 but after that your site is most likely broken as in it can't write anything.

If this is something which get considered to be merged on some point (I hope in J4), then we need to add a warning in 3.9 as it is the next release where we can ship deprecation notices. Like that we will give users enough time to make the switch.

What do you think the FTP layer is doing? Giving you just that option. Sending data plain-text across the wire is a security issue if you ask me.

If I use the FTP Layer it only communicate clear text in the isp network (where often connections are cleartext). Depending on the ISP the connection is only localhost so snooping this traffic is really hard.

If you don't save the credentials in the config then its a security feature because the CMS any only write if you allow to do so. Not on a file upload exploit. Alternative it would be ok if we can remote update installations.

Most of compromised websites only have a problem because the webserver can write to the filesystem. If the webserver can't write the attack potential is much lower for normal sites.

Why do you think CPU designers and OS developers spent so much time in the technologies like executable-space protection? Similar reason, if you have a zone where you can't execute but write is good, if you have a zone where you can execute but can't write its good. The combination is hell.

I know many people don't think about security implication on all layers but removing such a feature is a mess. I would remove the credentials and leave the possibility to have a read only webserver.

Surely if the credentials are not saved then you are sending them in the open from your PC to the server??

if you don't have ssl, thats true but in this case you already sending your superuser credentials per plaintext over the wire.

I'm willing to work on an education thing for looking to remove this in J5 or something if we can show clearly the use case is going down. But right now with all the objections raised I don't see the value in removing this given it give or take works

Thanks, a decision is better than no decision.

I wonder if the people who had string objections to the removal of the ftp layer (which doesnt work anyway) in August 2018 still have those strong objections three years later

According to my experience shared hosting has improved meanwhile, so for up to date hosting it should not be necessary anymore. But I have no idea if there are still cases where it could be necessary.
So I can't really say I have a strong opinion in the one or the other direction.

But I have no idea if there are still cases where it could be necessary.

These won't work for other reasons: outdated PHP, Mysql, etc. Honestly, FTP was a thing 15 years ago not anymore...
That said I remember back in 2015-16 we were talking that J4 had to support at least IE8. The conclusion here is that there are many devs that are totally disconnected from the real WEB world...

The conclusion here is that there are many devs that are totally disconnected from the real WEB world...

That's a bit harsh holding them to a comment they made three years ago.

FWIW I couldnt get joomla 3.9.26 to work with J3

FWIW I couldnt get joomla 3.9.26 to work with J3

"... work with J3"? Or "... work with FTP"?

oops
work with ftp #33079

Unfortunately our statistics don't tell us if FTP is used, do they?

I wish we had a kind of global poll where we just could outreach people and ask them.

No they dont. But then @PhilETaylor did volunteer that not one of 67,000 sites he provides services to use it

Even browsers are now removing FTP support.

Even browsers are now removing FTP support.

True.

https://www.ghacks.net/2021/04/16/firefox-90-wont-handle-ftp-sites-anymore/
https://chromestatus.com/feature/6246151319715840