Does this mean that you have found a way to test this?

Yes, I've created a VM with a deliberately botched installation. Installed Ubuntu Server, then apache, mysql, php and vsftpd. Added a user "ftpuser" and set /var/www as its home dir, changed the owner to ftpuser and set permissions to 755. Unfortunately I don't think I can share my VM somehow... But that triggered the permissions warning and stops the installation until you have fixed this.

This draft currently is still missing the possibility of simply displaying the configuration.php at the end and some cleanup.

Does this mean that you have found a way to test this?
Yes, I've created a VM with a deliberately botched installation.

er..? I spent half a day providing a complete dockerized test environment for this specific issue... I guess no one used it even after I was asked to do that.

@PhilETaylor I saw your docker container, but for my coding, I needed something else and rolled my own. For testing, your container is a good solution. Since the code currently does not have any of the features to get around an unwritable filesystem, I did my own solution.

So, this should be ready now. Please test! :-)

With invalid ftp details and using the verify settings button

With valid ftp details and using the verify settings button

After entering the mysql details and waiting for it to complete you are silently returned to the first page as shown below

@PhilETaylor please create a new issue

Installing to a remote mysql server did not go through the expected security checks

Installing to a remote mysql server did not go through the expected security checks

If you are using docker, then this can be correct and fine. There is a flag in the base Joomla docker images to bypass the security check (as its NORMAL in docker to not be using a localhost mysql)

see environment JOOMLA_INSTALLATION_DISABLE_LOCALHOST_CHECK here: https://github.com/joomla-docker/docker-joomla/blob/99611523032a31c85cfe37b5dfa750925106adc8/php7.3/apache/Dockerfile#L5

I have tested this item ? unsuccessfully on 1d6227e

Are the errors in this going to be addressed or should we simply remove the ftp option.

yes, these are going to be fixed eventually.

Eventually is a wonderful word. It can mean please wait it will be done next week. It could mean I will look at this in two years time when we are hopefully ready to release 5.0

Please check if this fixes your concerns.

So I get a bit further now (i think)
after entering the mysql details and clicking continue I get a 500 error

Warning: include_once(/var/www/html/4/configuration.php): failed to open stream: Permission denied in /var/www/html/4/libraries/loader.php on line 265

Warning: include_once(): Failed opening '/var/www/html/4/configuration.php' for inclusion (include_path='.:/usr/local/lib/php') in /var/www/html/4/libraries/loader.php on line 265

note that the configuration.php is created and all the values look correct

@Hackwar can you have a look at Brian comments

There is another issue which I am sure will occur is with the creation of the autoload_psr4.php file but as I havent been able to get as far as a complete install using ftp I cannot be 100% certain. However I cant see how it can be created on a site that requires ftp to create the configuration.php

I have tested this item ? unsuccessfully on 183c748

Forgot to record my tests before

there really is no point in having ftp during installation if you cant update with ftp

There is another issue which I am sure will occur is with the creation of the autoload_psr4.php file

This is written using the Filesystem API File::write() method, which is "FTP Aware".

There is another issue which I am sure will occur is with the creation of the autoload_psr4.php file

However, without any FTP Credentials yet stored, this file cannot be written when first visiting /installation/index.php

This comment is about 4.0-dev and not this PR, which is still downloading for me at the moment

The " Prebuilt packages are available for download." are not available for this PR either...

https://ci.joomla.org/artifacts/joomla/joomla-cms/4.0-dev/31857/downloads/40857 is a 404

I have tested this item ? unsuccessfully on 183c748

just like 4.0-dev, this PR doesn't address the initial save of the "autoload_psr4" when /installation/index.php is loaded, resulting in an error message

I have tested this item ? unsuccessfully on 183c748

If I allow the autoload_psr4 file to be written directly (by 777 the cache folder "for testing only") I get a little further...

If I then select "Manually" and attempt to install Joomla 4, I get to the point where its actually inserting data to the database and then I get an error

There is another issue which I am sure will occur is with the creation of the autoload_psr4.php file

However, without any FTP Credentials yet stored, this file cannot be written when first visiting /installation/index.php

Exactly my point - but its actually worse than that as FTP credentials are never saved now and you have to re-enter them each time they are needed. So this can never work for autoload

@rdeutz its been a month now. Is there really any point in pretending that this will ever be resolved. Looks like all release blockers are stuck in the same place

I have tested this item ? unsuccessfully

If I allow the autoload_psr4 file to be written directly (by 777 the cache folder "for testing only") I get a little further...

If I then select "verify FTP" and attempt to install Joomla 4... I then replicate @brianteeman issue where Joomla "appears" to install perfectly and it writes configuration.php (the file is there and its contents are valid and it contains the FTP credentials and ftp path correctly..

   4 -rw-------   1 www-ftp    2198 Apr 22 16:27 configuration.php

But then Joomla cannot read it as the PERMISSIONS are too restrictive:

If I manually change the permissions to 644 then refresh Joomla can then load to the final installer page:

Clicking remove installation folder button correctly uses FTP to remove the folder.

#33079 (comment)

I guess jumping on the FLoC bandwagon is more important than releasing joomla

Since @brianteeman asked so nicely: I tested this code on my machine with FTP and it works fine. With the information given by you, I can't debug this. The FTP user could be misconfigured to only allow reading for himself and not for other users, like the user of the webserver. Thus I would need more information to move this forward. You say that the configuration.php is created and seems to have the right content, so my assumption would be, that your FTP server is configured to write the files with 600 permissions or something similar.

I tested this code on my machine with FTP and it works fine.

I tested this code on my machine with FTP and it doesn't work fine. I reported some issues here #31857 (comment) and here #31857 (comment) which you have failed to acknowledge or address

The FTP user could be misconfigured to only allow reading for himself and not for other users,

Thats not misconfigured - that is standard linux permissions and standard mass-market FTP accounts.

You say that the configuration.php is created and seems to have the right content, so my assumption would be, that your FTP server is configured to write the files with 600 permissions or something similar.

.... and Joomla fails to take the FTP layer into consideration, thus leaving the configuration.php permissions at the standard permissions for writing files by FTP (Which is OWNED then by the FTP user that logged in) which doesn't allow READ by the web server process user... you know: STANDARD LINUX PERMISSIONS.... .

The problem is that Joomla fails to take this into account - this is a bug in Joomla, not in the way the files are written by the FTP server.

There is code in the writeConfigFile method that attempts to make files writable/not writable - but only if FTP layer is disabled. There is no concept of ensuring they are readable too.

Joomla doesn't take into account that the newly FTP Uploaded file is OWNED by the FTP User, and therefore might have permissions used might be wrong to allow READING by other users (I.e the web server process user.)

As stated before I used the docker container provided by @PhilETaylor no idea what extra information you need

He openly stated he did not use it

This draft currently is still missing the possibility of simply displaying the configuration.php at the end and some cleanup.

And thus incomplete and cannot be merged as doesn't resolve this issue.

Yes, I've created a VM with a deliberately botched installation. Installed Ubuntu Server, then apache, mysql, php and vsftpd.

That is not a "botched installation" - that is a standard installation that many many people use.

Ok, so you already did all the work to debug this and apparently also found the code and the place to fix this. You seem to have fun letting me do all that work again instead of creating a PR against my branch here to move us forward. I don't have time for that, so here we are. Again: Teaches me to volunteer to execute decisions the PLT has made.

Ok, so you already did all the work to debug this and apparently also found the code and the place to fix this.

Excuse me? No. I tested the PR as presented. Just like any other PR. I did not "find the code" to resolve your bugs in your PR I just commented knowing (and already spending CONSIDERABLE TIME with the FTP layer since it was designed)...

You really have an attitude issue and dont seem to be able to cope with people reporting the genuine issues with the PR you present. That is concerning.

Assuming and demanding that testers create PR's against your branch is unreasonable. Testing and creating PRs are two different tasks, with different time constraints and commitments.

All I have done is test the code you put out into the public - for testing - and reported the issues found.

Teaches me to volunteer to execute decisions the PLT has made.

Decisions that seem to be made and communicated in private...

The FTP user could be misconfigured to only allow reading for himself and not for other users, like the user of the webserver.

Its clear you do not understand the whole reason the FTP Layer exists, if you believe that this is a misconfiguration.

What you are describing as a "misconfiguration", a "botched installation" - is literally the whole point. Its NEITHER. Its a standard way of running a server, where the FTP user and the Apache/PHP user have different roles and one cannot write to the files/folders of the other.

So I have EXTENSIVELY AND COMPREHENSIVELY Tested this PR today. We are talking several hours of set up (several virtual machines, docker, and real life servers at Digital Ocean), testing various senarious, and reporting back, as well as PR creation for issues found. 6 coffees consumed. Half a package of biscuits eaten.

The issues encounted in testing this PR were (and not all are related to this PRs content)

1. Before you can even test this PR - autoload_psr4.php cannot be created - as commented here #31857 (comment) right from the beginning it is impossible to proceed because Joomla cannot write the /administrator/cache/autoload_psr4.php file as it cannot write to the folder as it cannot use the FTP layer as no FTP credentials have yet been provided. I have already spent considerable time debugging that and providing the PR for it here #33263

2. ConfigurationModel.php uses is_writable several times which returns factually incorrect information, because is_writable is not FTP aware, whereas File::write is FTP aware. This is a global issue and documented in #33238 already. What we need is a Folder::is_writable method in the Filesystem layer that is FTP aware (we dont have that yet, no PR)

3. Once configuration.php is written by the FTP Layer, the permissions are 0600. This means the configuration.php is unreadable by PHP and an Internal Server Error happens on the next page reload. The Filesystem API only writes the file, and doesnt check if the written file is actually readable by PHP after being written.

4. If I "mess" with permissions on /administrator/cache I can get the installation to throw an untranslated JLIB_CLIENT_ERROR_JFTP_CLIENT_BAD_RESPONSE - Now PR here #33284

5. After installation, if I click on the Open Site button I get ON THE FIRST PAGE LOAD ONLY "Joomla\Filesystem\File::move: Rename failed." Ive seen this before in Joomla 4.0-dev several years ago but was told I was going mad (cannot find that reference now!) The error is caused by logrotation (see new image in #33283) and is because the logrotation uses the FileSystem API from the Joomla framework which is not FTP aware instead of the CMS Filesystem package! Now PR here: #33283

6. There are many J* prefixed class names that cause issues in installation/src/Controller/InstallationController.php, installation/src/Model/ChecksModel.php and installation/tmpl/configuration/default.php, these need the J* prefix removing so that the Class can be resolved by the use import already there. New PR here Hackwar#49

7. If you select "Manually create file" at the start of the process, then at the end of the process no configuration.php is displayed to you to copy and create manually, and no button to then chech and proceed. Instead you are sent back to the start of the installation process. This apparently is by design currently as this PR Author states: "This draft currently is still missing the possibility of simply displaying the configuration.php at the end and some cleanup." - I do not know what additional cleanup is needed.

So out of these, the author of this PR needs only to address 3 in order to get a successful test result flagged by me. Ive done or are doing the rest.

But does it work in Joomla 3 right now. Because if its broken then there really is little point as it means no one is using it

But does it work in Joomla 3 right now. Because if its broken then there really is little point as it means no one is using it

Out of scope of this PR :) We could fix it and call it a new flagship feature of Joomla 4 ;-)

The Production Team decided to drop the FTP layer. See #33390 (comment) for further comments.