In my own software I use a hidden error div which is displayed when there is an error condition, replacing the progress (which is inside another div to hide it easily). I don't remember the exact reasons we didn't do that with Joomla! Update. It has something to do with the requirements back then but, yeah, it doesn't make a sense having a popup instead of an error div.

Can you provide a fix along these lines as I'm overwhelmed with work these days?

My time is very limited also as its full time daddy time until September 4th, and after a full on day of girly activities old men like me want nothing more than a strong coffee and sleep :-) - but I'll see what I can do.

My main point is that the upgrade should not fail (to a full stop with bad js dialog) because it cannot write 1,2,3 or 4 files - the zip file should be able to be extracted 100% and then a report made/displayed of the files that Joomla was unable to upgrade/overwrite - for the 1&1 example, this would allow a complete upgrade with the exception of /logs/index.html - which would be shown in the error div - allowing the user to manually upload any failed files - or not - up to them.

At the moment there is no feedback on which files the upgrade successfully overwrote before the error - and no reporting on what was still left to do after the error. 20% of the upgrade is complete (for example) before the unwritable file and the upgrade stops at the JS dialog. The version.php has upgraded so user believes - wrongly - that the upgrade was completed in some respect.

The equivalent would be an Akeeba Backup stopping and failing on the first unreadable file - which it doesnt do, it simply logs that fact and continues, and leaves the errors for the user to fix/investigate if and when they what.

Thats what the Joomla upgrade should do too. IMHO

@PhilETaylor In case you get some time for this please mind that there were some changes in Joomla update already merged for 3.5. You can find the relative PR here: #6769

Thanks @dgt41 but if they are merged then that is fine as I'll be working on the latest code available in git anyway :)

I don't think it's fully fair to compare how Akeeba Backup keeps going on one failed file compared to Joomla's upgrade system. One is making a backup of the filesystem and the other is overwriting it. In terms of the UI aspects of it, agree fully that things can be improved. But, I'm pretty sure I remember Nic mentioning this in the last few days too, depending on what files fail to overwrite, Joomla itself can't function in a half upgraded state.

@PhilETaylor the reason I mention this here is that the staging code is with mootools and the 3.5 dev is with jQuery, so pick as starting point the 3.5 repo ????

@mbabker It depends which "half" has not been upgraded :-) and with the 1&1 issue as an example that "half" that is not upgraded is a single blank html file :-)

"Joomla itself can't function in a half upgraded state." = Depends on what half was upgraded :-) and at the moment its impossible to tell - without a myJoomla.com audit or other tool for comparing each individual file against the sites "reported" version. I have even seen people just change the version number in version.php in an attempt to fool their client they had done a complicated upgrade (but thats a different matter :))

So we either

1. Stop on the first un-overwritable file/unwritable folder and kill the upgrade there as we do currently, even though there is a good chance that we could still overwrite ALL files but one (as in the 1&1) example - but with better feedback. The user would need to know what WAS upgraded and what WASNT upgraded to make informed decisions - and be informed with no doubt that their site is currently in some kind of limbo between two versions, with some files old and some files new versions. the WAS/WASNT list above seems overkill when we can do 2) below.

or

2. Continue on all errors on extracting the zip but log all the errors providing feedback about what worked and what did not. Or maybe on what did not overwrite only. It might only be 1 file that failed (or as in my other examples, could be core files the webhost has made unwritable due to hacking). or it could be all files :-)

I vote for the second, because it achieves a better end result for the user, and in the 1&1 case performs a complete upgrade with no further action really needed and no need to change the core default /logs folder path.

The second is definitely better for the end user. But, and here's the "well Michael's just being a negative Nancy again" thing. If we rely on logging that failure data to the filesystem, that might not work (see 1&1 and Plesk in general) so the user has no persistent log of failure data. If we rely on logging to the interface, let's just hope no JavaScript or layout changes break that functionality during an upgrade. If the upgrade failures come in the upgrade component or somewhere that ends up hitting a white screen of death type response, there's probably not going to be a log to work with anyway and that condition is no better than today's behavior in general.

So whatever we do has to be thought through and hopefully without killing Nic's business model

We can easily log to the interface from the ajax response and at the end of the upgrade stay on the same page - as the upgrade is done over ajax there is no moving to another page needed - and a failed upgrade would stay just like it does at the moment on the same page, however with a lot more helpful information for the user :) - as the "upgrade" is actually done "outside" of the Joomla stack mainly (IIRC) so that even if Joomla is destroyed the extract can still happen as the ajax doesn't run through the whole Joomla stack but direct to a php file (IIRC restore.php)

restore.php is 7647 lines long at present - and contains a lot of classes that are duplicates of other stuff @nikosdion has - hence why it might be better to address the "feedback of errors in the returned ajax response" upstream in his other products and then port back into restore.php instead of me messing in 7000+ files :)

Option 1 can be implemented quite easily. The restore.php script has an "ignore most errors" option which does exactly that. We can enable it at the expense of having no idea if something failed.

Option 2 can also be implemented very easily, in parallel with option 1 or completely separately. There's a "debug mode" in restore.php which generates a debug.txt file in the same directory as restore.php. The file logs all operations done on each extracted file and can tell you which file failed to be written to.

Another option that's currently possible through restore.php but NOT in Joomla! is the Hybrid file write mode. You give it the FTP connection information and it automagically figures out if it should write to a file directly or through FTP. This would eliminate half of our problem cases which tend to concentrate on sites suffering from "permissions hell" but have FTP already enabled.

Any solution is easy for me to implement. But I guess it requires a PLT decision. So I'm pinging @wilsonge and @Bakual to kindly request that they put this in the agenda of an upcoming PLT meeting.

So personally I prefer the second option as well. But as well as the log I think we'd need to put those files into the UI as well - if that's ok.

If you think the Hybrid FTP thing will fix a lot of problems I'm DEFINITELY in favour of that (that seems like a win for none of the costs that option 1 and 2 give you).

In terms of PLT - i'll give you a resurrection of a public mailing list so that you can see progress :) https://groups.google.com/forum/#!topic/joomla-wg-production/9rmUgaLgjoE

You can't have it posted back in the UI. It's the kind of change I'm not prepared to make (at least not yet). Having the log is good enough. The Hybrid mode does solve a lot of issues judging by the dramatic decrease in tickets I receive regarding restoration issues due to unwriteable files/folders. By "dramatic" I mean that they stopped being filed, at all, as soon as the Hybrid mode was made the default in Kickstart and Akeeba Backup. Considering that Joomla! Update is pretty much the same, well, it seems like the best solution.

but by promoting the hybrid solution does that mean we go back to the old days of promoting the saving of FTP credentials ? (hope not!) :-)

Not necessarily store anything. The UI has FTP connection fields. If your FTP info is stored it's pre-filled. Otherwise you must fill it in yourself. This information is only ever used if it's filled in AND writing directly to files is not working. It's a fallback.

Hi you created this issue sometime ago but have not provided any code for people to evaluate. As no one else has shown any interest in providing the code and you have not then I am closing this issue at this time. If code is provided (a pull request) it can always be re-examined but please remember the update component in 3.6 is not the same as when you reporte this