@mbabker This is an option that can temporary help until some one fix #18127

@izharaazmi Please test this if you have a time.

@csthomas
Do you mean we could test this by modifying/patching a 3.8.1RC pack, then manually updating a 3.6.0 by Upload & Update that downloaded modified 3.8.1RC using joomlaupdate?

yes,

you can also make changes after install failed, using ftp replace changed file, and refresh install page, and complete installation.

After all you could revert changes in installed joomla 3.8 if you fill uncomfortable with them.

Personally I used git checkout 3.6.0 on my local machine, start to update to 3.8.0 (link to update appears by self in joomla backend), wait until it fails, edit changed lines, refresh installation page, login again and done.

@csthomas We can also setup custom update server. I you wont I can contact @trzepiz on this.

I won't add any custom update server.
You can do what you want.

If that fix will be accepted then it would be useful to add some error message that database query failed, if there is not an installation process.

IMO If someone write patch based on #18127 then this PR should be closed.

Wonderful @mbabker ? You made my day weeks.

I was working on another PR that moves entire upgrade feature in a separate environment, targeting only J4. I discussed this with @infograf768 months ago where file delete on update had conflicts. Don't remember the exact thread.

I will utilise your Update script and this will hopefully make it possible in J3 itself. I'll continue my original work though, still targeting J4.

While I conclude that, this can be be merged without much thought.

@csthomas With custom udpate server this patches of this kind is eves easier to test.

@izharaazmi Any progress with Joomla! half feature?

@wojsmol Sorry, didn't get you! half what?

on Joomla 3.3.0 same error: Fatal error: Call to undefined method JApplicationAdministrator::isClient() in /plugins/system/logout/logout.php on line 48
PHP 5.4.25

@AlexRed Are you using php op_cache or similar?

I think that JApplication is loaded from old filesystem but plugin from new one.
Or some application.php file has not been deleted.

if anyone wants to test with custom update server I post link below:
https://update.joomla.pl/list.xml

no php op_cache or similar, a fresh Joomla installation on xampp in localhost.

@AlexRed Try with custom update server.

on Joomla 3.3.0 with custom update server https://update.joomla.pl/list.xml
error in the URL administrator/index.php?option=com_joomlaupdate&task=update.finalise

Fatal error: Call to undefined method JVersion::isInDevelopmentState() in /administrator/includes/framework.php on line 30

I found that libraries/cms/application/administrator.php still exists when the error occurs on page:
administrator/index.php?option=com_joomlaupdate&task=update.finalise

there is something wrong with calling deleteUnexistingFiles.

yes, all the folder libraries/cms/application still exists :(

@csthomas Maybe move move this folder up in the list?

also other 25 folders still exists in libraries/cms/

found that libraries/cms/application/administrator.php still exists when the error occurs on page:
administrator/index.php?option=com_joomlaupdate&task=update.finalise
there is something wrong with calling deleteUnexistingFiles.

No it is workling like designed .. &task=update.finalise is called before we delete the unexisting files. But when you come from a version that does not send the crf token you need to confirm your admin accout. After that (or the token check worked) we run this code: https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_joomlaupdate/models/default.php#L589 in the model that calls the script with "deleteUnexistingFiles"

@zero-24 Then we bust feast update to 3.6.5, if so then I will update my update server or we can mix two.

No it is workling like designed .. &task=update.finalise is called before we delete the unexisting files.

In Joomla 3.3 yes
In Joomla 3.6 it is not, because restore_finalisation.php is called before and deletes files.

The fastest way will be to disable plugin system logout for a while:)

@csthomas We ten do this is tom steps it is planed for 3.8.1 anyway.

My test with a 3.6.3 and using the custom https://update.joomla.pl/list.xml shows no error and a real fast update here.

@infograf768 Then we must revert the revert here joomla/update.joomla.org@241e171

Can't test with 3.3.3 as it does not install (maybe due to the use of PHP 7)

If upgrading from 3.3.x to 3.6.x works, and from 3.6.x to 3.8.x works. We can just then just setup the update channel that way until we have a better solution. That should be okay as an incremental upgrade procedure.

@infograf768 Then we must revert the revert here joomla/update.joomla.org@241e171

This was done as the decision was made that we don't want to do a updateserver fix. So i have cleaned up the update server PR for 3.8.1 and reverted the commit i have done before.

@izharaazmi Yes but @zero-24 for same reason than this commit joomla/update.joomla.org@241e171 @zero-24 please comment.

@izharaazmi Yes but @zero-24 for same reason than this commit joomla/update.joomla.org@241e171 @zero-24 please comment.

see just the message above ;)

@zero-24 But as you can see for now we don't have a better fix.

cc @mbabker

When do we have the next release scheduled?

@izharaazmi Release scheduled for 4. Oct. 2017.

Then we have about 1 day to release. IMHO tot short time to came with better fix.

@wojsmol Time ist mostly 18h (Vienna/Europa).

But we need @mbabker decision here.

I said I'm fine with an update server change as a temporary fix, not as the permanent solution.

ok, the @zero-24 Please revert joomla/update.joomla.org@241e171 and I will update the update server here during next hour.

and merge this PR for 3.8.1 stable @mbabker

I have tested this item ✅ successfully on e0fb8b8

Will work from 3.6.3 up

A question , since the error there is now suppresed

is it bad to call enqueueMessage() to add some warning of the error message ?

because this change will suppress our desired case, but also can suppress other non desired error cases in the future

@ggppdk For installation it should be suppressed, for later not. But how to check that? $app->input->get('option') == 'com_joomlaupdate'? If you prepare some I can merge it to this PR.

why not use the Logger?

@alikon Don't mention logging in context of 3.8.0. Where we have a failed update but in update log iwe have a successful update message.

1. A true update failure means that the actions involved in the update (package download, extraction, old file removal, and database migrations primarily) failed. This specific issue with the menu table is not an update failure.

2. The update log file may not even be configured to log error entries for non-update related error logging categories (because remember a system error such as this one will be thrown outside the context of what the update should be doing; and unfortunately there is a lot happening that has nothing to do with the update).

@mbabker This wall litle unrated. But on forum after 3.8.0 I see many updates logs where old files was not bean deleted but you have a successful update message.

Custom update server was bead updated. Please test @AlexRed

Without knowing the exact order that log statements are made in, it is difficult to say why their logs are the way they are. Also, without knowing exactly what log categories the update log file is being subscribed to and what log categories errors are being logged to in all parts of the API that work with updates, it's hard knowing if there are any errors occurring and if so how they are being communicated. Either way, the log file is not a definitive "yes this was successful" or "no this failed" thing.

@mbabker Do we should open a new issue for this and investigate?

I said I'm fine with an update server change as a temporary fix, not as the permanent solution.

reveiw for 3.8.1 joomla/update.joomla.org#70 or take joomla/update.joomla.org#69 without that update server changes.

The latest changes should also include a fix for the 3.6.0 => 3.8 Updates (now 3.6.x points to 3.6.5 first and than to 3.8.1).

Update server here: https://www.jah-tz.de/downloads/core/list_sts.xml (still uses 3.8.0 download's)

At a glance, it looks fine. But we need to test and validate that upload and update style updates will work from 3.6.5 to 3.8.0 as well. I have a feeling that's not going to work and we'll need some well crafted documentation to explain that people using that upgrade method will have to update to any 3.7 release before going to 3.8.

@zero-24 Check https://update.joomla.pl/list.xml Shroud be equal to jah-tz.

But we need to test and validate that upload and update style updates will work from 3.6.5 to 3.8.0 as well. I have a feeling that's not going to work

I have just tested a manual 3.6.5 to 3.8.0 update and that worked without problems.

@zero-24 Check Shroud be equal to jah-tz.

No it is not:

I have just tested a manual 3.6.5 to 3.8.0 update and that worked without problems.

When is the second login happening for upload and update then? If it's before the package gets extracted then I guess that'd make sense, but if it's at the same post-update spot then there's something fishy happening.

@zero-24 If you wont I cen update this in 6 minutes.

@zero-24 If you wont I cen update this in 6 minutes.

I have not access to joomla.pl. and jah-tz.de is uptodate. ;)

When is the second login happening for upload and update then? If it's before the package gets extracted then I guess that'd make sense, but if it's at the same post-update spot then there's something fishy happening.

There is no seccond login as the 3.6.5 version of com_joomlaupdate handle that by adding the token.

The second login would only happen on 3.6.0 => 3.8.0/1 (but first the updater points to 3.6.5 anyway)

But there the com_joomlaupdate update comes into action. Saddly we have just later implemented the "please first install the update for com_joomlaupdate" screen.

Wait a minute. I'm lost now. There's supposed to be a second login screen when you do upload and update (I got that on all the sites I updated to 3.8 before we actually released using that method). Or have I finally lost my mind?

On 3.6.1 there was added security token and then you do not need to relogin to update to J3.8.
On 3.6.0 and lower you will have to re-login and this need this PR to update to J3.8.

Wait a minute. I'm lost now. There's supposed to be a second login screen when you do upload and update

Yes that is true. But that is a different login screen. ;)

upload & update => captive login
update without session data => finaliseconfirm

Yes that is true. But that is a different login screen. ;)

upload & update => captive login
update without session data => finaliseconfirm

Right. So what I was saying before is if that captive login screen is happening before the package is extracted, we're fine. But if it comes after the package is extracted (basically the same spot as if the CSRF check fails), then it would hit the same problem people are having with doing "normal" updates from 3.6.0 or earlier. That's what I'm trying to be clear on is all.

@csthomas joomla.pl custom update serser updated on 3.6.5 fixed package is used.

But even without this change update servers ale effectivlelly the same before. Compare targetplatform.

And one more: on J3.5.1 the file restore_finalisation.php has been added that delete old files before relogin.
On J3.5.0 and earlier delete files is done after relogin and this generates problem at #18197 (comment)

@zero-24 @csthomas If any changes will by neeaded on joomla.pl custom update server for this PR and I will update asap.I have a FTP access to https://update.joomla.pl subdomain.

J3.8.1 has been released.