can you please take a Look at #18003 if this Discussion help?

https://forum.joomla.org/viewtopic.php?f=9&t=954795&p=3493297#p3493297

Looks like the generic issue updating from 3.5.1

@wilsonge I suspect it is related to #17967

Just shooting from the hip, you're probably going to hit that error if you are upgrading from a version that doesn't have the client_id column to a version that does and you hit the second login screen in the update component (either through upload and update or the CSRF check fails; 3.5.1 will hit the latter 100% of the time).

Confirmed probably this post of the forums will help? https://forum.joomla.org/viewtopic.php?f=9&t=954795&p=3493308#p3493299

The only fix I can think of is to force everyone on 3.6.4 or earlier to update to the 3.6.5 release first, then update from 3.6.5 to current.

Even then, it's a hack and not a fix, but I don't know if we can actually fix it because the framework is not in a reliable state; that still leaves the upload and update option off the table to go from 3.6.5 or earlier to 3.7.0 or later (that feature is in 3.6, right?) because the client_id column is added in 3.7.0 and if you hit the second login screen for any reason that's where you're going to run into trouble.

for the missed client_id column issue, the forum proposed fix should work
running https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_admin/sql/updates/mysql/3.7.0-2017-01-17.sql should fix

so we should seriously think about some kind of "upgrade fixed path"

The update queries are in the change deltas, we aren't missing anything.

The problem is if you hit that second login screen, the framework is in a mixed state between updates; the filesystem has been (mostly) updated but none of the steps in our PHP update script (which includes the database updates) have been run. And since that screen has the full admin UI, including the menu, the menu table gets queried and the data rendered by the module.

Running any part of the update "out of sequence" needs to be handled carefully. If you're hitting that second screen you've either hit a CSRF validation error (where we really shouldn't be running ANY update steps) or you're on the upload and update path and that one should not error out when hitting that screen.

the forced updated path should help in this scenario or i'm missing ?

There is not a "forced update path" that can be used without bypassing security mechanisms that are in place.

So what we do people since this is a MAJOR ISSUE? Can we get our brains together?

Is it not possible to deliver an update depending on the version of Joomla that is being updated ? Or is there going to have to be a 3.8.1 update to deal with the issue ? I agree with Leo this is a MAJOR ISSUE

What about copy this file https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_admin/sql/updates/mysql/3.7.0-2017-01-17.sql
to 3.8.1-2017-01-17.sql for a new joomla 3.8.1

A 3.8.1 release cannot fix this issue without bypassing security mechanisms that exist in the update component.

As I have already said, the root of the problem is the user is being redirected to the second login screen in the middle of the update at a state where the system is not reliable. No code fix can account for that consistently and correctly, you just cannot run Joomla with logic from two different versions in place (especially across minor branches where new APIs are added).

The problem is not that the SQL updates are not being applied. The problem is that the SQL updates are applied after you hit the second login screen if you are directed there, and this redirect happens at a point after the filesystem has been updated with the new release's files.

This is a major inconvenience, this is an issue that needs addressing in some form. But unless someone has a solution that does not involve compromising the security checks of the update component, the ONLY possible solution is to restrict the update system so that 3.6.0 and earlier are only proposed the 3.6.5 update and 3.6.1 and later can be proposed the current version; additionally. Even without that change, the upload and update mechanism cannot be used for updates from 3.6 to 3.7 or later because of this issue.

@ alikon

for the missed client_id column issue, the forum proposed fix should work running https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_admin/sql/updates/mysql/3.7.0-2017-01-17.sql should fix
=======================================================================
Thank you, but hat doesn't seem to work. I first tried it unchanged, that got me an error msg (no such table). Then I changed #__menu to te_de_menu according to my table prefix. Result:
=======================================================================
SQL query:
UPDATE te_de_menu_types SET menutype = 'main_is_reserved_133C585' WHERE client_id =0 AND menutype = 'main';
MySQL said: Documentation
#1054 - Unknown column 'client_id' in 'where clause'
=======================================================================
cf te_de_.sql.tar.gz (some private values replaced by xxxx)

@mbabker

The problem is not that the SQL updates are not being applied. The problem is that the SQL updates are applied after you hit the second login screen if you are directed there, and this redirect happens at a point after the filesystem has been updated with the new release's files.

Is there any way for the update to recognise that the Joomla version is 3.5.1 (or less) then serving an update to 3.7.5 before serving the update to 3.8.x ?

We can restrict what updates are shown through the update component (and we already do so for various scenarios). In the code we cannot reliably detect the version that was upgraded from. Even in the current release, we do not store the from version anywhere that can be used in this way.

Also, as I've pointed out before, it is not 3.5.1 direct to 3.8.0 that is the issue. 3.6.0 and earlier to 3.6.1 or later have extra steps because of the added CSRF checks (IIRC 3.6.3 introduced the second login screen instead of having a "hard" error message), any of these releases will most likely run into the same SQL related problem if they try to direct upgrade to 3.7.0 or later.

The absolute safest upgrade path is anything from 3.6.0 or earlier are only offered to upgrade to 3.6.5 at the latest. 3.6.1 or later, as long as using the "normal" online update method (as in not using the upload feature) should be able to safely direct upgrade to the current version.

Regardless of what changes we make to the update server and what updates are offered to users, as I have repeatedly said, there is not a code solution that can make Joomla reliably work if using the upload and update mechanism and going from 3.6.5 or earlier to 3.7.0 or later because of the schema changes to the menu table and the fact that the second login screen renders the admin menu module.

This is a big if. But, if it is possible to change that second login screen to a screen that is very close to the normal login screen plus the extra notification message without loading the rest of the administrator modules, we might be able to circumvent this issue. Even if we can circumvent this specific issue, we are still left with the "problem" of trying to perform actions on a site while it is in an unstable mixed state which means we probably will run into it again in the future.

(thinking out loud)
The script.php has the from_version so we could save this in a file somewhere, with this info we can hack the menu module to use a different sql when coming from a version 3.6.0 or earlier. It is an option but it sounds wrong to me.

@mbabker

The absolute safest upgrade path is anything from 3.6.0 or earlier are only offered to upgrade to 3.6.5
That's what I was trying to say but got the version numbers mixed up.

script.php isn't used/included until after the checks which redirect the update to the second login screen. The preflight action in the script would have to be moved before the CSRF check or whatever handles redirecting the upload and update mechanism.

closing this Issue, please comment at #18020

Set to "closed" on behalf of @franz-wohlkoenig by The JTracker Application at issues.joomla.org/joomla-cms/18012

Please reopen this as #18020 is a different issue.

Set to "open" on behalf of @franz-wohlkoenig by The JTracker Application at issues.joomla.org/joomla-cms/18012

reopened as stated in Comment

PLEASE... HELP ME #18114

I can confirm the same error (Unknown column 'a.client_id' in 'where clause') also in Joomla 3.8.1 RC.
No backend anymore if update from 3.5.1 to 3.8.1 RC or from 3.6.0 to 3.8.1 RC. (from all Joomla versions before 3.6.1 no backend anymore, also we can't update from Joomla 2.5).
Please add this info in the 3.8.1 FAQ
https://docs.joomla.org/Category:Version_3.8.1_FAQ
it is not yet present in the 3.8.0 FAQ :(
https://docs.joomla.org/Category:Version_3.8.0_FAQ

...sorry for my bad english.

after the 3.8.1 stable release, do you restrict the update system so that 3.6.0 and earlier are only proposed the 3.6.5 update?

@AlexRed please read Comment of @mbabker.

why redirect the open discussion to an close discussion?
If mbabker ask for "someone gives concrete and reproducible paths" I write above:
I can confirm the same error (Unknown column 'a.client_id' in 'where clause') also in Joomla 3.8.1 RC.
No backend anymore if update from 3.5.1 to 3.8.1 RC or from 3.6.0 to 3.8.1 RC. (from all Joomla versions before 3.6.1 no backend anymore, also we can't update from Joomla 2.5).
Please add this info in the 3.8.1 FAQ
https://docs.joomla.org/Category:Version_3.8.1_FAQ
it is not yet present in the 3.8.0 FAQ :(
https://docs.joomla.org/Category:Version_3.8.0_FAQ

...sorry for my bad english.

why redirect the open discussion to an close discussion?

there were similar Issues opened after Release so some get closed to have not too many Threads about same Issue. As always People can discuss also on closed one.

I am fully aware what versions are giving problematic updates. And I am pretty sure the exact reason is because of the second login screen in the update process if either the CSRF token validation fails (will happen from 3.6.0 and earlier because the update component in those versions didn't generate a token to validate) or, and this one is still yet-to-be-confirmed, if using the upload and update method (whenever that was added). The reason this is problematic is because the second login screen is reached at a point where the code base is in a 100% inconsistent state; the files have all been overwritten, but old files have not yet been deleted (which can affect what version of a PHP class is actually used) and the database updates have not yet been applied. The entire reason I opened #18127 is because we are having too many problems with updates failing because we are using the core application to update itself and opening the door to too many problems and this is ultimately the root problem that MUST be addressed.

It is known that updating core from 3.6.0 or earlier direct to 3.8, because of the second login screen, is problematic. Has anyone thought to test what happens if you are running a 3.x version at 3.6.0 or earlier and applying the separate update for the update component before trying to update core?

I have no issue locking down the update path. I do have an issue with blindly locking it. We do not know the exact reason why these update paths are failing. Until we know why, which means we have an issue report somewhere with concrete reasoning (update from 3.5.1 to 3.8 fails with error X caused by code change Y at a minimum), I am not locking it down. Once we have that concrete information, we know exactly what we need to look at in core to either fix it or to clearly document/communicate why the extra updates are required going forward. This also helps us to be aware for the future if there is some change that causes new conflicts going forward.

Someone give me all of the details needed so that anyone can debug and document the issue and we can make a proper decision going forward. Without all of the facts (we are still missing the cause of the issue, there are already 14275908 reports of the symptoms), it is irresponsible to take action.

@Marek-Moehling Please run https://forum.joomla.org/viewtopic.php?f=621&t=582860 and post only php configuration part.

@wojsmol , fpa-en.php generates a html result page, but when I click on the button "click here to generate post" nothing happens, so I uploaded a screenshot.

update from 3.5.1 to 3.8 fails with error 1054 Unknown column 'a.client_id' in 'where clause', PHP 7.0.21.
No update available for the "Joomla! Update Component".
In Joomla 3.5.1 I can find the 3.0.0 version for the "Joomla! Update Component".
The "Joomla! Update Component Update Site":
http://update.joomla.org/core/extensions/com_joomlaupdate.xml

<version>3.6.1</version>
<targetplatform name="joomla" version="3.6"/>

If I install manualy the com_joomlaupdate-3.6.1.zip in Joomla 3.5.1 after the update to Joomla 3.8 is ok.

Can we install the com_joomlaupdate-3.6.1.zip also in Joomla 3.4 ?

@Bakual If applying #18075 to 38.0 Stable is passible?
@AlexRed @Marek-Moehling Please open separate topics in https://forum.joomla.org/viewforum.php?f=710

why I need to open a topic in the forum ?

Can we install the com_joomlaupdate-3.6.1.zip also in Joomla 3.4 ?

Scanning the changelogs I'm not seeing anything that would keep it from working on older versions but the only way to know for sure is to test it.

ok, now I try it in Joomla 3.4 and 3.3.

But why we can't see and update available for the com_joomlaupdate in Joomla 3.5 ?

When I updated a site the other week I was first presented with an update to joomla 3.5.1
I then was presented with an update for com_joomlaupdate which I did
and then I had an update to joomla 3.8

In Joomla 3.4 no update for com_joomlaupdate available, I install the com_joomlaupdate-3.6.1.zip and ok the update to Joomla 3.8. (PHP 5.4.25)
In Joomla 3.3 no update for com_joomlaupdate available, I install the com_joomlaupdate-3.6.1.zip and when I open the component menu for the joomla update: Fatal error: Undefined class constant 'STABILITY_STABLE' in /administrator/components/com_joomlaupdate/models/default.php on line 135
(PHP 5.4.25)

I would expect that

@mbabker

I have no issue locking down the update path. I do have an issue with blindly locking it. We do not know the exact reason why these update paths are failing.

I can test updates from various versions on localhost if you wish ?

@AlexRed I asked to open a topic on the forum because this repository is about errors in Joomla! not trouble with the update.

we are talk about the Joomla update.
Sure we can't talk about the Joomla update problem?

In Joomla 3.3 no update for com_joomlaupdate available, I install the com_joomlaupdate-3.6.1.zip and when I open the component menu for the joomla update: Fatal error: Undefined class constant 'STABILITY_STABLE' in /administrator/components/com_joomlaupdate/models/default.php on line 135

So that tells me we can allow the standalone component package we have now to be installed from 3.4.0 up to 3.6.0. Which is helpful to users on those versions, including the 2.5 upgrade path which still goes to 3.5.1.

So right now that tells me that 3.3.6 and earlier need to be offered up to 3.6.0 and that 3.4.0 and later can be offered 3.8 (and newer), BUT users on 3.4.0 through 3.6.0 MUST install the standalone update component package before doing so. Again, we still don't have the root cause of the client_id issue (as the schema change was made in 3.7 but apparently doesn't affect the backend until 3.8), but at least it's progress in being able to make changes to the updates offered and provide documentation why it needs to be that way.

@mbabker client_id issue occurs then we have not up-to-date database structure and filed update do Joomla! 3.8.0. In @Marek-Moehling case update failed because php is runnig as Apache module. This issue is resolved by #18075.

The client_id issue is not always related to a failed update. An update which hits the second authentication screen coming from 3.6.0 or earlier will hit it too, and that is not because of a failure but because that screen renders the menu module while the code is in the middle of the update state therefore not reliable.

@mbabker This is true also, but see @Marek-Moehling FPA in #18012 (comment)

Right, all I'm saying is separate issues but the same error messages get presented to users. It's ultimately the same root cause for that specific one.

@AlexRed

Fatal error: Undefined class constant 'STABILITY_STABLE' in /administrator/components/com_joomlaupdate/models/default.php on line 135

Comment out lines 128 - 136 (inclusive). and Components >>> Joomla update can be displayed. And the Upload & Update tab is available to upload & install update versions of your choice.

3.3.6 using Components >>> Joomla update >>> Upload & Update tab
Upload and install Joomla_3.7.5-Stable-Update_Package.zip
Freezes at administrator/index.php?option=com_joomlaupdate&view=update&layout=finaliseconfirm and a lot of numbers
Edit the numbers off and administrator/index.php?option=com_joomlaupdate&view=update&layout=finaliseconfirm Shows:
The accept notice for data

Joomla Update is finishing and cleaning up
To complete the update Process please confirm your identity by re-entering the login information for your site "J336b" below.

And admin login
Clicking to login No Avail

Edit url to administrator/
Success ... Logs in and control panel shows notification for one component and for Update to 3.8.0

Database errors

Table 'exquw_menu' does not have index 'idx_client_id_parent_id_alias_language'. (From file 2.5.0-2011-12-24.sql.)
Table 'exquw_redirect_links' does not have index 'idx_old_url'. (From file 3.5.0-2016-03-01.sql.)
The Joomla! Core database tables have not been converted yet to UTF-8 Multibyte (utf8mb4).

@Webdongle PersonallyI would not do such a big jump in 1 step.

@wojsmol
Nor me for a live site but it was an attempt to see if it could be done. It can be done (with manual intervention of the generated urls) ... this might help pinpoint what the problems.

You have to manually edit url because of relogin form mention by @mbabker in #18012 (comment)

@wojsmol
because administrator/index.php?option=com_joomlaupdate&view=update&layout=finaliseconfirm788654345768887 (or whatever number) is appended ... just causes the browser to spin.

Browwser spin is for 2 reasons:

1. Big version jump - a lot to do
2. layout=finaliseconfirm is the re-login form.

For the update path I have opend the issue: #18192 that explains how to fix it. Please test there and report back your results.

I don't understand nor can confirm this happend by updating from 3.7.5 as there the client_id is inplace. (should be since 3.7) Please double check that @Marek-Moehling

@zero-24

• For the update path I have opend the issue: #18192
  I don't have a working admin interface anymore, so I don't know how to apply that

• Please double check that
  I updated Joomla 3.7.5 to 3.8, I wouldn't know what to check for now...

Please double check that
I updated Joomla 3.7.5 to 3.8, I wouldn't know what to check for now...

that means that despite being at J3.7.5 your database was not up-to-date (and then you update to J3.8.0),
since if you had J3.7.5 the missing column should have been added already

for which version did you update to J3.7.5 ? do you remember ?

@Marek-Moehling Please open topic on https://forum.joomla.org/viewforum.php?f=710 and I will assist you with broken site.

@ggppdk

for which version did you update to J3.7.5 ? do you remember ?

No, sorry.

Set to "closed" on behalf of @franz-wohlkoenig by The JTracker Application at issues.joomla.org/joomla-cms/18012

closed as stated by Comment

@franz-wohlkoenig Please re-open, because of discussion.

wojsmol

Please open topic on https://forum.joomla.org/viewforum.php?f=710 and I will assist you with broken site.

a) Thank you, but I applied this fix successfully just now: #18192 (comment)

b) I had an account at forum.joomla.org but presently I can't login or reactivate my account. (??? )

@Marek-Moehling Try resting the password.

Set to "open" on behalf of @franz-wohlkoenig by The JTracker Application at issues.joomla.org/joomla-cms/18012

reopen as stated above.

@ wojsmol

Try resting the password.

When using "I forgot my password" I get this:
"The email/username information submitted could not be found."
I deleted all cookies and I stored all user names and pw in a text file filling form fields with copy and paste. I tried creating new accounts but never receive activation mails.

@wojsmol isn't this Discussion for Forum?

@franz-wohlkoenig I mean devs discussion above. Additionally see #18012 (comment)

@Marek-Moehling Post your issue on the Forums. This is not a support platform

@Marek-Moehling If you cannot login to the forums use this form https://community.joomla.org/helpdesk/forum-un-ban-request.html

to solve this problem it took 1 minute

it is a database issue and need to be fixed

but without backend is not possible to access extention database, so y u need to do with full link

@theexx

to solve this problem it took 1 minute

Your fix was ?

you need as i wrote to perform a joomla database fix, normally it is an option inside extention menu

due is not possible to access it directly because no backend access you need to launch fix writing full link on your bronwer

got it ?

That does not fix all errors of a failed update. Please post in the forum for support as requested by @gwsdesk . Someone will help you fix the failed update correctly.

@theexx as mentioned by Kevin (webdongle) the DB-fix hides the real issues. You still have a lot of issues though not showing. Your next 'update' will be botched. You have not resolved the issue and as mentioned post on the forums since this on Github is not support channel.

Got it?

Leo

in my case it fixed the backend problem and "1054 Unknown column 'a.client_id' in 'where clause'" error

the issue was related to converting from utf8 to utf8mb4

@gwsdesk maybe i not got what you wrote due my bad english

@theexx

the issue was related to converting from utf8 to utf8mb4

No it wasn't ... that was just one symptom that the update didn't complete correctly. You will have more problems in the future. Got it ???

maybe my case is different.

i compare file by file and table by table affected site with a twin on we have (update was ok with it) and all is identical

For the life of me I do not understand why this hasn't been fixed yet....

@franz-wohlkoenig Close and post to Joomla forum as this is an upgrade issue?

Closed as stated above. Please ask help on the forums - Migrating and Upgrading to Joomla! 3.x for similar Issues.. This repository concerns in first Place Joomla coding.

Set to "closed" on behalf of @franz-wohlkoenig by The JTracker Application at issues.joomla.org/joomla-cms/18012

Its a major bug that is crippling multiple users, not a localized "upgrade issue". Why is it nobody takes this seriously....

This should NOT be closed until the issue is fixed.

@JoshWobbles We do take bugs seriously. This is not a bug but an isolated upgrade error. Post on the forums and we will assist you. It is correct that this is closed and should not be re-opened

Doesn't seem isolated to me, seems like it was easily reproducible making it a BUG!

@JoshWobbles please listen to this community and hear what this community is telling to you. You can continue posting your views but those views do not reflect what the majority of the developers think
and believe so your messages will not lead to any further actions on the Issue Tracker or on Github. So again post on the forums and we will help you at that platform

I looked on the forums, from what I see nobody with this issue has been helped because the BUG has not yet been fixed.

Did I see your post already on the forums? Please make your post wwith all details as requested including FPA-output and we address your issue and it is NOT a bug

https://forum.joomla.org/viewtopic.php?f=708&t=955172

But AGAIN, this is a BUG, so it belongs HERE until it is fixed.... The issue is reproducible and experienced by multiple users.

@franz-wohlkoenig Please lock this thread since this person is not accepting anything being explained related to this issue.... I will attend to this person's post on the forums and he simply should be stopped posting here

Why cant you just admit and address the issue? Why do you ignore everybody who tries to bring it up?

@brianteeman can you please lock this Thread as @gwsdesk comment above?

Please use the forum AND provide the requested information

This is now locked.