? ? Failure

User tests: Successful: Unsuccessful:

avatar richard67
richard67
16 Jul 2017

Pull Request for Issue #15719 comment .

Summary of Changes

This pull request (PR) adds a warning message about incomplete core update and a finalise core update button to the databae manager view and the extensions discovery view.

You will find some screenshots in a comment later below.

I made it an RFC because I would like to have feedback from people first.

If I get more positive than negative feedback, I will remove the "[RFC]" in the title.

As several posts in support forums and some issues in the issue tracker have shown, still some people use the old method of copying or unzipping the files from the update container and then running the database manager fix to update the database schema method to update the core to a new version.

The problem with that method is that the database manager cares only about DDL (data definition language, e.g. CREATE TABLE, ALTER TABLE), i.e. the database schema.

It does not care about DML (data manipulation language, i.e. INSERT, UPDATE, DELETE), which is also used in the schema update SQL scripts.

So if someone uses that old method mentioned above or for some reason an update with the Joomla Update component aborted somewhere in the middle e.g. because of a timeout on large databases, the database manager would then fix only the schema changes.

If that happened in past or if an installation was sequentially updated using the old method in past, the result is a database which often causes again problems on updating later with the Joomla Update component.

See discussion in issue #15719 .

A solution in many such scenarios up to now was to run @mbabker ' s Post-Manual Update Script, which runs the "finalise" and "cleanup" steps of the Joomla Update component.

This PR here was inspired by that script and does following:

  • If the schema version stored in the database is lower than the version obtained from the available schema update files on the file system, and the core software version stored in the database is lower than the JVERSION constant obtained from the new core file, a warning message about a possibly incomplete Joomla update is shown in the Extensions -> Manage -> Discover and the Extensions -> Manage -> Database views, and a button "Finalise Joomla Update" is shown in the toolbar.

  • Otherwise, if the condition is not true, warning and button are not shown.

  • If the button is clicked, the "finalise" and "cleanup" steps of the Joomla Update component are executed, then the page is reloaded.

So with this PR, it will become more unlikely in future that people have not properly updated databases, and even the old update method could be supported again officially or inofficially ;-)

Testing Instructions

Preparation

Either you have an installation which has been updated from a version older than 3.7.0 to latest version 3.7.3 or a 3.4.0 beta or RC or the latest nightly 3.7.x build, and you have a backup (e.g. SQL export) of the databse from before such an update.

Or you install Joomla version older than 3.7.0 and make a backup (e.g. SQL export) of your database and then update this to latest version 3.7.3 or a 3.4.0 beta or RC or the latest nightly 3.7.x build.

Now apply the patch of this PR here e.g. with the patchtester component or just copying the changed files to the new installation.

Then restore the database to the old version before the update, i.e. delete all tables and then import the database backup.

The result is that the Joomla filesystem has latest version (+ this PR), but the database is the one of the older pre-3.7.0 Joomla version.

Procedure

Step 1: Log in on backend and go to Extensions -> Manage -> Discover.

Result: A warning message about a possibly incomplete Joomla! update is shown, and the toolbar contains a button "Finalise Joomla Update".

Step 2: Click the "Finalise Joomla Update" button.

Result: The page is reloaded, the warning message and button are not shown anymore.

Step 3: Check the end of administrator/logs/joomla_update.php

Result: The log shows the messages from the "finalise" and "cleanup" steps of the Joomla Update component in the same way like with an update using the Joomla Update component.

Step 4: Restore the old database backup to restore the starting conditions.

Step 5: Log in on backend and repeate the above test steps 1 to 3 on Extensions -> Manage -> Database.

Expected result

When a possibly incomplete core update has been detected, a warning message and a button "Finalise Joomla Update" are shown in the Extensions -> Manage -> Discover and the Extensions -> Manage -> Database views.

Clicking the button "Finalise Joomla Update" runs the "finalise" and "cleanup" steps of the Joomla Update component.

Actual result

None because it is a new feature.

Documentation Changes Required

A description of the new warning message and the new button should be added to the updating instructions https://docs.joomla.org/Special:MyLanguage/J3.x:Updating_from_an_existing_version

The instructions might have to be reviewed anyway for particular languages, because e.g. the German version refers in section "Prüfe die Seite" (check the site) to some methods A and B fom old times when the "copy or unzip and then use database fixer" was officially supported and described in the update instructions, but these methods are not described or linked on that page anymore.

avatar richard67 richard67 - open - 16 Jul 2017
avatar richard67 richard67 - change - 16 Jul 2017
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 16 Jul 2017
Category Administration com_installer Language & Strings
avatar richard67 richard67 - change - 16 Jul 2017
Labels Added: ? ?
avatar mahagr
mahagr - comment - 16 Jul 2017

? For long time I have been wondering why simple feature like this had not been implemented. There are times where updating fails on some unknown and maybe even temporary error and that for the most users mean incomplete update even after fixing the database. For example upgrading older J3 to J3.5+ was problematic if you used default opcache settings (it failed on an error because of old version of JVersion was being used).

avatar richard67
richard67 - comment - 16 Jul 2017

See following screenshots about the expected results:

  1. Extensions -> Manage -> Discover
    screen shot 2017-07-16 at 15 33 46

  2. Extensions -> Manage -> Database
    screen shot 2017-07-16 at 15 33 51


    This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/17146.

avatar richard67
richard67 - comment - 16 Jul 2017

@BastianWie Would this PR here be Ok for you, too?

@mbabker Please let me know your opinion.

@brianteeman If you find time, please check the texts. And of course your general opinion on this is welcome, too.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/17146.

avatar richard67 richard67 - change - 16 Jul 2017
The description was changed
avatar richard67 richard67 - edited - 16 Jul 2017
avatar richard67 richard67 - change - 16 Jul 2017
The description was changed
avatar richard67 richard67 - edited - 16 Jul 2017
avatar richard67
richard67 - comment - 16 Jul 2017

@brianteeman I am almost sure you will not like the <br /><br /> in the texts, and that you would like to have the texts shorter. I thought it is more readable like it is now, and I wanted to describe things as clearly as possible within that little space, but if you want I will change that. And maybe the button names should be in strong letters?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/17146.

avatar brianteeman
brianteeman - comment - 16 Jul 2017

Sorry but I dont see the point in writing code to support people who chose not to update joomla with the provided and supported tools.

avatar richard67
richard67 - comment - 16 Jul 2017

Well, it still could help people who used the right method where the update has timed out or been aborted for some other reason. It is at least a better way out than the database fixer for the reasons mentioned in the description.

avatar mbabker
mbabker - comment - 16 Jul 2017

And this is different from the Upload & Update tab in the install component how?

avatar richard67
richard67 - comment - 16 Jul 2017

It is less likely to timeout because of slow uploads.

avatar richard67
richard67 - comment - 16 Jul 2017

@mbabker It does the same as your script Post-Manual Update Script does.

avatar BastianWie
BastianWie - comment - 16 Jul 2017

@richard67 yes that looks usefull for the users/admins.

avatar richard67
richard67 - comment - 16 Jul 2017

Seems I have 2 pros and 2 cons up to now. Of course it already can be tested.

avatar mbabker
mbabker - comment - 16 Jul 2017

Honestly, I really hate the fact we keep turning what is supposed to be a database status page into the one stop shop to fix all of your core issues. And honestly, if you're having issues with timeouts because of the package download or uploading to a crap server, that's a scenario where you should just use my script. The project does not support upgrades outside of our update component. We shouldn't be stepping backwards to support 1.5 style manual upgrades again.

avatar richard67
richard67 - comment - 16 Jul 2017

@mbabker In general I share your opinion, but again and again reading issues and forum posts related to broken updates made me feel the need to propose this here as a workaround until the database manager is able to fix broken updates in a better way or there is another real solution. For me the existing methods and 1 time in past your script after I had an incomplete update caused by PHP 7 opcache problems were sufficient. But I thought for people who don't know about your script it could be helpful to have something more visible.

Anyway I will accept any opinion and especially of PLT or other responsibles and have no problems to close this PR if there is no acceptance.

avatar brianteeman
brianteeman - comment - 16 Jul 2017

This just encourages people not to use the installer

avatar mbabker
mbabker - comment - 16 Jul 2017

The database fix stuff isn't broken. We as a project have misused that resource since the day the underlying API was introduced and we continue to fail to address procedural (and inherently architectural).

We as a project also failed in turning the database status page into a one stop shop for fixing all the things. The database fix button no longer just attempts to get your database schema to the required state; it includes two function calls to code in other components that is in no way related to schema updates (the remove deleted files function and purging the core update check record in the updates table).

We really need to stop monkey patching things and fix our core issues the right way. Yes, that is going to mean retraining users/contributors and breaking muscle memory. Yes that is going to mean for once a code related issue is going to have to get priority over whether buttons in the toolbar are too wide or the drop shadow the status module creates.

For me, Upload & Update is the biggest compromise we can make as far as supporting manual upgrades. We should not introduce something into the UI that even gives the impression to a user they can do a 1.5 style manual upgrade, click a button, and everyone be rainbows and unicorns happy. Our update process is too complicated for that, compounded by the project's misuse of resources resulting in some of our changes breaking sites (every INSERT/DELETE statement we have in our update SQL files for the menu table has resulted in the corruption of the nested tree model in that table as an example).

avatar brianteeman
brianteeman - comment - 16 Jul 2017

I hope in J4 we wont repeat these mistakes of the past

avatar richard67
richard67 - comment - 21 Jul 2017

@mbabker @brianteeman And how about showing the message somewhere with a link to the upload and install tab of the update component in case if an incomplete update was detected? Would that be ok for you, i.e. you only dislike the buttons? Or you dislike any help for the people who are in that kind of trouble?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/17146.

avatar richard67
richard67 - comment - 30 Aug 2017

@brianteeman @mbabker I try it again with my question before I will close this PR here:

How would it be show show a message in the database checker view and the discover extensions view if an incomplete update was detected?

Only show a warning message telling that it could be fixed with the Joomla Update component, installing Joomla files again using the Upload & Install way, nothing else?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/17146.

avatar brianteeman
brianteeman - comment - 9 Sep 2017

Revisiting this RFC I still don't think this is something that should be accepted

avatar richard67 richard67 - change - 9 Sep 2017
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2017-09-09 12:03:29
Closed_By richard67
avatar richard67 richard67 - close - 9 Sep 2017

Add a Comment

Login with GitHub to post a comment