? ? Pending

User tests: Successful: Unsuccessful:

avatar richard67
richard67
10 Dec 2022

Pull Request for #39333 (comment) .

Summary of Changes

With Pull Request (PR) #36506 , the possibility to add an installer hint /** CAN FAIL **/ to SQL statements in update SQL scripts has been added, so that the installer ignores the failure and doesn't abort the SQL updates but continues with the next statement.

This makes it possible to handle e.g. SQL errors when trying to add columns which already exist, which can happen if an update SQL scripts is executed multiple times or the update history of a component is complicated. The reason why this is needed in that use case is because "IF EXISTS" or "IF NOT EXISTS" clauses on adding or removing columns is only supported by MariaDB but not by MySQL.

However, depending on the database type (MySQL or MariaDB) and the version and which database client (MySQLi or PostgreSQL) are used, and also on the kind of SQL statement, this doesn't work in all cases since the database client throws different kinds of exceptions depending on all that. In case of the MySQLi driver, the client doesn't return false but raises an exception when executing the SQL statement fails.

See the discussion beginning with this comment: #39333 (comment) .

This PR here fixes that by adding \RuntimeException to the try catch where we check the installer hint.

Testing Instructions

This test needs to be executed with a MySQL or MariaDB database with the MySQLi database driver and with PHP 8.1.

With PDO or with PHP 7.4 and the MySQLi driver, everything works without this PR.

  1. Download and install the following extension zip package: https://test5.richard-fath.de/com_testupdatesql_1.0.0.zip
    This will install version 1.0.0 of a dummy component named "Test Update SQL" (com_testupdatesql), which has a database table named #__testupdatesql_table_1 (with #__ equal to your database prefix).
    The table has an id and a title column and one record with id=1 and title='Record 1'.

  2. In phpMyAdmin, execute the following SQL statement to add a new columd description to that table:

ALTER TABLE `#__testupdatesql_table_1` ADD COLUMN `description` varchar(255) NOT NULL DEFAULT '';

(replace #__ by your database prefix).

This will simulate the situation after a failed update attempt to the next version 1.0.1 (see next step below) or the situation having a remainder from a previously installed version 1.0.1 where the database table has not been properly removed on uninstallation, or the situation when the extension developer had not provided SQL scripts in the right way so.

  1. Update to version 1.0.1 of that component by downloading and installing the following extension zip package: https://test5.richard-fath.de/com_testupdatesql_1.0.1.zip
    This update will add a new column description to the database table, then update the existing first record by a suitable value for that column, and then insert a new, 2nd record into the table.
    Result:
    When this PR is not applied, the update fails with SQL error "Duplicate column name 'description'", see section "Actual result BEFORE applying this Pull Request"
    below.
    When this PR is applied, the update succeeds, see section "Expected result AFTER applying this Pull Request"
    below.

  2. Verify the table structure and content in phpMyAdmin.

  3. Examine the installation and update SQL scripts of the 2 versions mentioned above and make sure you understand them and that they are doing everything right like it should be in update SQL scripts for Joomla 4.

Hint: I could have created the 2nd zip in a way so it causes the error immediately without the need for executing step 2, but I wanted the scenario to use correct packages so testers understand what happens.

Actual result BEFORE applying this Pull Request

The update fails with SQL error "Duplicate column name 'description'".

com_testupdatesql_update-error

The SQL statement to add the column and all statements after that haven't been executed, so the table still has one record only and no description column.

com_testupdatesql_update-error_db

Expected result AFTER applying this Pull Request

The update succeeds.

com_testupdatesql_update-ok

All database changes have been applied, so the table has two records with the right description values.

com_testupdatesql_update-ok_db

Link to documentations

Please select:

  • Documentation link for docs.joomla.org:

  • No documentation changes for docs.joomla.org needed

  • Pull Request link for manual.joomla.org:

  • No documentation changes for manual.joomla.org needed

Additional remarks on documentation

It doesn't need to update any existing documentation because we don't have any which tells extension developers in detail how their update SQL scripts should be so that they are accepted by the database checker, too, and how to use the "/** CAN FAIL **/" installed hint or "INSERT IGNORE".

We should create such a documentation, but this should not stop this PR here.

avatar joomla-cms-bot joomla-cms-bot - change - 10 Dec 2022
Category Libraries
avatar richard67 richard67 - open - 10 Dec 2022
avatar richard67 richard67 - change - 10 Dec 2022
Status New Pending
avatar richard67 richard67 - change - 10 Dec 2022
The description was changed
avatar richard67 richard67 - edited - 10 Dec 2022
avatar richard67 richard67 - change - 10 Dec 2022
Title
Fix installer hint "/** CAN FAIL **/" for update SQL scripts not working in all cases
Fix installer hint "/** CAN FAIL **/" for update SQL scripts not working with MySQLi driver
avatar richard67 richard67 - edited - 10 Dec 2022
avatar richard67 richard67 - change - 10 Dec 2022
The description was changed
avatar richard67 richard67 - edited - 10 Dec 2022
avatar ReLater ReLater - test_item - 10 Dec 2022 - Tested successfully
avatar ReLater
ReLater - comment - 10 Dec 2022

I have tested this item successfully on 814bc90

PHP 8.1.7
MySQLi 10.5.18-MariaDB-1:10.5.18+maria~ubu2004-log

Tested with a custom component, but followed the testing instructions of this pr.
Thank you!


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar dawe78 dawe78 - test_item - 11 Dec 2022 - Tested successfully
avatar dawe78
dawe78 - comment - 11 Dec 2022

I have tested this item successfully on 814bc90

Tested sucdessfully with provided test component and my own test component. Thanks @richard67 for this fix and pull request!!!


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar richard67 richard67 - change - 11 Dec 2022
Status Pending Ready to Commit
Labels Added: ?
avatar richard67
richard67 - comment - 11 Dec 2022

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar richard67
richard67 - comment - 11 Dec 2022

Thanks guys for testing. As the upcoming 4.2.6 is already being prepared, this PR will very likely go into 4.2.7. So don’t be surprised about that.

avatar richard67 richard67 - change - 12 Dec 2022
The description was changed
avatar richard67 richard67 - edited - 12 Dec 2022
avatar richard67 richard67 - change - 12 Dec 2022
The description was changed
avatar richard67 richard67 - edited - 12 Dec 2022
avatar richard67 richard67 - change - 12 Dec 2022
Labels Added: ? ?
avatar richard67 richard67 - change - 12 Dec 2022
Status Ready to Commit Pending
avatar richard67
richard67 - comment - 12 Dec 2022

Back to pending after changes.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar richard67
richard67 - comment - 12 Dec 2022

@ReLater @dawe78 Sorry, I had to make a change. Could you test again with the latest changes? Thanks in advance.

avatar ReLater ReLater - test_item - 12 Dec 2022 - Not tested
avatar ReLater
ReLater - comment - 12 Dec 2022

I have not tested this item.

PHP 8.1.7
MySQLi 10.5.18-MariaDB-1:10.5.18+maria~ubu2004-log

Tested with a custom component, but followed the testing instructions of this pr. Plus several other sql lines.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar richard67
richard67 - comment - 12 Dec 2022

@ReLater Can it be that you have forgotten to select the test result?

avatar ReLater ReLater - test_item - 12 Dec 2022 - Tested successfully
avatar ReLater
ReLater - comment - 12 Dec 2022

I have tested this item successfully on 56b8071

PHP 8.1.7
MySQLi 10.5.18-MariaDB-1:10.5.18+maria~ubu2004-log

Tested with a custom component, but followed the testing instructions of this pr. Plus several other sql lines.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar dawe78 dawe78 - test_item - 13 Dec 2022 - Tested successfully
avatar dawe78
dawe78 - comment - 13 Dec 2022

I have tested this item successfully on 56b8071

Tested successfully with provided component and own test component.

Joomla 4.2.6-rc1
PHP 8.1.13
MariaDB 10.3.37

Works, BUT... does really any driver return RuntimeException if execute() fails? Eg. in PDODriver.php an ExecutionFailureException will be thrown; is it a kind of RuntimeException and will be catched within installer.php if only RuntimeException is defined? Just my 2 cents, maybe I'm wrong... I'm not very familiar with RuntimeExceptions...


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar richard67 richard67 - change - 13 Dec 2022
Status Pending Ready to Commit
avatar richard67
richard67 - comment - 13 Dec 2022

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar richard67
richard67 - comment - 13 Dec 2022

Remove updates requested label


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/39391.

avatar roland-d
roland-d - comment - 17 Dec 2022

@richard67 User @dawe78 asked this:

Works, BUT... does really any driver return RuntimeException if execute() fails? Eg. in PDODriver.php an ExecutionFailureException will be thrown; is it a kind of RuntimeException and will be catched within installer.php if only RuntimeException is defined? Just my 2 cents, maybe I'm wrong... I'm not very familiar with RuntimeExceptions...

Now I wonder if we should use Exception instead of RuntimeException.

avatar richard67
richard67 - comment - 17 Dec 2022

@richard67 User @dawe78 asked this:

Works, BUT... does really any driver return RuntimeException if execute() fails? Eg. in PDODriver.php an ExecutionFailureException will be thrown; is it a kind of RuntimeException and will be catched within installer.php if only RuntimeException is defined? Just my 2 cents, maybe I'm wrong... I'm not very familiar with RuntimeExceptions...

Now I wonder if we should use Exception instead of RuntimeException.

@roland-d All the possible exceptions extend from runtime exceptions. Of course exception would also work. Feel free to provide a better fix, then I close my PR in favor of that.

avatar roland-d
roland-d - comment - 17 Dec 2022

@richard67

@roland-d All the possible exceptions extend from runtime exceptions.

In that case, I am fine and we do not need any further changes.

avatar richard67
richard67 - comment - 17 Dec 2022

@roland-d You can consider it as a workaround because on the long run we indeed have to revisit exception handling in the framework in the database drivers. There have been made changes in the PHP clients (mysqli and PDO) with 8.0 and 8.1 so it can be that they throw exceptions in these versions and returned false in other versions for certain methods. But that’s a bigger piece of work so for the meantime we can fix it here.

avatar richard67
richard67 - comment - 17 Dec 2022

P.S.: In principle the right fix would be like suggested in that comment here #39333 (comment) by @dawe78 , but it would need to investigate if that should also be done in other methods and for other drivers for making a clean solution in the database drivers. That’s why I did not use that suggestion but fixed it here now.

avatar roland-d roland-d - change - 17 Dec 2022
Labels Removed: ?
avatar roland-d roland-d - change - 7 Jan 2023
Status Ready to Commit Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2023-01-07 19:20:02
Closed_By roland-d
avatar roland-d roland-d - close - 7 Jan 2023
avatar roland-d roland-d - merge - 7 Jan 2023
avatar roland-d
roland-d - comment - 7 Jan 2023

Thank you

Add a Comment

Login with GitHub to post a comment