User tests: Successful: Unsuccessful:
Pull Request for Issue #32542 (part).
This pull request (PR) removes display width modifiers for integer data types in DDL statements in all SQL scripts for MySQL and MariaDB databases.
It is only relevant for MySQL and MariaDB databases, i.e. not PostgreSQL.
It's the J4 equivalent of PR #32606 for staging. See that PR for a detailed description of the reasons for this change.
In addition to the change for the display width modifier, this PR fixes also old null dates being used in SQL scripts for the integration tests. I had to touch the script for MySQL anyway, so I decided to fix that, too, also for PostgreSQL. This change will not need any tests beside the integration tests passing in drone.
You need a MySQL or MariaDB database.
Testers please report back which of the following 2 kinds of databases you have used for the test:
If you have both scenarios available, please test both.
This test is optional and can be only executed when having a MySQL database server of version 8.0.17 or later.
If you don't have that kind of MySQL database server, skip this test.
During my tests I wasn't able to configure my MySQL 8.0.20 server in a way that the deprecation warnings for the display widths could be shown in the server log file, and you can believe me I've tried a lot of things.
Furthermore, it appeared that when importing an SQL script using display widths for integers in DDL into phpMyAdmin, there are also no warnings shown.
So if you want to see the issue you have to do as follows:
In phpMyadmin, select a database.
Copy the first CREATE TABLE
statements for the #__assets
table (or any other which contains display widths) from file installation/sql/mysql/base.sql of the 4.0-dev branch (i.e. not from this PR) into the SQL command window.
Hint: You can use also several statements at once, but you should not use the complete content of the SQL script since in this case phpMyAdmin (or the server?) seem to suppress the warnings due to their high count.
Execute the SQL in the SQL command window (no need to replace the table name prefix before).
Result: You get warnings Warning: #1681 Integer display width is deprecated and will be removed in a future release.
.
Remove the table created in the previous step.
Repeat steps 2 and 3 but using a statement from the installation/sql/mysql/base.sql file of this PR.
Result: Don't get the warnings from step 3.
Make a new installation with the patch of this PR applied, using an empty database.
Login to backend.
Go to "System-> Manage -> Database" and verify that no database problems are shown.
Result: No database problems found (except of one problem about database version not matching the CMS version when using the installation package created by drone for this PR).
Result:
The starting point is a clean Joomla 3.10 installation (3.10-dev branch or latest 3.10 nightly build).
Login to backend.
Make sure to see any database errors or warnings by switching on "Debug System" and setting "Error Reporting" to "Maximum" or "Development" in Global Configuration.
Update to current 4.0-dev + the patch of this PR applied by using one of the following 2 methods:
While the update runs, verify that no SQL errors or warnings are shown in backend or in your MySQL or MariaDB server's log file or in the PHP error log.
Go to "System-> Manage -> Database" and verify that no database problems are shown.
Result: No database problems found except of one problem about database version not matching the CMS version. This is normal when using updates package created by drone for PR's.
Result:
SQL scripts use the unnecessary and for MySQL versions 8.0.17 or later deprecated display width modifier for integer data types.
No display width modifier is used for integer data types at any place in SQL statements.
After a new installation on MySQL database servers with versions lower than 8.0.17 or any version of MariaDB servers, database columns with integer data types have the default display widths for the particular data type. On MySQL database servers with versions 8.0.17 or newer, such database columns have no display widths.
When updating from 3.10, display widths haven't been changed for database columns which existed already before the update and haven't been modified by the update for other changes than display widths. For newly created or modified columns, the above statement for new installations applies.
None.
@wilsonge When PR's #32606 and #32607 will be merged up from staging or 3.10-dev into 4.0-dev after this PR has been merged, it will cause merge conflicts in the installation/sql/mysql/base.sql
file. In this case use the file from the 4.0-dev branch, there should not be any changes on this file coming from staging or 3.10-dev as far as I know.
Status | New | ⇒ | Pending |
Category | ⇒ | SQL Administration com_admin Installation |
@brianteeman it’s hard to test because during installation we don’t get these deprecated warnings. So the remaining work here to be done is to find some test description. Maybe you can test my other PR #32605 ? That one is ready.
Ok will do.
Btw I spotted this originally because the SQL editor I use (heidisql) alerted me to the deprecation.
Labels |
Added:
?
|
Category | SQL Administration com_admin Installation | ⇒ | SQL Administration com_admin com_banners com_contact com_finder com_newsfeeds Installation |
Category | SQL Administration com_admin Installation com_banners com_contact com_finder com_newsfeeds | ⇒ | SQL Administration com_admin com_banners com_contact com_finder com_newsfeeds Installation Unit Tests |
Labels |
Added:
?
|
Labels |
Added:
?
Removed: ? |
Labels |
Added:
?
Removed: ? |
Title |
|
@brianteeman Ready for test. See also PR's #32606 and #32607 .
Will try to find some time tomorrow. My time is limited right now
I've simplified testing instructions. Ready for testing.
Labels |
Added:
?
Removed: ? |
Labels |
Added:
?
Removed: ? |
@brianteeman As the other 2 PR's already have been tested, only this one here remains for the display width issue. Maybe you can find some time for testing?
Labels |
Added:
?
Removed: ? |
@brianteeman Do you think you can find the time to test this PR here? I will care for merge conflicts later, and if it is merged before the upmerge of the rest from 3.10-dev, it could even safe @wilsonge time because in the SQL files he can just say "accept yours" for the merge conflicts. I would not ask if it was not a PR for an issue created by you.
sorry forgot all about this one. looking at it now
@brianteeman That's the usual version not matching database problem after an update with a PR package. I've forgotten to mention that maybe. The result looks ok to me.
ah - ok thats good to know.
run out of time for today to run test3
Thanks so far. I've updated the testing instructions by a hint at the 2 places where it's relevant.
as far as i can tell test3 was also successful
I have tested this item
Could we have more detailed testing instructions for test 3:
Display widths of database columns which existed already before the update and which haven't been modified for other changes haven't been changed.
What table and what fields for that table to check, please? Just a sample table should be enough
New tables have been created with display widths for columns with integer data types depending on the type of database:
Again, what table. Just a table to check should be enough.
Display widths of database columns which existed already before the update and which haven't been modified for other changes haven't been changed.
What table and what fields for that table to check, please? Just a sample table should be enough
What table: Those which exist on J3 and J4, like most of the core tables, e.g. #__assets
, #__categories
, #__content
, #__extensions
, ... any of these which has some columns with integer data types.
What fields: All with an integer data type, which can be tinyint
, smallint
, mediumint
, int
and bigint
.
As said in the description, check also the description of the meanwhile merged J3 PR #32606 .
It needs to check that these were not changed to the default display widths on your MySQL 5.7 but keep the display widths they had before the update.
Default display widths see the values in brackets:
New tables have been created with display widths for columns with integer data types depending on the type of database:
Again, what table. Just a table to check should be enough.
Tables which exist in J4 but not in J3.10. Should be easy to find those. They are easy to find by checking the update SQL scripts for "CREATE TABLE" statements, except of the one script where we modify lots of tables for com_finder, this drops tables and later creates them again. You will find then all tables for workflows, mail templates, webauthn and csp, i.e. for the new functions added in J4. On your MySQL 5.7, all these tables should have the default display widths mentioned above for columns with integer data types.
The easiest way to check all the above is to make a database export into an SQL file of only the structure but not the data, using a tool like phpMyAdmin, one time before and one time after the update, and compare these 2 files with a good code comparison tool like e.g. Beyond Compare. You will see which tables are new in J4, and you will see the differences for the existing tables.
I'm merging this so I can do the merge conflicts from staging on the relevent PRs from staging/3.10 - however would be good to see you finish the testing @joomdonation as additional verification that everything has been done here.
Status | Pending | ⇒ | Fixed in Code Base |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2021-05-02 22:16:32 |
Closed_By | ⇒ | wilsonge | |
Labels |
Added:
?
Removed: ? |
Thanks everybody.
Other than code review and installation checks is there any other way to test? Subject to any side effects from the vaccination I should have time to test this tomorrow