User tests: Successful: Unsuccessful:
Pull Request for fix conflict of #4115 .
conflict fixed and added mssql and postgresql compatibility
This PR consists all the PRs merged in to this one regarding the index additions. Two approaches can be followed in order to test this PR. One is in an already installed site and one is for fresh distributions.
For already installed sites first apply the PR and then add follow the bellow commands in order to add the indexes manually to the database
ALTER TABLE #__languages ADD INDEX 'idx_published_ordering' ('published','ordering');
(#11)
Related Query Before changes
SELECT *
FROM #__languages
WHERE published=1
ORDER BY ordering ASC
Related Query Before changes
SELECT lang_id,lang_code,title,title_native,sef,image,description,
metakey,metadesc,sitename,published,access,ordering
FROM #__languages
WHERE published=1
ORDER BY ordering ASC
ALTER TABLE #__session DROP PRIMARY KEY, ADD PRIMARY KEY(session_id(32))
( #8 )
ALTER TABLE #__session DROP INDEX time, ADD INDEX time(time(10))
Index Description
This Index added in order to speed up the session update query allowing the sub string index usages
ALTER TABLE #__template_styles ADD INDEX idx_client_id('client_id')
( #13 )
Index Description
Without applying this index the below query cannot use any index to retrieve the rows from the database. After applying this index it clearly shows in the explanation section that the query uses the index to retrieve the rows
Related Query
SELECT id, home, template, s.params
FROM #__template_styles as s
LEFT JOIN #__extensions as e
ON e.element=s.template
AND e.type='template'
AND e.client_id=s.client_id
WHERE s.client_id = 0 AND e.enabled = 1
ALTER TABLE #__contentitem_tag_map ADD INDEX idx_alias_item_id ('type_alias','content_item_id')
( #14 )
Index Description
Without applying this index the below query cannot use any index to retrieve the rows from the database. After applying this index it clearly shows in the explanation section that the query uses the index to retrieve the rows. Also one another change did relevant to this index changes. Which is removing the t.*
from the relevant query and adding the table fields instead. This change can be found (#14) and please refer to that PR also.
Related Query Before changes
SELECT m.tag_id,t.*
FROM #__contentitem_tag_map AS m
INNER JOIN #__tags AS t ON m.tag_id = t.id
WHERE m.type_alias = 'com_content.article' AND m.content_item_id = 5196 AND t.published = 1
AND t.access IN (1,1,5)
Related Query After changes
SELECT m.tag_id,'id',t.parent_id,t.lft,t.rgt,t.level,t.path,t.title,t.alias,t.note,t.description,t.published,
t.checked_out,t.checked_out_time,t.access,t.params,t.metadesc,t.metakey,t.metadata,
t.created_user_id,t.created_time,t.created_by_alias,t.modified_user_id,t.modified_time,
t.images,t.urls,t.hits,t.language,t.version,t.publish_up,t.publish_down
FROM #__contentitem_tag_map AS m
INNER JOIN #__tags AS t ON m.tag_id = t.id
WHERE m.type_alias = 'com_content.article' AND m.content_item_id = 5196 AND t.published = 1
AND t.access IN (1,1,5)
ALTER TABLE #__extensions ADD INDEX 'idx_type_ordering' ('type','ordering')
( #16 )
Index Description
Without applying this index the below query uses filesort
to retrieve the rows from the database, which is an expensive retrieval operation. After applying this index it clearly shows in the explanation section that the query uses the index to retrieve the rows except the filesort
.
Related Query
SELECT folder AS type, element AS name, params
FROM #_extensions
WHERE enabled >= 1
AND type ='plugin'
AND state >= 0
AND access IN (1,1,5)
ORDER BY ordering
ALTER TABLE #__menu ADD INDEX 'idx_client_id_published_lft' ('client_id','published','lft')
( #17 )
Index Description
Without applying this index the below query uses filesort
to retrieve the rows from the database, which is an expensive retrieval operation. After applying this index it clearly shows in the explanation section that the query uses the index to retrieve the rows except the filesort
.
Related Query
FROM #_menu AS m
LEFT JOIN #_extensions AS e
ON m.component_id = e.extension_id
WHERE m.published = 1
AND m.parent_id > 0
AND m.client_id = 0
ORDER BY m.lft
ALTER TABLE #__viewlevels ADD INDEX
idx_ordering_title(
ordering,
title)
( #21 )
Index Description
Without applying this index the below query uses filesort
to retrieve the rows from the database, which is an expensive retrieval operation. After applying this index it clearly shows in the explanation section that the query uses the index to retrieve the rows except the filesort
. Also changed the JROOT/libraries/cms/html/access.php
file's query which is shown below by removing the GROUP BY
clause since the PRIMARY KEY
includes in the selected fields. And this reduce the execution time
Related Query Before Change
SELECT a.id AS value, a.title AS text
FROM ltzvy_viewlevels AS a
GROUP BY a.id, a.title, a.ordering
ORDER BY a.ordering ASC,`title` ASC
Related Query After Change
SELECT a.id AS value, a.title AS text
FROM ltzvy_viewlevels AS a
ORDER BY a.ordering ASC,`title` ASC
Status | New | ⇒ | Pending |
Labels |
Added:
?
|
Category | ⇒ | SQL |
@richard67 should work with utf8mb4 too, yes it is very old comes from gsoc 2014
This is what I did:
With all the changes in the installer I concluded that this procedure is not possible anymore.
Well as I just checked in "libraries/cms/schema/changeitem/mysql.php", the database fix does not handle "DROP PRIMARY KEY" and "ADD PRIMARY KEY" statements, so these will not be detected by the databaase fix as structural change and so will be skipped.
Furthermore, the "DROP INDEX" and "ADD INDEX" statements in 1 row are also not detected as a structural change, because the database fix checks only for index names, not for details.
But the "ADD INDEX" for really new indexes should have worked.
So as it is now, the schema updates of this PR will only be applied when using Joomla! Update component for updating, but not when using the "copy or extract the files and use the database fix" method.
But regardless of all that, the correction for utf8mb4 I provided above has to be made.
@wilsonge Please confirm what I found.
we already have some sql on update to 3.6.0 that for what i understand from your comments will not work ... https://github.com/joomla/joomla-cms/tree/staging/administrator/components/com_admin/sql/updates/mysql
so the Database -> Fix behaviour needs to be Fixed
Yes, is all known issues. Just wanted to explain here so people will not test in the wrong way.
But despite of this, my correction for utf8mb4 has to be applied, otherwise 1 of these new indexes will not work because max index size exceeded.
yes richard i understand that
I have tested this item
Tested by patching + database fix
This is what I did:
Done a code review and all seems fine.
PS: While checking i notice some intensive queries (example: SHOW FULL COLUMNS FROM xxxxx_assets
) and others with "filesort" or "NO INDEX KEY COULD BE USED" but they are already there before the PR.
I want to test that PR.
I have 2 requests/questions:
[DELETED]
In #__contentitem_tag_map
there is no unique/primary key. This is bad because mysql create hidden additional primary key for that.
I suggest to create a UNIQUE KEY or PRIMARY KEY as in example:
ALTER TABLE `#__contentitem_tag_map` ADD UNIQUE `idx_cid_type_alias_tag_id` (`type_alias`, `content_item_id`, `tag_id`);
Can you look at resolving the conflicts so that this can be tested please or is it so out of date that it should be closed?
It has been 10 months since the request for this pr to be updated without reply. I am going to close it as an abandoned issue. It can always be reopened if updated
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2018-07-21 21:55:48 |
Closed_By | ⇒ | brianteeman |
Category | SQL | ⇒ | SQL Administration com_admin Postgresql MS SQL Installation Libraries |
Do all the new indexes work with utf8mb4? I ask because it seems to be a bit older.