No Code Attached Yet Information Required bug
avatar Stuartemk
Stuartemk
29 Feb 2020

Steps to reproduce the issue

mysqldumpslow -a -s r -t 1 /var/log/mysql/mysql-slow.log

Reading mysql slow query log from /var/log/mysql/mysql-slow.log
Count: 621 Time=0.01s (7s) Lock=0.00s (0s) Rows=23347.0 (14498487), server[server]@localhost
SELECT id,name,rules,parent_id
FROM web_assets
WHERE name LIKE 'com_content.%' OR name = 'com_content' OR parent_id = 0

mysqldumpslow -t 1 -s at -r /var/log/mysql/mysql-slow.log

Reading mysql slow query log from /var/log/mysql/mysql-slow.log
Count: 594 Time=0.01s (6s) Lock=0.00s (0s) Rows=23347.0 (13868118), server[server]@localhost
SELECT id,name,rules,parent_id
FROM web_assets
WHERE name LIKE 'S' OR name = 'S' OR parent_id = N

Expected result

The results would be indexed or called only when changing, adding or deleting information

Actual result

Server overload on a news page with accumulated daily new articles.

System information (as much as possible)

Setting Value
 
PHP Built On Linux server 5.3.0-40-generic #32~18.04.1-Ubuntu SMP Mon Feb 3 14:05:59 UTC 2020 x86_64
Database Type mysql
Database Version 5.7.29-0ubuntu0.18.04.1-log
Database Collation utf8_general_ci
Database Connection Collation utf8mb4_general_ci
PHP Version 7.2.24-0ubuntu0.18.04.3
Web Server Apache/2.4.29
WebServer to PHP Interface fpm-fcgi
Joomla! Version Joomla! 3.9.15 Stable [ Amani ] 27-January-2020 15:00 GMT
Joomla! Platform Version Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT

Additional comments

avatar Stuartemk Stuartemk - open - 29 Feb 2020
avatar joomla-cms-bot joomla-cms-bot - change - 29 Feb 2020
Labels Added: ?
avatar joomla-cms-bot joomla-cms-bot - labeled - 29 Feb 2020
avatar alikon
alikon - comment - 29 Feb 2020

i'm afraid no indexes can help on a query with LIKE and OR
anyway how many articles do you have ? 20K ?

avatar Stuartemk
Stuartemk - comment - 29 Feb 2020

@alikon @wilsonge @SharkyKZ @richard67 @brianteeman @astridx @C-Lodder @Quy @zero24 @zero-24

On a news page with 10 categories such as sports, finance, technology, etc., and with 10 subcategories per category such as soccer, baseball, tennis, etc., adding an article by subcategory and also multilanguage, it is not an exaggeration or crazy to arrive soon to quantities of more than 20,000 items, since the items are also saved and not deleted. Joomla is an excellent and extraordinary CMS for its modular system, user management, flexibility and diversity compared to any other CMS, giving it tremendous power and potential, but one of its few adversities against other CMS such as Drupal or Wp is that since its origin Joomla has a big problem in the structure of the way he handles large volumes of articles and the same thing happens with menus; Neither Drupal nor WP have problems with this handling of large volumes of Articles, specifically the structure of the assets kills all the benefits and benefits of Joomla over other CMS. If it is possible to eliminate the bottleneck of the assets in Joomla, it would no longer have a rival as the best CMS and the same with the bottleneck of large volumes of menus and submenus.

avatar HLeithner
HLeithner - comment - 29 Feb 2020

Could a fulltext index be used here?

avatar alikon
alikon - comment - 1 Mar 2020

iirc FULLTEXT index can be used starting mysql ≥ 5.6

avatar HLeithner
HLeithner - comment - 1 Mar 2020

True but would it help j4+?

avatar Stuartemk
Stuartemk - comment - 1 Mar 2020

@HLeithner @alikon @wilsonge @SharkyKZ @richard67 @brianteeman @astridx @C-Lodder @Quy @zero24 @zero-24

Possibly there is ignorance of me in relation to the existence or structure of the Assets, but it definitely has an error in the structure that goes beyond creating indexes, since for example imagine that you have 100,000 word documents on your computer and that for create, change, or delete a word file the computer had to make a comparison with the 100,000 documents ... because that makes joomla with the Assets, it's crazy! It only makes sense for very specific cases in which it is required to do a search for a specific comparative such as finding repeated files and even then joomla does it wrong because it does not define author or any date but does so with 100% of the files that are much worse because what it does with the Assets is also with the plugins, themes, modules etc. since the Assets are also all of them, it is as if you were going to create a word document and the non-only computer will search in the 100,000 Word documents but also include all excel, powerpoint, etc, etc, etc, perhaps only excluding images. It is definitely an error of logic, structure and design.

avatar ssnobben
ssnobben - comment - 13 Jun 2020

@Stuartemk did you come up with a solution for Joomla for this?

avatar Fedik
Fedik - comment - 13 Jun 2020

@Stuartemk for note: spamming everyone will not help to solve the issue at all, even opposite.

Just showing "slow logs" does not helps much, it may be due to bad server performance.
Would be much useful if you tell more detail and be more specific, eg: in which cases you have a slow down: when add/edit content, or when doing something else.

All this water about windows and files will no one read.

avatar Quy Quy - change - 14 Jun 2020
Labels Added: Information Required
avatar Quy Quy - labeled - 14 Jun 2020
avatar Notebit
Notebit - comment - 20 Apr 2021

I join cause I also have the same issue, a slow query on assets table with a com_content of more than 30K articles, any workaround sofar?
That query is executed every time a com_content page is loaded in frontend, with or without cache, and as far as I can see, that query is executed even if I load the page as a visitor or guest

avatar Notebit
Notebit - comment - 30 Apr 2021

small update: the assets query takes 22ms in joomla debug but if I run the same query in phpmyadmin it's blazing fast

avatar Hackwar
Hackwar - comment - 6 Nov 2022

@Notebit 22ms would actually not be slow in my opinion. In any case, here is a PR which should reduce the number of rows in that table, improving performance especially for large sites like these: #39165
Please test that PR and see if this helps with your issues.

avatar Hackwar Hackwar - change - 21 Feb 2023
Labels Added: No Code Attached Yet bug
Removed: ?
avatar Hackwar Hackwar - labeled - 21 Feb 2023

Add a Comment

Login with GitHub to post a comment