No Code Attached Yet Performance
avatar MayaSima
MayaSima
13 Jun 2024

Description:

I am experiencing performance issues with Joomla permissions on version 5.1.1. I created a group intended to have backend access, specifically to create or edit content. While everything works fine in general, accessing Content >> Articles is extremely slow and often times out or crashes the page. This issue arises when attempting to list articles. However, this problem does not occur when you log in as a Super User.

Steps to Reproduce:

  1. Create a new user group with permissions to create or edit content.
    
  2. Log in as a user from this group.
    
  3. Navigate to Content >> Articles.
    

Expected Result:

The articles list should load efficiently without significant delay or timeouts.

Actual Result:

The page loads extremely slowly, often timing out and crashing.

Debugging Information:

Using the Joomla debugging tool, we discovered that:

  • For non-Super Users, Joomla checks the asset table permissions for each article individually.
    
  • This behavior does not occur with Super Users.
    

Screenshot:

The attached screenshot shows a profile trace comparison:

  •     Left: Profile trace of a Super User.
    
  •     Right: Profile trace of a user with set permissions.
    

Attachment

Screenshot 2024-06-13 143030

Question:

Can you confirm if this is a bug or native Joomla behavior?

Environment:

  • Joomla Version: 5.1.1
    
  • Number of Articles: ~9000
    

Additional Information:

The issue seems to be tied to how Joomla processes permissions for non-Super Users, causing a performance bottleneck when listing a large number of articles. Any insights or workarounds to alleviate this issue would be greatly appreciated.

avatar MayaSima MayaSima - open - 13 Jun 2024
avatar joomla-cms-bot joomla-cms-bot - change - 13 Jun 2024
Labels Added: No Code Attached Yet
avatar joomla-cms-bot joomla-cms-bot - labeled - 13 Jun 2024
avatar Fedik Fedik - change - 13 Jun 2024
Labels Added: bug
avatar Fedik Fedik - labeled - 13 Jun 2024
avatar Fedik
Fedik - comment - 13 Jun 2024

This happens because Access::preloadPermissions() preloads permissions for ALL 9000 articles
(which is around 90MB from your screenshot)

protected static function preloadPermissions($assetType, $reload = false)

I set it as bug, however I have no idea how it can be fixed.
But I can understand that it gives a really bad experience of use the content editing.

It probably for all Joomla versions. Cannot find older similar issue about it.

avatar Fedik
Fedik - comment - 13 Jun 2024

Any insights or workarounds to alleviate this issue would be greatly appreciated.

You can try hack the

public static function check($userId, $action, $assetKey = null, $preload = true)

And change $preload to $preload = false

However it is illegal. And I told you nothing, someone hacked my account. This message will self destroyed in few days :neckbeard:

avatar MayaSima
MayaSima - comment - 17 Jun 2024

I tried changing $preload to $preload = false in joomla-cms/libraries/src/Access/Access.php at line 155, but the issue still persists.

avatar Fedik
Fedik - comment - 17 Jun 2024

Okay, I think now instead one large DB query you got a few hundrets of small DB queries. That is why it is still slow.

Another thing to try is to plreload only needed items, that in the current view.
Try, edit the articles list layout administrator/components/com_content/tmpl/articles/default.php
And place following code for preload, somewhere around line 70 (before the html rendering):

$accessWorkAround = new class () extends \Joomla\CMS\Access\Access
{
    // Preload only needed items
    public static function verySmartPreload($assetType, $assetsList, $key = 'id')
    {
        $extensionName = self::getExtensionNameFromAsset($assetType);

        /** @var \Joomla\Database\Mysql\MysqlDriver $db */
        $db    = Factory::getContainer()->get(\Joomla\Database\DatabaseInterface::class);
        /** @var \Joomla\Database\Mysql\MysqlQuery $query */
        $query = $db->getQuery(true);
        $query->select($db->quoteName(['id', 'name', 'rules', 'parent_id']))
            ->from($db->quoteName('#__assets'))
            ->where(
                $db->quoteName($key) . ' IN ('
                . implode(',', $query->bindArray($assetsList, \Joomla\Database\ParameterType::STRING)) . ')'
            );

        $assets = $db->setQuery($query)->loadObjectList();

        foreach ($assets as $asset) {
            self::$assetPermissionsParentIdMapping[$extensionName][$asset->id] = $asset;
            self::$preloadedAssets[$asset->id]                                 = $asset->name;
        }

        // Temporary mark as preloaded
        self::$preloadedAssetTypes[$assetType] = true;
    }

    public static function resetVerySmartPreload($assetType)
    {
        unset(self::$preloadedAssetTypes[$assetType]);
    }
};
$assetsList = [];
foreach ($this->items as $item) {
    $assetsList[] = 'com_content.article.' . $item->id;

    if (!empty($item->catid)) {
        $assetsList[] = 'com_content.category.' . $item->catid;
    }
}

$accessWorkAround::verySmartPreload('com_content.article', $assetsList, 'name');

then at the end of the file:

<?php $accessWorkAround::resetVerySmartPreload('com_content.article'); ?>

And please tell us if it help.
The editing form still will be slow, however the list view should be much faster.

avatar Fedik
Fedik - comment - 8 Sep 2024

After some testing, I think the actual slowdown is somewhere in ‎ArticlesModel::getListQuery() in the Administrator model

->select(
[
$db->quoteName('fp.featured_up'),
$db->quoteName('fp.featured_down'),
$db->quoteName('l.title', 'language_title'),
$db->quoteName('l.image', 'language_image'),
$db->quoteName('uc.name', 'editor'),
$db->quoteName('ag.title', 'access_level'),
$db->quoteName('c.title', 'category_title'),
$db->quoteName('c.created_user_id', 'category_uid'),
$db->quoteName('c.level', 'category_level'),
$db->quoteName('c.published', 'category_published'),
$db->quoteName('parent.title', 'parent_category_title'),
$db->quoteName('parent.id', 'parent_category_id'),
$db->quoteName('parent.created_user_id', 'parent_category_uid'),
$db->quoteName('parent.level', 'parent_category_level'),
$db->quoteName('ua.name', 'author_name'),
$db->quoteName('wa.stage_id', 'stage_id'),
$db->quoteName('ws.title', 'stage_title'),
$db->quoteName('ws.workflow_id', 'workflow_id'),
$db->quoteName('w.title', 'workflow_title'),
]
)
->from($db->quoteName('#__content', 'a'))
->where($db->quoteName('wa.extension') . ' = ' . $db->quote('com_content.article'))
->join('LEFT', $db->quoteName('#__languages', 'l'), $db->quoteName('l.lang_code') . ' = ' . $db->quoteName('a.language'))
->join('LEFT', $db->quoteName('#__content_frontpage', 'fp'), $db->quoteName('fp.content_id') . ' = ' . $db->quoteName('a.id'))
->join('LEFT', $db->quoteName('#__users', 'uc'), $db->quoteName('uc.id') . ' = ' . $db->quoteName('a.checked_out'))
->join('LEFT', $db->quoteName('#__viewlevels', 'ag'), $db->quoteName('ag.id') . ' = ' . $db->quoteName('a.access'))
->join('LEFT', $db->quoteName('#__categories', 'c'), $db->quoteName('c.id') . ' = ' . $db->quoteName('a.catid'))
->join('LEFT', $db->quoteName('#__categories', 'parent'), $db->quoteName('parent.id') . ' = ' . $db->quoteName('c.parent_id'))
->join('LEFT', $db->quoteName('#__users', 'ua'), $db->quoteName('ua.id') . ' = ' . $db->quoteName('a.created_by'))
->join('INNER', $db->quoteName('#__workflow_associations', 'wa'), $db->quoteName('wa.item_id') . ' = ' . $db->quoteName('a.id'))
->join('INNER', $db->quoteName('#__workflow_stages', 'ws'), $db->quoteName('ws.id') . ' = ' . $db->quoteName('wa.stage_id'))
->join('INNER', $db->quoteName('#__workflows', 'w'), $db->quoteName('w.id') . ' = ' . $db->quoteName('ws.workflow_id'));

it takes a few seconds to load 20 articles from 40k on my test.
If I remove workflow Joins, it still more than a second.
Interestingly ‎ArticlesModel::getListQuery() in the Site model works much faster.
@richard67 do you have an idea what can be a bottleneck in this query?

Access preloading still preloads a lot, but it is doing it fast.

avatar Fedik
Fedik - comment - 8 Sep 2024

It sems a few things:

  • workflow joins,
  • languages joins,
  • users joins,

When I removing them, everything works much faster.

avatar Fedik
Fedik - comment - 8 Sep 2024

Ah, it the same as #43701 I missed it, sorry.

avatar richard67
richard67 - comment - 8 Sep 2024

@richard67 do you have an idea what can be a bottleneck in this query?

@Fedik No idea. Is already the part which you have linked (lines 232 to 266) so slow, without all that stuff which might be added later below? What I remember from some other issue is that it could be faster when changing the 3 inner joins to the workflow tables to left joins:

->join('INNER', $db->quoteName('#__workflow_associations', 'wa'), $db->quoteName('wa.item_id') . ' = ' . $db->quoteName('a.id'))
->join('INNER', $db->quoteName('#__workflow_stages', 'ws'), $db->quoteName('ws.id') . ' = ' . $db->quoteName('wa.stage_id'))
->join('INNER', $db->quoteName('#__workflows', 'w'), $db->quoteName('w.id') . ' = ' . $db->quoteName('ws.workflow_id'));

avatar richard67
richard67 - comment - 8 Sep 2024

Ah, it the same as #43701 I missed it, sorry.

@Fedik Yes, that's the one where I saw the thing with inner or left join for the workflow tables.

avatar Fedik
Fedik - comment - 8 Sep 2024

Yea, with LEFT it 0.5s faster, but still slow in total 😄
Hmhm

it probably would be much smarter to split it to few queries:
load Articles,
load language data
load User data
load workflow data

Okay, maybe not that easy.

avatar brianteeman
brianteeman - comment - 8 Sep 2024

Kill workflow, problem solved :)

avatar Fedik
Fedik - comment - 8 Sep 2024

Kill workflow, problem solved

It is not only workflow

avatar brianteeman
brianteeman - comment - 8 Sep 2024

but it makes no sense to query workflow tables when its not being used.

avatar Fedik Fedik - change - 14 Sep 2024
Status New Closed
Closed_Date 0000-00-00 00:00:00 2024-09-14 10:03:48
Closed_By Fedik
avatar Fedik Fedik - close - 14 Sep 2024
avatar Fedik
Fedik - comment - 14 Sep 2024

Issue with access check hopefully will be fixed in #44078.
I close this issue for now. Thanks for reaport.

avatar Fedik Fedik - change - 15 Sep 2024
Labels Added: Performance
Removed: bug
avatar Fedik Fedik - unlabeled - 15 Sep 2024
avatar Fedik Fedik - labeled - 15 Sep 2024

Add a Comment

Login with GitHub to post a comment