Feature PR-6.0-dev Performance Pending

User tests: Successful: Unsuccessful:

avatar Fedik
Fedik
14 Sep 2024

Pull Request for Issue #43649.

Summary of Changes

Currently while checking access, the Access::preload() pulling everything from the access table.
It is neglectable for small sites, however for large sites it has huge impact on the site performance.

I added a new method for "smart preload", which allows to preload only needed items. And applied this method for most of the com_content views, where it make sense.

It works as following:

In the View class (or other places before access check) developer call to pre-load access for loaded items with (example for com_content):

$assetsList = ['com_content.article.1', 'com_content.article.2', 'com_content.category.1'];
Access::preloadItems('com_content',  $assetsList);

Where the $assetsList is a list for asset name for $items.

This will load access information for these items and its parent in to memory.
And each following call of $user->authorise('core.xxx', $assetName) will reuse preloaded item.

Testing Instructions

Create a site with 20k articles (more is better). Can use following plugin to generate random content https://github.com/Fedik/plg_sampledata_bigdata

Enable debug.
Open article detail or blog view.
Look for memory usage in the debug bar, and for time.

Actual result BEFORE applying this Pull Request

In my test:
memory usage around 45Mb
time 200-300ms

Expected result AFTER applying this Pull Request

In my test:
memory usage around ~12Mb
time ~100ms

Link to documentations

Please select:

  • Documentation link for docs.joomla.org:
  • No documentation changes for docs.joomla.org needed
  • Pull Request link for manual.joomla.org:
  • No documentation changes for manual.joomla.org needed
avatar Fedik Fedik - open - 14 Sep 2024
avatar Fedik Fedik - change - 14 Sep 2024
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 14 Sep 2024
Category Administration com_categories com_content Front End Libraries
avatar Fedik
Fedik - comment - 14 Sep 2024

Will rebase it after upmarge

avatar Fedik Fedik - change - 14 Sep 2024
The description was changed
avatar Fedik Fedik - edited - 14 Sep 2024
avatar Fedik Fedik - change - 14 Sep 2024
Labels Added: Feature bug PR-5.2-dev
avatar Fedik Fedik - change - 14 Sep 2024
Title
[5.3][Performance] Access smart preload, reduce memory use and time for large sites
[5.3][Performance] Access Rules smart preload, reduce memory use and time for large sites
avatar Fedik Fedik - edited - 14 Sep 2024
avatar Fedik
Fedik - comment - 14 Sep 2024

Is it faster if you add these indexes?
ALTER TABLE `#__assets` ADD INDEX `idx_lft` (`lft`), ADD INDEX `idx_rgt` (`rgt`);

Nope, unfortunately. Before applying the index it took ~250ms, and after ~250ms.

It probably skipping index because of range check:
Screenshot 2024-09-14_14-09-28

avatar Fedik Fedik - change - 14 Sep 2024
The description was changed
avatar Fedik Fedik - edited - 14 Sep 2024
avatar Fedik Fedik - change - 14 Sep 2024
The description was changed
avatar Fedik Fedik - edited - 14 Sep 2024
avatar Fedik Fedik - change - 14 Sep 2024
The description was changed
avatar Fedik Fedik - edited - 14 Sep 2024
avatar Fedik Fedik - change - 14 Sep 2024
The description was changed
avatar Fedik Fedik - edited - 14 Sep 2024
avatar Fedik Fedik - change - 1 Oct 2024
The description was changed
avatar Fedik Fedik - edited - 1 Oct 2024
avatar Fedik Fedik - change - 3 Oct 2024
Labels Added: Performance
Removed: bug
avatar Fedik Fedik - change - 18 Jan 2025
Labels Added: PR-5.3-dev
Removed: PR-5.2-dev
c1b507d 18 Jan 2025 avatar Fedik cs
avatar Fedik Fedik - change - 1 Feb 2025
Title
[5.3][Performance] Access Rules smart preload, reduce memory use and time for large sites
[5.3][Performance] Access Rules smart preload, reduce memory and time use, for large sites
avatar Fedik Fedik - edited - 1 Feb 2025
avatar HLeithner
HLeithner - comment - 4 Mar 2025

This pull request has been automatically rebased to 6.0-dev.

avatar HLeithner HLeithner - change - 4 Mar 2025
Title
[5.3][Performance] Access Rules smart preload, reduce memory and time use, for large sites
[6.0] [Performance] Access Rules smart preload, reduce memory and time use, for large sites
avatar HLeithner HLeithner - edited - 4 Mar 2025
avatar Fedik Fedik - change - 31 May 2025
Labels Added: PR-6.0-dev
Removed: PR-5.3-dev
avatar Fedik
Fedik - comment - 7 Jun 2025

@SniperSister please have a look also this one. In combination with #45542 it should be a bigger improvement in performance for the Content component for large sites.

avatar Fedik Fedik - change - 10 Jun 2025
The description was changed
avatar Fedik Fedik - edited - 10 Jun 2025
avatar Fedik Fedik - change - 10 Jun 2025
The description was changed
avatar Fedik Fedik - edited - 10 Jun 2025
avatar Fedik
Fedik - comment - 10 Jun 2025

I updated the PR description, with example how it works.

Add a Comment

Login with GitHub to post a comment