Why use such a low PHP memory limit?

The default memory limit in php before 5.2 was 8MB
it was increased to a default of 16MB in php 5.2.0
It is currently a default of 128MB

Hi Phil,

Point taken, and that may be a temporary solution, but I think you may be missing the scalability issues that this method is causing. Without that method Joomla is using ~20MB for a Blog view (loading 20 articles) on my install. With that method being called the same page is requiring ~75MB to load. In other words, that one function is accounting for 73% of the memory usage. That amount is only going to grow as more articles are created.

I don't yet know the full purpose of that method though so I don't know what the trade off is (ie, what is Joomla benefiting from that function that justifies its use?).

If you have 30,000+ com_content items then surely you would be on a decent server ;)

AFAIK the asset preload methods were created to avoid make a query for each asset being checked in a page (which, depending on the page, can be a lot: modules, components, articles, categories, and so on), ie, is used to make only one query for all assets used of a particular type (ex: com_content.article.*) + all component assets + the root asset. In summary, AFAIK was created for a performance otimization.

See https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/access/access.php#L347

This can have that side effect on sites with large number of assets, ie, is not particular good for large scale installations because of big asset tables, but should improve performance in small/medium installations.

I see a few solution (non B/C too).
May be some of it could be applied.

1 Each asset could be moved to table row of item and works similar to params:

$item->params = Registry($item->params);
$item->asset = JAsset($item->asset);

2 Use leftJoin for each item:

SELECT c.*, a.rules FROM #__content c LEFT JOIN #__assets a ON c.asset_id = a.id

and for other assets which name does not contains two dots load them normal as before.
Then in preload will be less items.

3 Do not by default add assets to article (to item in general) if the parent asset rules are identical.
Fresh joomla by default for com_content has:

{"core.admin":{"7":1},"core.manage":{"6":1},"core.create":{"3":1},"core.edit":{"4":1},"core.edit.state":{"5":1}}

category:

{}

article:

{"core.admin":{"7":1},"core.manage":{"6":1},"core.create":{"3":1},"core.edit":{"4":1},"core.edit.state":{"5":1}}

Now article asset rules are almost redundant (current there is no inheritance, but if I remove it then there will be inheritance from com_content).

Hi andrepereiradasilva, csthomas,

Thank you for that information. As I continued my research into this I discovered that this method (and JAccess in general) was modified in the upcoming 3.7 release to be less memory intensive and in 4.0 this method will be completely deprecated (returning void). See #12809

That being the case I'm going to close this ticket, as there is already a solution being implemented in future releases.

Thanks everyone.

@csthomas i think at least the 3 can and should be done without B/C break. probably that is actually considered a bug because it seems joomla is forcing the parent asset permissions when item is created (at least this seems to happen when you create a module) instead of allowing item to inherit parent (com_modules in this case) permissions.
That means if you change parent permissions it will not reflect on the current modules items, so it seems is breaking inhertence. Unless i'm missing something.

@drewgg yes it was changed but i think that will not solve your problem. But please test, replace your 3.6.5 access.php file by the joomla 3.7.0 access.php, should be a drop-in file replace. And see if improves.

For now 3rd option is good I agree. I would like to see button "Clear redundant permissions" or some similar functionality when I update to 3.7.

Maybe someone willing to write a PR?

@andrepereiradasilva I finished reading that PR thread and your replacement RP that was merged for 3.7. So I'm caught up in that respect (nice work!). I switched out access.php with the 3.7.0 and compared a few runs of each. I'm getting about a 10mb savings between versions.

@csthomas Could you explain how that would solve this issue as it relates to JAccess? Is the idea that you'd still pull all the asset rows (in my case the ~29,000) but only store the ones are different from their ancestors (resulting in a much smaller array)?

yes, that would be the idea i think, but don't know if it will reduce a lot of memory, but will probably improve performance a bit.

As long as the hash of the recursive asset rules for a item already exists in memory, it will not be saved again in memory, thus saving memory. See https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/access/access.php#L615-L629

for instance, if an article as an empty asset rule {} the JAccessRule for that asset would not be saved in memory because is the same of it's parent category asset, neither needs to be recalculated (will just use parent category asset rule already precalculated in memory).

But probably the main issue you are facing now with 30.000 articles (30.000 com_content.article.* assets) is probably the memory usage of saving all articles assets in memory when preloading here https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/access/access.php#L365 and that will probably not be solved with this enchancement.

That would probably requires @csthomas suggestions 1 or 2 or something like that.

Every time when you add a new article a new row to Assets is added.
Joomla does not check if it is required or not.

Usual user does not change permission per article so you do not need to add that Assets to database because it can be calculated/inherited from global com_content and category assets rules.

If you never change permission in article edit form then I guess that you have a lots of redundant rows.

I suggest to find some more info on forum.joomla.org.

@csthomas yes, but "as it is" don't forget you need the asset hierarchy relation, that's why you need an empty com_content.article.* asset for each article - that asset has a parent_id which is the parent category asset id.

IIRC without that com_content.article.* asset row JAccess will fallback to com_content asset and not to the parent category asset. So is not the same thing

When I'm watching on code in com_content article model I would like to have something like below but it is not B/C.

// Use if previous does not exists in database
$fallback_asset_name = 'com_content.category.1';

if ($user->authorise('core.edit', 'com_content.article.234', $fallback_asset_name))

Update:
I ended up modifying the 3.7 version of access.php to speed things up. What I did:

1. One thing I noticed is that when getAssetRules() is called if an asset is not found in the $assetPermissionsParentIdMapping array (the array that gets populated from preloading) then getAssetRules() gets that asset's rules from the database (obviously). However, once done, it does not update the $assetPermissionsParentIdMapping or $preloadedAssets arrays. So if the rules for that asset are requested again (which happens nearly every page load and sometimes the same asset is requested 10+ times) it gets those rules the slow way every time instead of utilizing $assetPermissionsParentIdMapping which would be much faster. I patched my JAccess to fix this.

2. The next thing I did was have the three preloading methods (JAccess::preloadPermissionsParentIdMapping(), JAccess::preloadPermissions(), JAccess::preloadComponents()) no longer grab every row (of the same component) from the assets table. Now it only grabs the 100 most recently requested rows.

I did this via a new table, #__assets_cache, that has two columns: id_asset (foreign key to #__assets.id) and last_seen (TIMESTAMP with UPDATE ON MODIFY set). When the preloading methods are called they'll only grab the rows from #__assets which are also present in #__assets_cache. Whenever an asset rule is requested that's not part of that preloaded group (ie, wasn't in #__assets_cache) then it'll...

A) Get the rules the slow way
B) Update $assetPermissionsParentIdMapping and $preloadedAssets (as already mentioned)
C) Add a row to #__asset_cache for this newly seen asset
D) "Truncate" #__asset_cache to 100 rows

So what I'm doing is preloading the most recently 100 asset rules, which should cover 90+% of requests to getAssetRules(). Then any requests for assets not in that group of 100 are retrieved, cached right away, and added to the most recent list of 100 (pushing out the oldest one) so they'll be part of the preloading next time.

I'm not sure if this is something worth trying to submit to the core but it seems to be working well for me. It should be B/C too, though I haven't attempted to look into that aspect yet.

The point 1 should be easy to merge. Do you want to create a PR for that?
Then I/we can do a review and after two success tests IMHO it may go to forward to merge to joomla 3.7.

I have created a PR #14279 which should help you.

Blindly preloading all assets is a not-well-though-trough idea than leads to issues like this. An answer to performance problems is a database normalization (replace json rules with related table) plus a method to do authorization of mutiple items in one call (and with single query).

I have same problem.

#14268 has been closed and not merge. How do I handle it?

There are 123.900 records in my com_content.

Capture debug of category blog layout.

Capture debug or article single.