User tests: Successful: Unsuccessful:
I want to fix a few bugs in memcache structure:
Index structure before:
array(
stdobject(name="[secret]-cache-[com-content]-[cache_id]", size=1024),
stdobject(name="[secret]-cache-[com-content]-[cache_id]", size=1024),
stdobject(name="[secret]-cache-[com-content]-[cache_id]", size=1024),
stdobject(name="[secret]-cache-[com-content]-[cache_id]", size=1024),
...
)
Index structure after:
Now index is separated into more arrays ($index and $index2 per group key)
$index is array of group_keys. Example: array("com_contentE", "com_contentF");
$index = array(
[_system],
[com_contentA],
[com_contentB],
[com_contentC],
...,
[com_contentE],
[com_contentF],
...
);
// index2 has address at "[secret]-[group_key]-index]"
$index2 = array(
"[com_content]-[md5sum_of_cache]" => 1025,
"[com_content]-[md5sum_of_cache]" => 1325,
"[com_content]-[md5sum_of_cache]" => 1425,
...
);
Instead of 1 index we will have 1 + total group_keys indexes and then lock time won't be a problem.
This patch contains some debug staff.
You can see log debug in logs/memcache.php after you click on "Clear Expired Cache" on backend.
In progress. Please code review before.
Status | New | ⇒ | Pending |
Labels |
Added:
?
|
Category | ⇒ | Cache |
I have reverted to throw exception.
Cache index in memcache storage of joomla 3.5
Example cache id:
$cache_id = "11112222333344445555666677778888-cache-_system-429adb9b368342c818ec393fb9780000"
JOOMLA 3.5 index structure:
$index = array(
stdClass($name='11112222333344445555666677778888-cache-_system-429adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-com_content-029adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-com_content-039adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-com_content-349adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-com_content-329adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-mod_menu-139adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-mod_menu-149adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-mod_menu-229adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-mod_menu-239adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-mod_menu-249adb9b368342c818ec393fb9780000', $size=1024),
stdClass($name='11112222333344445555666677778888-cache-mod_menu-259adb9b368342c818ec393fb9780000', $size=1024),
...
stdClass($name='11112222333344445555666677778888-cache-mod_menu-269adb9b368342c818ec393fb9780000', $size=1024),
);
Serialize and unserialize $index for methods like get/remove/clean/store takes a lots of time.
Array keys are numbers (long int) and may overflow for long running memcache/php.
Cache index in the new memcache storage will be split into more indexes
Primary $index and secondary indexes $index2 (per group key).
Group key is a combination of group name ex. 'com_content' and first char (hex value 0-9a-f) from md5sum or only group name otherwise:
11112222333344445555666677778888-cache-mod_menu-269adb9b368342c818ec393fb9780000
11112222333344445555666677778888-cache-_system-429adb9b368342c818ec393fb9780000
For technical reason I have to replaced group key 'mod_menu2' to 'mod_menuC'.
Group key is generated by code below:
$key = $group;
// If the group does not begin with '_' and does not end with number
// then add suffix A or B or ... or P.
// For groups like _system or com_custom_part1 do not add suffix
if ($group[0] !== '_' && !ctype_digit(substr($group, -1, 1)))
{
// The cache id suffix looks like:
// com_content-329adb1b3633424818ec393fb9780000 and then key will be com_contentD
// Total can be 16 different keys per one group
$key .= chr(hexdec($cache_id_sfx[strlen($group) + 1]) + 65);
}
return $key;
Now primary index is:
$index = array(
'_ssytem',
'com_contentA',
'com_contentD',
'mod_menuB',
'mod_menuC',
...
);
and secondary $index2 (per group key) looks like this example for 'mod_menuC'
//$index2 = $memcache→get('11112222333344445555666677778888-mod_menuC-index');
$index2 = array(
'mod_menu-229adb9b368342c818ec393fb9780000' => 1024,
'mod_menu-239adb9b368342c818ec393fb9780000' => 1024,
'mod_menu-249adb9b368342c818ec393fb9780000' => 1024,
'mod_menu-259adb9b368342c818ec393fb9780000' => 1024,
);
It more looks like a tree structure, where index is parent and indexes of index2 are leafs.
I have plan to remove some debug logs and may be memcache log file at all.
Error/warning logs will be changed a little.
I want to stay with debug logs for a testing and better understanding current changes.
If there is still issue with my description may be should I create some graph about cache indexing?
Title |
|
Saving data failed for com_jmapA (size: 2538849, compression: 0)
Which means that memcached server can not store values bigger that 1MB, but when we enable compression then memcache store that variable.
Lock index2 failed for mod_menuK after 0.037997
Timeout after 0.037997 second, server is very busy
Total index2 arrays: 172, max full index2 was com_contentK: 127
Total expired: 1198, max saving time in last 24h: 0.109072
This became from (gc function) button "Clear expired cache".
There is 172 arrays of index2, the biggest array of index2 contains 127 items.
Gc function removed from index2 arrays 1198 items and max time of store cache takes 0.109072 second.
Category | Cache | ⇒ | Libraries Cache |
Labels |
Added:
?
|
Labels |
Added:
?
|
As memcache is not used for php 7.x and
I have no more interest with it I close it.
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2017-06-28 10:59:51 |
Closed_By | ⇒ | csthomas |
As a high level API class, the cache storage adapters shouldn't be tied to
the application classes or making decisions on whether a message gets
enqueued for display or not. The adapter should simply fail and a lower
level class should be handling that error. So yes, the adapter should be
throwing an Exception and only that. Tying it to the application classes
creates a very deep level of coupling to those specific implementations and
the static global factory, both of which should be minimized as practical
in the high level library classes.
On Sunday, May 22, 2016, Tomasz Narloch notifications@github.com wrote: