Looks like this PR reveals a memcache issue with our storage controller and HHVM
Fatal error: [] operator not supported for strings in /libraries/joomla/cache/storage/memcache.php on line 194

Look like HHVM doesn't like " assigning values to the array by omitting the key, resulting in an empty pair of brackets ([])."

$arr[key] = value;
$arr[] = value;
// key may be an integer or string
// value may be any value of any type

If $arr doesn't exist yet, it will be created, so this is also an alternative way to create an array. This practice is however discouraged because if $arr already contains some value (e.g. string from request variable) then this value will stay in the place and [] may actually stand for string access operator. It is always better to initialize variable by a direct assignment.

I believe we can fix this by changing line 194 from $index[] = $tmparr; to array_push($index, $tmparr);

I can change it but I think that HHVM do not use unserialize on $index.

$index all time should be array or false if not set.
Memcache implicitly serializes array to string and stores that string in memory.
May be HHVM do not use unserialize after get variable and we have to do it explicitly.

From what I can tell the following line is returning a string rather than an array or false.

$index = static::$_db->get($this->_hash . '-index');

I can not confirm that on my local hhvm.

with hhvm 3.14.4 server from (deb http://dl.hhvm.com/ubuntu xenial main):

hhvm -m server -d hhvm.server.port=9000 -d hhvm.server.source_root=/home/xxx/public_html/demo362/

URLs:

• http://127.0.0.1:9000/administrator/index.php
• http://127.0.0.1:9000/

works.

I have got only:
"Warning: Division by zero in /home/tomash/public_html/demo362/plugins/system/debug/debug.php on line 589"

I'm just going by the conditions necessary to reproduce the error as found on Travis CI.

You can see the mock code at this link https://3v4l.org/HLGlZ

You can change $index to an array, null or to false and there is no error. But if $index is a string, the array modify opperator with throw the error.

The only conditions for $index to be a string in the cache MEMCACHE store() code is for the following line to return a string

$index = static::$_db->get($this->_hash . '-index');

It's possible that it's only returning a string under our PHP unit tests.

IMHO there is bug in travis tests, or php-memcache is too old.
I use this one 3.0.9 from:
https://launchpad.net/ubuntu/xenial/+source/php-memcache

Example which I do not understand:

if [[ $INSTALL_MEMCACHE == "yes" ]]; then phpenv config-add build/travis/phpenv/memcached.ini; fi

Why there is memcacheD extension loaded instead memcachE?

Please correct me if I'm wrong.

Why did not travis see that memcachE can not connect to server after applied PR #9622.
Human tests missing memcachE too.

This is telling Travis CI to adjust the PHP environment if the PHP version needs to make special settings for MEMCACHE

if [[ $INSTALL_MEMCACHE == "yes" ]]; then phpenv config-add build/travis/phpenv/memcached.ini; fi

It's specific to Travis and the container settings and only sets up MEMCACHE in the PHP environment. It's also not required for some PHP versions. In some versions both MEMCACHE and MEMCACHED are supported.

Currently both MEMCACHE and MEMCACHED are supported, even though MEMCACHE is outdated and will go away at some future date. (HHVM only supports MEMCACHE and PHP 7 only supports MEMCACHED)
I would agree there is a bug if the MEMCACHE connection wasn't being made and php unit Travis CI was still passing the unit tests.

Looking deeper into what is happening at this line.

$index = static::$_db->get($this->_hash . '-index');

Memcache::get returns a string unless it was passed an array of keys.

http://php.net/manual/en/memcache.get.php

This is why $index is a string when the array modifier opperator tries to append $tmparr to the value. Causing the error.

Currently both MEMCACHE and MEMCACHED are supported, even though MEMCACHE is outdated and will go away at some future date. (HHVM only supports MEMCACHE and PHP 7 only supports MEMCACHED)

I thought that too but when I checked yesterday MEMCACHE is still working on php7.0 - you can test it on ubuntu 16.04 LTS.

if [[ $INSTALL_MEMCACHE == "yes" ]]; then phpenv config-add build/travis/phpenv/memcached.ini; fi

IMHO this is a bug/typo, it should be:

if [[ $INSTALL_MEMCACHE == "yes" ]]; then phpenv config-add build/travis/phpenv/memcache.ini; fi

I created a PR for that #11576.

@mbabker I changed problematic stuff.
I also removed protected $_persistent = false; because it is not using now.
If it is not B/C then I can revert it, but it is useless.

I have removed ternary and rebased to the latest staging to resolve the conflict.
Summary of changes has been updated.

@csthomas Would you change line 190 in the store() method from from $index[] = $tmparr; to array_push($index, $tmparr); ? This change will solve the HHVM error Fatal error: [] operator not supported for strings in /libraries/joomla/cache/storage/memcache.php on line 190 that occurs when $index = static::$_db->get($this->_hash . '-index'); returns a string. (or otherwise deal with the case when Memcache::get returns a string so we don't have this fatal error hanging out there)

I've submitted a PR to your branch for this change.

Is the more issue there?

Can I ask for test.

Test should be simple: check whether cache works or not, test clear cache, etc.
For more advanced testers: code review.

Test should be simple: check whether cache works or not, test clear cache, etc.

Finding people with access to certain PHP configurations is challenging at times. I have MemcacheD set up on my local system but not Memcache and I don't have any servers running the latter to test this against.

If you have access to root account you could run: pecl install memcache-3.0.8 for php5.
After test you can display installed items by pecl list and uninstall by pecl uninstall memcache.

May be someone with HHVM can test, memcache extension is probably built in.

Example which I use for hhvm on kubuntu 16.04 LTS.

1. Install hhvm
   $ sudo apt-get install hhvm

2. Run www server on port 8000 and set source_root to joomla directory
   $ hhvm -m server -d hhvm.server.port=8000 -d hhvm.server.source_root=/home/[...]/public_html/demo362/

3. Go to http://localhost:8000/administrator/index.php and test:)

@brianteeman Can I ask you to change priority to higher? Currently cache handler MEMCACHE on J3.6.2 does not work at all.

There is no current need for users to test against HHVM for code acceptance, same with php7.1. these are allowed failures in the unit tests while we make necessary code changes to add support.

Code changes for HHVM and PHP 7.1 just need to check that they do not break existing supported PHP versions.

@photodude I know that but HHVM has memcache built in.

May be it will be easier for testers to test on hhvm (if I can assume: if pass on hhvm then should also pass on php).

If you have access to root account you could run

Everything I've got deployed is on PHP 7, Memcache isn't compatible with it. I'd have to spend some time fighting with Homebrew and my Mac's setup to get Memcache installed on an older PHP version.

Everything I've got deployed is on PHP 7, Memcache isn't compatible with it.

https://www.howtoforge.com/tutorial/install-apache-with-php-and-mysql-on-ubuntu-16-04-lamp/
http://www.ubuntuupdates.org/package/core/xenial/universe/base/php-memcache
https://github.com/websupport-sk/pecl-memcache

I know the problem may be with compile new version.
For now I only know easy way for Ubuntu 16.04 LTS.
sudo apt-get install php-memcache which works with php7.

That's the first I've seen that someone's even working on the memcache extension, let alone that there's supposedly a PHP 7 compatible build. Good to know.

May be it will be easier for testers to test on hhvm (if I can assume: if pass on hhvm then should also pass on php).

Few have HHVM set ups, and I doubt the CMS would currently run on HHVM without issues (we still have a lot of unit test failures on HHVM), additionally HHVM does have deviations in some behaviors from PHP and there are php extensions that HHVM does not currently support, while other extensions do not replicate all functions. As such I wouldn't directly assume that if HHVM passes that PHP will too.

static::$_db->get(...) .i.e Memcache::get It could have a string type if it was passed a string key, or it could have an array type if an array of keys was passed to the method, or it could return the exact value boolean false on failure.

Why do you think that

$index = static::$_db->get($this->_hash . '-index');

may contains string (which does not come from previous error)
$index as string come from memcacheD storage which use other format of serialize.
Then memcache internal unserialize may fail and return string like '{a:0:{}}'

The only way to get $indexas string is error come from internal memcache unserialize.

$index usually look like:

Array
(
    [0] => stdClass Object
        (
            [name] => d41d8cd98f00b204e9800998ecf8427e-cache-_testing-aa178765666f875a86e12a7590eb296e
            [size] => 8
        )

    [1] => stdClass Object
        (
            [name] => d41d8cd98f00b204e9800998ecf8427e-cache-_testing-1167a6c94988dd05be7ccc5faed80fa6
            [size] => 8
        )

)

If we allow to create structure like:

Array
(
    [0] => {a:0:{}}

    [1] => stdClass Object
        (
            [name] => d41d8cd98f00b204e9800998ecf8427e-cache-_testing-aa178765666f875a86e12a7590eb296e
            [size] => 8
        )

    [2] => stdClass Object
        (
            [name] => d41d8cd98f00b204e9800998ecf8427e-cache-_testing-1167a6c94988dd05be7ccc5faed80fa6
            [size] => 8
        )

)

then we will get errors in other methods like in '$memcache->getAll()'

static::$_db->get(...) .i.e Memcache::get It could have a string type if it was passed a string key, or it could have an array type if an array of keys was passed to the method, or it could return the exact value boolean false on failure.

I read some similar text somewhere but it means ($m - instance of memcache):

$m->set('a', 'a1');
$m->set('b', 'b1');
$m->get(array('a', 'b')); // this return array
$m->get('b'); // this return string

but

$m->set('x', array('a', 'b'));
$m->get('x'); // this return array

Why do you think that

$index = static::$_db->get($this->_hash . '-index');

may contains string (which does not come from previous error)

As I have stated, it's by definition of the Memcache::get() method. Check the function return values listed in the manual

Returns the string associated with the key or an array of found key-value pairs when key is an array. Returns FALSE on failure, key is not found or key is an empty array.

memcacheD::get has a different return value from memcache::get. Since we are specifically dealing with memcache, it is the memcache definition and return values that have to be considered. (easy to get tripped up between the two closely related extensions)

$m->set('x', array('a', 'b'));
$m->get('x'); // this return array

It's probably not unexpected to see an array in this case, it's probably one of those undocumented things where the "string" associated with the string key is actually an array so the return is an array.

try something like

$m->set('x', 'a');
$m->get('x'); // this should return a string

The magic sentence from http://php.net/manual/en/memcache.get.php is:

Memcache::get() returns previously stored data if an item with such key exists on the server at this moment.

Which indicate that set trigger serialize if data is not string and then get trigger unserialize.

But:
http://php.net/manual/en/memcache.set.php

Note:
Remember that resource variables (i.e. file and connection descriptors) cannot be stored in the cache, because they cannot be adequately represented in serialized state.

Tutorial for testers who does not have access to php memcache extension:)

1. Create demo on https://demo.joomla.org/
   You have to create demo with email/password. Do not create demo with "Lanch Test Drive" because you may not have access to cpanel.

2. After that you should be at https://ua.siteground.com/my_account.htm

3. Click on button "Access cpanel"

4. On cpanel on the felt side is button "Flush cache". Click on it.

5. Then click on "Level 3: Memcached" and enable Memcached server "On"

6. Take a look at details of memcached server, PORT, remeber it.

7. Back to your new joomla site and go to backend.

8. login to administrator area (subdomain, password)

9. Go to System->System Information->Php Information and serach for text "memcache support".
   If it is enabled go next.

10. Go to System->Global Configuration->System. Now you can not select a memcache. It is missing.

11. Install https://github.com/joomla-extensions/patchtester/releases/download/3.0.0-alpha2/com_patchtester.zip

12. Go to Components->Joomla Patch Tester

13. Click on fetch data and wait a while

14. Search for this PR, means type 11565

15. Apply that patch

16. Go to System->Global Configuration->System and enable memcache, set PORT as you saw before in cpanel and Save.

17. Now you are on new memcache handler.

Warning: If you miss something (set bad PORT or not turn on memcached server) then you may have problem because joomla do not allow you to change any configuration after connection to memcache fails.
Then you have to back to cpanel, open file manager and turn off cache in configuration.php file manually.

@csthomas I think there maybe a documentation error in the php Memcache::get() manual

Based on your example testing, I think it should say something like the following

Memcache::get returns previously stored data of an item if such key exists on the server at this moment
Return Values
Returns the value associated with the found key, or an array of found key-value pairs when key is an array. Returns FALSE on failure, key is not found or key is an empty array.

in other words

so memcache::get might return a string for a single key if the key value was a string.

So in our code: we need to know what the value of $this->_hash . '-index' is in the unit test when that key gets passed to Memcache::get(...). If $this->_hash . '-index' exists and it's a single value which is a string, then $index will be a string when we try to use the Array modifier [] syntax at line 190

First I want to ask for testing.

About documentation of php memcache: there is a little mess, but this PR is not for that.

Ok, I explain a little - it will be helpful to understand my next PR #10565:
memcache/memcahed for get/set should work similar (return type).

$m->get('a') => return false # because it does not exist or was set to false
$m->set('a', array()) => $m->get('a')     => return array();
$m->set('a', (object) array()) => $m->get('a')   => return (object) array();
$m->set('a', array((object) array())) => $m->get('a') => return array((object) array());
$m->set('a', 123) => $m->get('a')       => return 123;
$m->set('a', false) => $m->get('a')     => return false;

// but:

$m->delete('a');
$flags = 0;
$m->get('a', $flags) => return false; // and if flags === 0 this means that value was not found
$m->set('a', false);
$flags = 0;
$m->get('a', $flags) => return false; // and if flags !== 0 this means that value was set as false

Memcache/(-d) use serialize for all types which is not string.
If something can not be serialized then throw error.

This performance problem is big $index with contains list of objects.
Imagine 1000 items in cache.
This is 1000 objects in array which have to be unserialize/serialize on any memcache get index and set index.

There was 1 error:

1) JCacheStorageMemcachedTest::testCacheHit

Memcached::get(): could not read long value, too big

/home/travis/build/joomla/joomla-cms/tests/unit/core/helper/helper.php:52

/home/travis/build/joomla/joomla-cms/libraries/joomla/cache/storage/memcached.php:187

/home/travis/build/joomla/joomla-cms/tests/unit/core/case/cache.php:91

This means that memcache left some data which memcached interpreted as big int.
More info: https://secure.php.net/manual/en/book.memcache.php#115666

So to help clean up that stuff let's fix the TestCaseCache::teardown() method to completely clean the cache storage instead of just cleaning a single group. For this, I'd say add a new method to JCacheStorage, a clear() method (consistent with PSR-6) that just deletes all the things.

clear and clean. Is not too much mess?
May be flush method?

I only suggest clear since that's what the caching PSR uses. flush works too.

Clearing and cleaning post test is exactly what I tried to implement broadly in my pr for unit test memory management. #10685

All you're doing there is unsetting the object. There isn't a destructor in the classes that's flushing the cache so all it's doing is helping with the PHP process' memory use, not actually clearing the cache storage.

If during our unit tests Memcached::get(): could not read long value, too big, resulting in left over data being in the cache, are we sure that a similar situation could not happen in a production environment?

Sure we can, and should, fix the unit tests; but IMO we should take reasonable steps to ensure that the same errors won't occur in production.

That's in part why the user has a way to clear the cache via the admin UI or if they're able to run CLI scripts we ship one to clear expired cache (could probably add another to just clear the cache completely). We can't account for every possible error condition, i.e. when data fails to serialize and unserialize out of the cache store correctly, so some things will require manual intervention no matter what. I think that scenario hits the manual intervention criteria.

What do you thing about the last two commits?
Could I add the same changes (from memcache) to memcached in this PR?

Travis passed:)

If it works, go for it. I honestly don't know the systems well enough, I can just create very high level API interactions.

I'm curious, may be I should use clear method instead 'flush' and use it as a replacement for clean in J4.
Then set clean as deprecated and create clear:

public function clear($group = null)

then clear won't be full B/C because there won't be mode="notgroup".
I have not seen any use of $mode='notgroup' except tests.

@mbabker I'm referring to the change at line 181. I still don't think that solves the underlying issue (cache has left over junk) and possibly introduces new issues. The whole cache clearing thing in the unit tests are revealed in the issue caused by line 190 in hhvm

IMHO The best solution for cache storages would be use the same design as in database tests.
Means move create storage from setUp to setUpBeforeClass and then $storage->flush() on tearDownAfterClass() method.

Take a look at https://github.com/joomla/joomla-cms/blob/staging/tests/unit/core/case/database.php

@csthomas I have no disagreement with the proposed changes for the Unit Test cache storages. I think it's a good choice to use the same design as the database tests.

My only disagreement is with the change from a255c63 as noted in the line comment.

@photodude We have:

$index = static::$_db->get($this->_hash . '-index');

then $this->_hash . '-index' is always string, right?

Now $index can be:

• false, means not exists
• array, means exists and it is OK, but (may contains invalid data in items)
• string, means that this is invalid data because we always use array value for key $this->_hash . '-index'.

Now if we get string, means invalid data, what then?

You suggest to change it to array and push string at the first place in array.
This off course trigger warning in array_push but who cares.

Most important things is that $index is a list of objects, but now we will have string at key=0.
This will generate errors on getAll or clean method.

I think that is enough for that PR.
Memcache and Memcached has been changed.

Travis tested memcache and memcached and passed.

Please test at least one storage.

If your confident that when MEMCACHE::get is a string, it is always invalid data and can be ignored in this case; then I'm fine with that change.

Yes but only for that key=$this->_hash . '-index'.
For other keys it may be string as valid data.

I have tested this item ✅ successfully on 3aa94f5

Works, with this patch Memcache is now a selectable option for cache. Where before memcache only worked for the session handler. everything functions correctly after enabling memcache in global settings

Thanks @yvesh for code review.
Thanks @photodude for your test.

After my commit I probably still need 2 success tests.

Maybe a stupid question but we have Memcached running on our servers and on our Joomla 3.6.2 websites without any problems (PHP 5.6.24 with cloudlinux) So what do we need to fix which works for us?

Maybe a stupid question but we have Memcached

This PR only add improvement for memcacheD

But you can not use MemcachE on J3.6.2

If memcache handler does not work on J3.6.2 and nobody complain about it (except me) then could testers test at least memcached handler.

There are the same changes in both, except getConnection(), which has a few more differences.

@photodude @mbabker Can I ask you to to make a test?

I have tested Memcached ✅ on PHP 5.6.24/Cloudlinux/Mariadb/cgi-fcgi on Joomla 3.6.2 successfully This runs well on a Joomla 3.6.2 multi lingual testsite gwsdev2.work. I also tested this on the same site with PHP7 enabled and works fine as well

Short remark on Memcache. Memcache is not available yet for PHP7. This is the reason why we have removed it from the PHP-extensions in the PHP-selector of Cloudlinux altogether and for now just provide Memcached. This is not any issue since in the CL environment you get OPCache and that works just so nice and fast in combination with Memcached

Thank you @gwsdesk.

Short remark on Memcache. Memcache is not available yet for PHP7

Official not, but at least one linux distribution have memcache working with PHP7.

If you have Ubuntu 16.04 on some machine then you can install php-memcache (PHP7) from default repository. There is probably unofficial version of memcache compiled in deb.

#11565 (comment)

Thanks Thomasz but Cloudlinux does not support Memcache in PHP7 yet. In
Centos we can also install Memcache from default and that works I can
confirm. CL no go at present though they have a Beta version

On 8/25/2016 3:33 PM, Tomasz Narloch wrote:

Thank you @gwsdesk https://github.com/gwsdesk.

Short remark on Memcache. Memcache is not available yet for PHP7

Official not, but at least one linux distribution have memcache
working with PHP7.

If you have Ubuntu 16.04 on some machine then you can install
php-memcache (PHP7) from default repository. There is probably
unofficial version of memcache compiled in deb.

#11565 (comment)
#11565 (comment)

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#11565 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHzLNkC-weu902hW0b08oA4bdnuGK06zks5qjVNfgaJpZM4JiqOc.

Can your test can be mark as successful at https://issues.joomla.org/tracker/joomla-cms/11565

If you have a few minutes You can test memcache as follow:
#11565 (comment)

I have tested this item ✅ successfully on 4e5e96b

I have tested Memcached on PHP 5.6.24/Cloudlinux/Mariadb/cgi-fcgi on Joomla 3.6.2 successfully ✅ .This runs well on a Joomla 3.6.2 multi lingual testsite gwsdev2.work. I also tested this on the same site with PHP7 enabled and works fine as well

I have tested this item ✅ successfully on 4e5e96b


Marking successful, with the following note.

As soon as I applied this revised patch I got the following notice

Warning: Error loading component: com_languages, Could not connect to memcache server

I assume this warning was due to an old cache state; as it went away after turning cache off and then back on in Global settings. Since it was a transient message and resolved after resetting the cache I don't think there is anything to do but note the message.

@photodude this means that it is not successful since it shows the original error (which goes away if you disable caching or change to file. Please remove the 'successful' tag and replace with 'unsuccessful' ? ❌
This is not just a 'warning' is means it does not work!
I retested and cannot replicate that warning

I don not know @photodude steps but I assume he enabled memcachE before applying patch.
Warning was stored in session and displayed at next page view.

Besides above.
There was not any changes added to memcachE handler from the first @photodude test.
The last 2 commits only added changes to memcacheD handler.

Here is what I had done. @gwsdesk
1. tested the patch in its old state using patch tester (see first test)
2. Removed the patch
3. Applied the patch in its current state
4. Warning appeared at this point
5. Turned off cache in global settings (forced logout)
6. (Logged in) turned cache back on
7. No warnings or issues using joomla front end or back end.

As you can see the warning was due to the inconsistent state the cache was in between reapplying this patch after it was updated. I stand by my choice in marking it as a successful test.

@photodude I read it correctly now and only noticed now your initial test as well. Thanks for successful testing ;-)

@rdeutz can this be marked RTC now?

@photodude yes it can

@joomla-cms-bot can you please help here? Thanks ?