Conflicting Files ? ? Failure

User tests: Successful: Unsuccessful:

avatar andrepereiradasilva
andrepereiradasilva
14 Nov 2015

Description

After adding some options added for URI exclusions by @OctavianC in the cache plugin (#7767), this PR adds some other improvements:

  • Checks SEF and non SEF URI
  • Since it also checks the non sef URI now you can exclude all pages of a
    component (ex: com_users)
  • No need to backslash the slashes
  • Do not cache or load from cache if site is offline
  • Some other minor code improvements
  • Show debugger even in cached pages

How to test

With a fresh Joomla install with sample items, turn on cache plugin and exclude some SEF or Non-SEF page URI in "Exclude Pages" field, for instance:

  • com_contact
  • option=com_content
  • /login
  • /article-categories-view
  1. Check if pages is being cached or not.
  2. Put the site offline and see that it does not generates or load from cache.

B/C

Should be ok.

Notes

Since i'm not expert in Joomla code, if there are betters ways to do this, improvements are welcome.

avatar andrepereiradasilva andrepereiradasilva - open - 14 Nov 2015
avatar andrepereiradasilva andrepereiradasilva - change - 14 Nov 2015
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 14 Nov 2015
Labels Added: ? ?
avatar andrepereiradasilva
andrepereiradasilva - comment - 14 Nov 2015

First time travis free in one go!

avatar andrepereiradasilva
andrepereiradasilva - comment - 14 Nov 2015

@brianteeman Thanks. Done

avatar OctavianC
OctavianC - comment - 16 Nov 2015

I haven't tested the code, but there are two things I've noticed:

  • I'm not really fond of the % regex delimiter. PHP examples use the forward slash and if you search on the web, almost everyone uses it as well.
  • This expression %<input type="hidden" name="[A-Za-z0-9]+" value="1"\s+/>% will incorrectly replace these as well:

<input type="hidden" name="agree" value="1" />

<input type="hidden" name="id" value="1" />

<input type="hidden" name="catid" value="1" />

Basically all hidden fields that contain numbers and characters (that's a big range) will get replaced by the plugin and break functionality for the respective extensions.

avatar andrepereiradasilva
andrepereiradasilva - comment - 16 Nov 2015

This expression %<input type="hidden" name="[A-Za-z0-9]+" value="1"\s+/>% will incorrectly replace these as well: [...] Basically all hidden fields that contain numbers and characters (that's a big range) will get replaced by the plugin and break functionality for the respective extensions.

The problem is when you cache a page with a form token, that form token will be cached too. So it will be served to all users. This code was to try to correct that.
And yes, you are right, my mistake.
Any suggestions how to do this without breaking functionality?

I'm not really fond of the % regex delimiter. PHP examples use the forward slash and if you search on the web, almost everyone uses it as well.

In PHP preg expressions you can use any character as delimiter. Since URI use a lot of slashes i thing is more suitable and more user friendly for this case using another character, that is why i used % instead of / as regex delimiter.

A delimiter can be any non-alphanumeric, non-backslash, non-whitespace character.

See http://php.net/regexp.reference.delimiters

avatar OctavianC
OctavianC - comment - 16 Nov 2015

I'm not arguing about the purpose of the expression, I understand what it's trying to do, I was just saying that it doesn't work well.

avatar andrepereiradasilva
andrepereiradasilva - comment - 16 Nov 2015

Ok. I will see if i can get some way to replace the token without breaking something.
If not i will remove it from the code.

Update:

From the code i discover the token is a 32 md5 hash character generated by the "md5" PHP function.

https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/session/session.php#L268-L274

https://github.com/joomla/joomla-cms/blob/staging/libraries/cms/application/helper.php#L67-L70

Returns the hash as a 32-character hexadecimal number.
http://php.net/manual/en/function.md5.php

So i can fix the regex to find only a a-f0-9 string with 32 characters ([a-f0-9]{32}).
That way none of your examples will match.

avatar OctavianC
OctavianC - comment - 16 Nov 2015

Looking at the code it seems JCache::getWorkarounds() already does what you're looking for, although I'm not sure where it's used. See 10b1f57

avatar andrepereiradasilva
andrepereiradasilva - comment - 16 Nov 2015

Looking at the code it seems JCache::getWorkarounds() already does what you're looking for, although I'm not sure where it's used. See 10b1f57

I didn't saw that. Will test.

Update:
Ok. I tested in an earlier version that didn't do that replacement. That is already being done in Joomla.
So i will remove that from this commit code when i have time.
Thanks @OctavianC

avatar andrepereiradasilva
andrepereiradasilva - comment - 16 Nov 2015

Ok. done and some minor otimizations

avatar andrepereiradasilva
andrepereiradasilva - comment - 22 Nov 2015

@OctavianC can you check?

avatar zero-24 zero-24 - change - 25 Nov 2015
Category Plugins
avatar zero-24 zero-24 - change - 25 Nov 2015
Labels
avatar OctavianC
OctavianC - comment - 8 Jan 2016

Sorry for the long vacation :) Will test and report.

avatar OctavianC
OctavianC - comment - 11 Jan 2016

Took a look at the code and I'm not really fond of the part where the SEF URL and the non-SEF URL are concatenated:

if (preg_match('/' . preg_quote($exclusion, '/') . '/i', $this->_cache_key . ' ' . $internal_uri, $match))

There must be a cleaner way to do that. Why is it preg_quoted()? This means that regular expressions won't work, which is the point of having that feature.

Also, if matching components is a feature, why don't we add a list of all components installed and have the user select them instead of typing them in?

avatar andrepereiradasilva
andrepereiradasilva - comment - 11 Jan 2016

Took a look at the code and I'm not really fond of the part where the SEF URL and the non-SEF URL are concatenated: There must be a cleaner way to do that.

This is for performance. Why do two preg_match operations when we can do only one?

Why is it preg_quoted()? This means that regular expressions won't work, which is the point of having that feature.

I don't like it too. i will change.

Also, if matching components is a feature, why don't we add a list of all components installed and have the user select them instead of typing them in?

Yes a field for that could also be added.

avatar andrepereiradasilva
andrepereiradasilva - comment - 11 Jan 2016

Updated the code. Made some code improvements, comments and a small change to show the debug even if read from cache.

Didn't make the components select field part. Will not do it in this PR.

Please test.

11ebaaf 11 Jan 2016 avatar andrepereiradasilva ups
avatar brianteeman brianteeman - change - 27 Apr 2016
Labels
avatar brianteeman brianteeman - change - 27 Apr 2016
Category Plugins Language & Strings Plugins
avatar conconnl
conconnl - comment - 27 Jun 2016

Can you extend your test scenario with the information how people can see if things are being cached.
Then we can let some people test this PR, improvements within the cache feature are very welcome


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/8431.

avatar andrepereiradasilva
andrepereiradasilva - comment - 5 Sep 2016

Can you extend your test scenario with the information how people can see if things are being cached.
Then we can let some people test this PR, improvements within the cache feature are very welcome

you can look at the generated files in the /cache directory to see if the page cache is generated or not

avatar euismod2336 euismod2336 - test_item - 4 Nov 2016 - Tested successfully
avatar euismod2336
euismod2336 - comment - 4 Nov 2016

I have tested this item successfully on 9f0369a

Items added through the menu-item or via the advanced exclude field are not cached while others are. While site is offline no items are cached.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/8431.

avatar joomla-cms-bot joomla-cms-bot - change - 2 Jan 2017
Category Plugins Language & Strings Administration Language & Strings Front End Plugins
avatar SamuelSchepp SamuelSchepp - test_item - 21 Aug 2017 - Tested successfully
avatar SamuelSchepp
SamuelSchepp - comment - 21 Aug 2017

I have tested this item successfully on ad59b79

Tested:

  1. Added few URIs and component names in "Exclude URLs"
    1.1 /login
    1.2 index.php
    1.3 sample-site
  2. Visited these pages
  3. Edited these and other cached pages
  4. Visited these pages again
    Pages got cached as expacted.

Site offline testing:
Tested:

  1. Set site to online
  2. Enable cache plugin
  3. View site
  4. Set site to offline
  5. Reload site

Without patch:
Cached version gets loaded.

@icampus

With patch:
Site is loaded as offline.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/8431.

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 21 Aug 2017

@euismod2336 can you please retest?

avatar eXsiLe95 eXsiLe95 - test_item - 22 Aug 2017 - Tested successfully
avatar eXsiLe95
eXsiLe95 - comment - 22 Aug 2017

I have tested this item successfully on ad59b79

Tested:

  1. Added URIs and components in "Advanced/Exclude URLS"
  2. Visited this pages, made changes
  3. Changed site state to offline
  4. Visited the pages again
  5. Changed site state to online
  6. Installed bugfix
  7. Visited the pages again
  8. Changed site state to offline again
  9. Visited the pages again

Behaviour

Before bugfix

After the site was turned offline, the sites were still cached.

After bugfix

After the site was turned offline, the site showed the preset offline message.

Tested @icampus


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/8431.

avatar franz-wohlkoenig franz-wohlkoenig - change - 22 Aug 2017
The description was changed
Status Pending Ready to Commit
avatar joomla-cms-bot joomla-cms-bot - edited - 22 Aug 2017
avatar franz-wohlkoenig
franz-wohlkoenig - comment - 22 Aug 2017

RTC after two successful tests.

avatar brianteeman
brianteeman - comment - 22 Aug 2017

This has merge conflicts so cant be merged as is. Not sure if @andrepereiradasilva is around to fix them

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 30 Oct 2017

@andrepereiradasilva can you please resolve conflicting Files?

avatar mbabker
mbabker - comment - 7 Jul 2018

Replaced by #21012 with merge conflicts resolved.

avatar mbabker mbabker - change - 7 Jul 2018
Status Ready to Commit Closed
Closed_Date 0000-00-00 00:00:00 2018-07-07 16:59:00
Closed_By mbabker
Labels Added: Conflicting Files ?
avatar mbabker mbabker - close - 7 Jul 2018

Add a Comment

Login with GitHub to post a comment