?
Referenced as Pull Request for: # 9035
avatar durian808
durian808
28 Jan 2016

Please re-open #8757 and #8731 (combine them).

See my comment at the bottom of #8731.

thanks.

Votes

# of Users Experiencing Issue
1/1
Average Importance Score
5.00

avatar durian808 durian808 - open - 28 Jan 2016
avatar durian808 durian808 - change - 28 Jan 2016
Title
REOPEN - "You are not permitted to use that link to directly access that page..."
REOPEN - You are not permitted to use that link to directly access that page...
avatar durian808 durian808 - change - 28 Jan 2016
Title
REOPEN - "You are not permitted to use that link to directly access that page..."
REOPEN - You are not permitted to use that link to directly access that page...
avatar mbabker mbabker - change - 29 Jan 2016
Priority Critical Medium
avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

There is nothing to fix here. There is no existing problem in Joomla 3.4.8 after you have logged out and cleared your session cookie and logged in again after the upgrade.

If you want to report a problem, you need to provide exact steps to replicate each time - simply saying there is an issue is not enough for developers to waste time on, developers are not also going to waste time having to refer to four different, years old, threads to pick out the bits of comments that may or may not still relate to code released in December 2015!

A LOT of time has been spend over Christmas by the excellent @tampe125 on session issues and as I believe at the moment there are no known issues remaining.

avatar durian808
durian808 - comment - 29 Jan 2016

Yes I understand. I will try to reproduce on Joomla 3.4.8.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar brianteeman
brianteeman - comment - 29 Jan 2016

@durian808 even better would be to reproduce on 3.5beta2


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 29 Jan 2016

I'll try it from a SiteGround demo site first.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

In the meantime I recommend this is closed until we have a replicable issue.

avatar durian808
durian808 - comment - 29 Jan 2016

As long as the opener can re-open it, right?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

There is nothing in this GH Issue of revalue to reopen it - just open a new one with steps to replicate following the normal pattern of reporting issues

https://docs.joomla.org/Filing_bugs_and_issues

avatar durian808
durian808 - comment - 29 Jan 2016

Please leave it open. I just reproduced the problem on the SiteGround demo.
screen shot 2016-01-29 at 04 17 13


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

I cannot debug from a screenshot of a page!

avatar durian808
durian808 - comment - 29 Jan 2016

Duh! Here are the steps:

Go To
http://durian808.demojoomla.com

Click LOGIN
username: durian808
password: demo123+

Click on BLOG

In a loop until the error occurs... proceed at a fairly good pace w/ no pause:
1) Click on the edit icon
2) Add a few characters to the top line of the content, "123"
3) Click Save
4) immediately go to #1


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

I believe what is happening is this....

So what you are describing is a race condition occurring with session write/close and the speed at which you are working - not something that is a Joomla problem or anything that should/could be fixed.

You fail completely to understand how a PHP session works, how its opened, written to, closed, and stored.

What is happening is that you are reloading a page, before your last page action has completely closed the session, so when you open the next page the session data you get is the one before the one that is currently saving.

If you slow down then there is no problem,

Furthermore, on my very fast internet connection I was unable to replicate this issue following the steps you provided.

avatar durian808
durian808 - comment - 29 Jan 2016

No, no. I'm not doing it THAT fast. Just proceed at a normal pace. Don't wait around.

If you already believe there is not a problem, that may prevent you from reproducing it.

It would be best if this test was tried by a few different people.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 29 Jan 2016

When this very same error is seen in the field, it's not because of a looped edit test like this, it's just somebody going to click the edit icon, normally, and they get the error. That's my feeling about it.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

Belief is not enough to make something work. If only life was that easy.

I have tried now fast and slow and I cannot replicate the issue you describe - but I can think it through and the session race condition is an acceptable description of the reason behind this.

Furthermore YOU don't have to be fast/slow for the race condition to exist, the SCRIPT has to not have finalised and stored the session which is the deciding factor.

I have not - not even once - been able to replicate with the details you have provided.

avatar brianteeman
brianteeman - comment - 29 Jan 2016

I tested multiple times as can be seen by the edits.

On 29 January 2016 at 10:40, durian808 notifications@github.com wrote:

When this very same error is seen in the field, it's not because of a
looped edit test like this, it's just somebody going to click the edit

icon, normally, and they get the error. That's my feeling about it.

This comment was created with the J!Tracker Application
https://github.com/joomla/jissues at issues.joomla.org/joomla-cms/9013
https://issues.joomla.org/tracker/joomla-cms/9013.


Reply to this email directly or view it on GitHub
#9013 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

and? could you replicate it @brianteeman ?

avatar brianteeman
brianteeman - comment - 29 Jan 2016

no - or I would have said.

Your explanation does make sense to me.

I have seen it myself in the admin when trying to open multiple articles
for editing into their own tabs. I have never seen it when working in just
one tab

On 29 January 2016 at 10:41, Phil Taylor notifications@github.com wrote:

and? could you replicate it @brianteeman https://github.com/brianteeman
?


Reply to this email directly or view it on GitHub
#9013 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar durian808
durian808 - comment - 29 Jan 2016

Ok, I will do a bit more testing.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016
avatar durian808
durian808 - comment - 29 Jan 2016

@PhilETaylor, IF indeed you are saying that this is only a race condition... In the real world of everyday people editing articles in the front end of Joomla (non-technical people), on Internet connections of various speeds, if they see the browser window finish loading and they are looking again at an article with an edit icon next to it, you are saying they are expected to know they can't click that edit icon until X seconds elapse? And your answer to this is that it's OK for Joomla to say, "You are not permitted to use that link to directly access that page" + 403 Error. From a UI perspective, that seems ridiculous to me. And further, you are saying there is no programmatic solution?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

If this is a PHP session race condition then this is not a Joomla problem. Do your research. The web is full of people discussing this - for 10 years.

There are programmatic ways of fixing this, but solutions can only be designed for scenarios that can be replicated and proved over and over. Joomla already has much code in a good design way, to close the session and handle redirects correctly

And yes, if you are trying to access a page when you don’t have sufficient rights to (because the id is not in the session) then yes it is perfectly acceptable to get a HTTP Error code 403 and a message to that effect.

I have been unable to replicate this issue on any site I have, or with the demo site you provided, with and without internet speed throttling, with the default PHP session handler in Joomla

Im guessing that your SiteGround site might be using memcached as a session storage, I have emailed their CTO to ask, - which may be slower in their infrastructure (or might not be) but might be slower than standard files based storage - and Im trying to set up a test of other session storage providers as a test of this theory.

Just because your browser has finished loading - doesn't mean the server has finished processing the PHP thread to destruction or closed the session!

I don’t expect you, and non-technical people to understand PHP internals, but I do expect that you respect those of us that do when we use that experience to make an educated guess of a problem we are unable to replicate.

avatar durian808
durian808 - comment - 29 Jan 2016

"There are programmatic ways of fixing this, but solutions can only be designed for scenarios that can be replicated and proved over and over. Joomla already has much code in a good design way, to close the session and handle redirects correctly"

Good, good. And that's why people like me who have been working with Joomla users for 10 years try to help discover why these real world issues are cropping up and frustrating Joomla users, regardless of where the root cause lies. We want a good UI, not excuses. And you see, as developers and people who work with actual business clients, we need respect, too.

I actually have huge respect for the developers of Joomla.

"And yes, if you are trying to access a page when you don’t have sufficient rights to (because the id is not in the session) then yes it is perfectly acceptable to get a HTTP Error code 403 and a message to that effect."

Sorry, sir. That simply does not fly with real-world end users. It is not acceptable for a CMS with the reputation and quality of Joomla. So that is why we are looking for a solution to this.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.
avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

Until someone can replicate this issue and provide repeatable steps to replicate I recommend it is closed - there is nothing here to see.

avatar durian808
durian808 - comment - 29 Jan 2016

Not so fast there please. I am testing. What's you're rush?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

Well once you have something more than words... let me know.

avatar durian808
durian808 - comment - 29 Jan 2016

"Just because your browser has finished loading - doesn't mean the server has finished processing the PHP thread to destruction or closed the session!"

How long then, typically/roughly, would it take in the case of an article like this with a few bytes of data in it, once the user clicks the "Save" button, for the server to get the request, for Joomla to process it to completion? How long would you guess, based on an Internet connection speed of... whatever, standard home broadband connection?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 29 Jan 2016

If my theory is correct about this being a session issue ..

You simply dont understand then that this is not an internet connection, browser, broadband issue at all. This is a server side code processing issue!! It should take milliseconds to close the session - but as the comments in the PHP.net thread and endless forum threads will show - this doesnt always happen.

Joomla sessions do not contain "a few bytes of data" they contain a lot more, and on sites with lots of extensions messing with the session they can grow quite bit. Furthermore since Joomla 3.4.7 they are also now base64 encoded/decoded which adds additional processing time.

This is not related to the browser/connection at all.

When you get the 403 page - what is the exact URL that you are on, that is not shown in your screenshot!?

avatar durian808
durian808 - comment - 29 Jan 2016

"If my theory is correct...this is not an internet connection, browser, broadband issue at all."

OK, good to know. But you said...

"I have been unable to replicate this issue on any site I have, or with the demo site you provided..."

How come then I could easily produce this condition on this demo site on SiteGround, but you couldn't on the same site? (if it was not related to internet connection, etc.) I mean, I created those articles and then just went into the front-end and edited the top article maybe 4 times and the error came up, just like that. I haven't been able to reproduce it again now no matter how quickly I do the test.

"It should take milliseconds to close the session - but as the comments in the PHP.net thread and endless forum threads will show - this doesnt always happen."

OK, fine. So what could be the reason, typically, that milliseconds would be lengthened to, say, a few seconds? What would be the chances of that happening, for example on this SiteGround server? Also, I have just measured the time from when I click the Save button to when the browser screen is refreshed (Article successfully saved) ... it's about 4 sec.

"When you get the 403 page - what is the exact URL that you are on, that is not shown in your screenshot!?"

I'll let you know next time I can precipitate the error. I will do some more testing tomorrow.

Thanks for your input on this.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar fruppel
fruppel - comment - 29 Jan 2016

I tried it with your test instructions and I also can't reproduce it


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 30 Jan 2016

First I'd like to say that this potential issue under investigation is probably unlike a lot of issues that are dealt with here. It is very difficult to reproduce, yet has been experienced and reported by many over the years since at least 2012. As such, I don't think it should be treated as one would usually treat most potential issues in Joomla. I think we should be open to all information that has been collected about the issue. I understand @PhilETaylor's concern to not waste time/effort on non-issues; however, I think there is a good chance that what we are observing here is, in fact, a real issue. That is of course my perspective, but it is a view based on my observation of the issue over the span of many years. Sometimes with issues of this nature, where reproducing the problem may prove to be too difficult, what is required is an approach of looking at symptoms and then theorizing root cause, followed by logical analysis of how the root cause could arise, and then inspection of related code. For some issues, the approach of "we're not going to look at this because we can't reproduce it" falls down.

I think I also need to emphasize that, although the issue can be precipitated by a repeated test as discussed above in this tracker item, the issue has also been seen in the field under normal-use conditions, confounding many a Joomla end user with a message that is essentially meaningless to most. There is only one place in the Joomla core that a 403 error can be raised along with the message "You are not permitted to use that link to directly access that page...", and that's in components/com_content/controller.php. There are also 14 more instances inside the administrator components where this same message can be displayed, however not along with a 403 error. All cases relate to editing a resource, and all use essentially the same code and rely on the function, $this->checkEditId().

Theory:

I would now like to outline a theory as to why this 403 error is produced when attempting to edit an article in the front end, based on my observations during yesterday's testing and also from recent server log data that shows the same problem in the field. Here is a summary of the theory:

  1. Under certain conditions which are difficult to reproduce, the data kept by Joomla associated with the login session and/or editing session becomes corrupted. (Since I don't know the specific details of the underlying mechanisms in the software here, I will just refer to this as "session data".)

  2. Once corrupted, a subsequent click of the/an article edit icon will result in Joomla reporting to the user, "You are not permitted to use that link to directly access that page...". (Most users probably don't notice that this is also a 403 error, but this fact is a key to noticing occurrences of this in server logs.)

  3. At the point where the 403 error is produced by components/com_content/controller.php, this is already well past the point where the root cause of the issue corrupted the session data.

  4. I have observed this issue recently in both Joomla 2.5.x in the field and Joomla 3.4.8 during testing on a fresh install demo site at SiteGround. The code that produces the 403 error in components/com_content/controller.php is the same in both Joomla versions noted here.

Analysis:

If this theory is correct, then the logical place to look for root cause is in the code that locks the resource (article) for the operation (editing). If the resource is not locked properly, either by a design flaw or by a bug/defect, then one possible outcome is that the same user session will be able to open the resource more than once, at the same time, for the operation. My test results yesterday actually showed probable evidence of this. As I was performing the test, and before the 403 error was produced, I made an edit to the article, saved it, and then re-opened the article to find that the previous edits were absent. In other words, it was as if the new editing session had loaded the previous copy of the content, and as if the previous save request had not completed**. Whatever flaw in the code allowed this to happen, I think this is the point where the session data was corrupted. Clearly this is a flaw, because the resource should be locked right up to the point where the save operation comes to full completion. Mutual exclusion must be maintained for access to the resource – evidence seems to indicate it was not. @PhilETaylor quickly suspected that a race condition was happening, however this is perhaps a different race than he imagined. Again, this only happens under certain hard-to-reproduce conditions. (**Or perhaps the previous save operation failed without reporting a problem.)

@PhilETaylor noted, "...YOU don't have to be fast/slow for the race condition to exist, the SCRIPT has to not have finalised and stored the session which is the deciding factor." The question becomes, why do we need to be concerned about a timing-related race condition if mutual exclusion is being properly handled? We are concerned here with the finalizing of the storing of the resource (the article and its content). Timing cannot be relied upon as a way to protect the resource.

Next Steps

I will continue trying to reproduce the issue. It would be great if someone familiar with the design and coding of this particular realm of Joomla could take a look at what I've presented here and give some feedback.

thanks,
JC


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.
avatar durian808
durian808 - comment - 30 Jan 2016

FORMATTING CORRECTION TO LAST POST:

First I'd like to say that this potential issue under investigation is probably unlike a lot of issues that are dealt with here. It is very difficult to reproduce, yet has been experienced and reported by many over the years since at least 2012. As such, I don't think it should be treated as one would usually treat most potential issues in Joomla. I think we should be open to all information that has been collected about the issue. I understand @PhilETaylor's concern to not waste time/effort on non-issues; however, I think there is a good chance that what we are observing here is, in fact, a real issue. That is of course my perspective, but it is a view based on my observation of the issue over the span of many years. Sometimes with issues of this nature, where reproducing the problem may prove to be too difficult, what is required is an approach of looking at symptoms and then theorizing root cause, followed by logical analysis of how the root cause could arise, and then inspection of related code. For some issues, the approach of "we're not going to look at this because we can't reproduce it" falls down.

I think I also need to emphasize that, although the issue can be precipitated by a repeated test as discussed above in this tracker item, the issue has also been seen in the field under normal-use conditions, confounding many a Joomla end user with a message that is essentially meaningless to most. There is only one place in the Joomla core that a 403 error can be raised along with the message "You are not permitted to use that link to directly access that page...", and that's in components/com_content/controller.php. There are also 14 more instances inside the administrator components where this same message can be displayed, however not along with a 403 error. All cases relate to editing a resource, and all use essentially the same code and rely on the function, $this->checkEditId().

Theory:

I would now like to outline a theory as to why this 403 error is produced when attempting to edit an article in the front end, based on my observations during yesterday's testing and also from recent server log data that shows the same problem in the field. Here is a summary of the theory:

  1. Under certain conditions which are difficult to reproduce, the data kept by Joomla associated with the login session and/or editing session becomes corrupted. (Since I don't know the specific details of the underlying mechanisms in the software here, I will just refer to this as "session data".)

  2. Once corrupted, a subsequent click of the/an article edit icon will result in Joomla reporting to the user, "You are not permitted to use that link to directly access that page...". (Most users probably don't notice that this is also a 403 error, but this fact is a key to noticing occurrences of this in server logs.)

  3. At the point where the 403 error is produced by components/com_content/controller.php, this is already well past the point where the root cause of the issue corrupted the session data.

  4. I have observed this issue recently in both Joomla 2.5.x in the field and Joomla 3.4.8 during testing on a fresh install demo site at SiteGround. The code that produces the 403 error in components/com_content/controller.php is the same in both Joomla versions noted here.

Analysis:

If this theory is correct, then the logical place to look for root cause is in the code that locks the resource (article) for the operation (editing). If the resource is not locked properly, either by a design flaw or by a bug/defect, then one possible outcome is that the same user session will be able to open the resource more than once, at the same time, for the operation. My test results yesterday actually showed probable evidence of this. As I was performing the test, and before the 403 error was produced, I made an edit to the article, saved it, and then re-opened the article to find that the previous edits were absent. In other words, it was as if the new editing session had loaded the previous copy of the content, and as if the previous save request had not completed^^. Whatever flaw in the code allowed this to happen, I think this is the point where the session data was corrupted. Clearly this is a flaw, because the resource should be locked right up to the point where the save operation comes to full completion. Mutual exclusion must be maintained for access to the resource – evidence seems to indicate it was not. @PhilETaylor quickly suspected that a race condition was happening, however this is perhaps a different race than he imagined. Again, this only happens under certain hard-to-reproduce conditions. (^^Or perhaps the previous save operation failed without reporting a problem.)

@PhilETaylor noted, "...YOU don't have to be fast/slow for the race condition to exist, the SCRIPT has to not have finalised and stored the session which is the deciding factor." The question becomes, why do we need to be concerned about a timing-related race condition if mutual exclusion is being properly handled? We are concerned here with the finalizing of the storing of the resource (the article and its content). Timing cannot be relied upon as a way to protect the resource.

Next Steps

I will continue trying to reproduce the issue. It would be great if someone familiar with the design and coding of this particular realm of Joomla could take a look at what I've presented here and give some feedback.

thanks,
JC


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.
avatar fruppel
fruppel - comment - 30 Jan 2016

can be precipitated by a repeated test

it can't

avatar PhilETaylor
PhilETaylor - comment - 30 Jan 2016

Sometimes with issues of this nature, where reproducing the problem may prove to be too difficult, what is required is an approach of looking at symptoms and then theorizing root cause, followed by logical analysis of how the root cause could arise, and then inspection of related code.

You mean just like I have?

someone familiar with the design and coding of this particular realm of Joomla could take a look at what I've presented here and give some feedback.

I did - and you did not like my response!

4 people have tested and not one person can replicate your issue

this only happens under certain hard-to-reproduce conditions

Code doesnt change. Thats the beauty of code,

So Im out... I see nothing to fix here until it is reproducible - unsubscribing now, and recommend this issue is closed until some action is possible else we end up with a tracker full of issues no one cares about and no one can action.

avatar mbabker
mbabker - comment - 30 Jan 2016

@PhilETaylor I've seen it happen and frankly I've never cared enough to debug into what causes the issue. But it is something I have seen randomly over the last couple of years and it's not something that has explicit reproduction instructions (like I said, it was pretty random; I can't recall it ever having anything to do with session timeouts which could point back to the internal state tracking getting corrupted somehow but frankly it's just quicker to log out and log back in than take the time to figure it out).

avatar PhilETaylor
PhilETaylor - comment - 30 Jan 2016

Ask @tampe125 about Joomla sessions - he will tell you what absolute crap state their are in... He will be able to explain that this is a huge rabbit hole...

Im not saying there is not an issue - but there is nothing new in this Github Issue that is not already in the minds of developers to watch out for, there is not something that can be acted upon. Therefore its just another one of those Github issues that will linger for years until someone gets the guts to close it. You cannot fix what you cannot see broken over and over enough to be able to identify the root cause and address it with code changes.

Simply stating there is a problem is not good enough.

Joomla already saves far too much data into its session ... to the determent of its security (see Christmas 2015 for that!) and to decouple the editing process from the session is a MASSIVE undertaking.

avatar durian808
durian808 - comment - 30 Jan 2016

In response to @PhilETaylor: You can, indeed, fix problems that you cannot easily reproduce. It's just not commonly done. For some of the most difficult-to-resolve issues, what other place do you suggest the Joomla community come together to solve those problems, other than here in the Github? Further, you didn't address anything I said in my carefully-worded post about properly handling mutual exclusion for access to the resource, in this case the article. (Can anyone direct me to that code? I'll have a look for myself!) Yeah, simply providing inroads to solving a problem isn't good enough – you actually have to solve the problem, not complain about how bad things are or deny that anything can be done, or say that it's too difficult. So what exactly are we doing here, folks? I don't get it... to me, this is not the Joomla spirit!

To others who have responded about not being able to reproduce the problem: thank you for trying; however, just because you can't reproduce it, doesn't mean that it can't be reproduced, and doesn't mean it's not a problem. Others may be able to reproduce it, under certain conditions. @mbabker and @brianteeman have seen instances of this issue – it is indeed a known problem. It's an annoying problem, and a high-quality CMS like Joomla shouldn't be spitting out this kind of worthless message that don't mean anything to 99% of end users. I'm a software engineer – I understand that something like this is tolerable and benign, to a point, but to have it persist for years in a leading CMS... I think it's just not acceptable.

@PhilETaylor also says, "decoupling the editing process from the session is a MASSIVE undertaking." Can someone please explain to us how the proper handling of mutual exclusion in the case of the edit operation somehow implies "decoupling the editing process from the session"? If the answer to that is no, then the right people are not yet looking at this tracker issue.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 30 Jan 2016

I'm out - I'll feed the troll no longer

Your words show you know nothing about software for the web or respect those that do

Mutual exclusivity has nothing to do with this problem at all!! You can't just make up stuff to fit your ideas! It's clear you don't understand the underlying issue here in the slightest.

Http is stateless - the use of sessions is the way to persist data, but your session IS exclusive ....

Life is too short to debug one http request that failed per day - 4 people including you can no longer repeat that failure and I spent 39 mins straight on your site attempting to break it following your provided instructions

I have spent a further 1.5 hours setting up 4 servers with different session handlers to test a theory

I then spent 3 hours looking at code related to form editing in the frontend

All while trying to replicate this issue repeatedly - and not a single failure!

There are much better bugs that need resolving in joomla than the one that happens on 0.0000000001% of requests for that one person (yes I made that figure up!)

So ill concentrate on those - important show stoppers - rather than this And leave you to your words

Im happy to bet this issue will never be resolved.

Sent from my iPhone

On 30 Jan 2016, at 03:05, durian808 notifications@github.com wrote:

In response to @PhilETaylor: You can, indeed, fix problems that you cannot easily reproduce. It's just not commonly done. For some of the most difficult-to-resolve issues, what other place do you suggest the Joomla community come together to solve those problems, other than here in the Github? Further, you didn't address anything I said in my carefully-worded post about properly handling mutual exclusion for access to the resource, in this case the article. (Can anyone direct me to that code? I'll have a look for myself!) Yeah, simply providing inroads to solving a problem isn't good enough – you actually have to solve the problem, not complain about how bad things are or deny that anything can be done, or say that it's too difficult. So what exactly are we doing here, folks? I don't get it... to me, this is not the Joomla spirit!

To others who have responded about not being able to reproduce the problem: thank you for trying; however, just because you can't reproduce it, doesn't mean that it can't be reproduced, and doesn't mean it's not a problem. Others may be able to reproduce it, under certain conditions. @mbabker and @brianteeman have seen instances of this issue – it is indeed a known problem. It's an annoying problem, and a high-quality CMS like Joomla shouldn't be spitting out this kind of worthless message that don't mean anything to 99% of end users. I'm a software engineer – I understand that something like this is tolerable and benign, to a point, but to have it persist for years in a leading CMS... I think it's just not acceptable.

@PhilETaylor also says, "decoupling the editing process from the session is a MASSIVE undertaking." Can someone please explain to us how the proper handling of mutual exclusion in the case of the edit operation somehow implies "decoupling the editing process from the session"? If the answer to that is no, then the right people are not yet looking at this tracker issue.

This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

Reply to this email directly or view it on GitHub.

avatar durian808
durian808 - comment - 30 Jan 2016

"Im happy to bet this issue will never be resolved."

Wow, I would not want this guy on my team. I appreciate your help, but you contradict yourself constantly. First, you constantly attack the messenger, then spend hours working on the issue... it doesn't make sense. Yes, please do not respond to this item further.

No troll here. Just a dedicated web developer and someone who cares about my clients.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar cpaschen
cpaschen - comment - 30 Jan 2016

Just a thought here about this issue 'loosely' and other issues like it
(that are very hard to reproduce, but that have been 'seen' by many
people - I've also been one that has seen this) ...
What if we were to have a category of 'issues' for things like this that
should be addressed at some point - but shouldn't take up the time of
our 'top tier' developers here. Maybe a list for things that could be
attacked during a bug-squash day - where you might have a bunch of
people available for an intense effort. Maybe not to actually resolve
the problem (during that bug squash), but at least to better document it
so that it could then be resolved later?
I agree with durian808 that I'd hate to see this completely ignored
(forever) ... but I also agree with Phil that things that are only very
occasionally evident shouldn't be taking up that much time here.
Do we have a 'some day' category? (Or can we create one?)

Just a thought ...

On 1/29/2016 9:42 PM, durian808 wrote:

"Im happy to bet this issue will never be resolved."

Wow, I would not want this guy on my team. I appreciate your help, but
you contradict yourself constantly. First, you constantly attack the
messenger, then spend hours working on the issue... it doesn't make
sense. Yes, please do not respond to this item further.

No troll here. Just a dedicated web developer and someone who cares
about my clients.


_This comment was created with the J!Tracker Application
https://github.com/joomla/jissues at
issues.joomla.org/joomla-cms/9013
https://issues.joomla.org/tracker/joomla-cms/9013.


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar gwsdesk
gwsdesk - comment - 30 Jan 2016

@durian808 I cannot reproduce this also in what has been described but have seen it in the past in admin as well as @brianteeman mentioned. Please show some respect to @PhilETaylor. Like him or not but he is one of our great contributors and belongs to the Founding fathers of our Mambo/Joomla environment together with Brian who posted here also so he does show some attention and respect when he replies and spends considerable time to replicate the issue you post?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 30 Jan 2016

@gwsdesk well, @PhilETaylor gets what he deserves in this case. I will not sit by while he pisses on me, and I don't care what level of God-like status he may have in the Joomla community. I appreciate everyone who has worked hard to bring Joomla to what it is today... I really do. But this bloke's manners suck badly... perhaps he needs to take a break from it all.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 30 Jan 2016

Ok so I have driving home for 3 hours on the motorway, giving this issue my undivided attention - so much so I drove 30 miles in the wrong direction after missing my junction.

I feel I need to explain this issue in full - because quite frankly Im fed up with people blaming Joomla without a full knowledge of the truth, facts or the "issue" they are reporting.

This "Issue" will not be resolved, as its not a "bug" but a whole concept, and one that Joomla cannot fully control.

Firstly some maintenance, The linking and bundling of #8757 and #8731 to this one is simply incorrect. The reason for this is that those issues directly are related to the important security work conducted in later Joomla versions in December 2015 to specifically address security issues with the Joomla session and old PHP versions. The issues reported against those Joomla versions with the old chestnut "You are not permitted to use that link to directly access that page" were directly caused by, and resolved by changes made in the Joomla Session layer as part of the excellent work by @tampe125 and the Joomla Security Strike Team. These issues were CONTINUOUSLY REPRODUCIBLE by the select few in the JSST that were involved in the week before Christmas and were directly attributable to the changes made to secure the session. Once identified this SPECIFIC issue was addressed by @tampe125 and Joomla 3.4.8 was released which SPECIFICALLY fixed the regression, and further enhanced the upgrade process by migrating of sessions from the old format to the new format. That SPECIFIC issue resulted in the GENERIC "You are not permitted to use that link to directly access that page" error message as the input to the request was invalid - more on that in a moment. Furthermore, an additional issue was found during this debugging that for some reason allowed an expired state session not to redirect you to the logout/login page if you were logged in, and your session expired, This would give the same error message but is now fixed. For this reason, your reported issue cannot be bundled with this one, as yours is unreproducible in the main, with only sporadic reports of issues over the last 10 years. However Im going to address that here now.

I am a full stack developer - I approach this from a complete understanding of the Joomla code which I work with every single day of my life since the 9/11 event when the code was still Mambo code, the PHP Session Layer, and the storage backends for session storage, Im not interested in fluffy "it doesn't work" or "well sometimes this happens" - I work with code, and not people, because code doesn't change! For a given set of "INPUT" code will always product the same "OUTPUT" - which is the way my brain likes to work, I don't deal in half truths, only in absolutes.

So lets start with your report.

You are reporting that occasionally, over the last 10 years, you randomly, and unreproducible get an error from Joomla that states "You are not permitted to use that link to directly access that page" or generic messages to that effect.

So lets start your education here... for all of us...

Joomla works, at the most simple concept, of inputting a set of inputs, letting the code do its job and that will output something.

The inputs to Joomla are many

  • Server Environment
  • Request Data & Method, including Cookie data & Browser Information (the URL, user agent etc)
  • Session Data
  • (and more but lets not go there...)

Some of these inputs we have control over to a greater and lesser degree.

Joomla (From now on called the PHPApp because THIS IS NOT A JOOMLA ISSUE) takes this information and then processes it the way it is designed to.

Depending on the inputs, the output will be different, or the same. For a given state, for a given set of inputs the result WILL ALWAYS be the same, that is why code is so nice - its not a women!

Most PHPApps use sessions. The http protocol is, by design, a stateless protocol and so to persist something (like a login) there needs to be a special way to do that. The way that PHP does it is with sessions. A session has two bits of information in two different places. A cookie in your browser, and the session data stored in the session handler on the remote server. There are many session handlers, Joomla for instance by default (rightly or wrongly in 2016!) uses the database to store sessions. This is mainly a legacy thing as more appropriate session handlers are now available. For example SiteGround allows uses of the Memcache and Memcached session handler to persist the session into the RAM of the server. Each session handler has its plus and minus points - the database handler obviously relies on a Mysql database, which in turn relies on the mysql service, which in turn relies on the loads on the server down the stack we go...

When you load a page of a PHPApp your browser sends your cookies to the server, PHP takes that and loads up your session from the session handler - database in Joomla's default. THIS THEN BECOMES AN INPUT TO THE REQUEST. So lets say half way through the experience in your PHPApp Admin, the server admin restarts mysql, truncates the table, the mysql service stalls the request, the sessions table is locked or the session row is removed from the table, the PHPApp admin forces a user to be logged out using the User Manager - all these things can effect the INPUT TO THE REQUEST - and all of these things you have no control over. The same is true if you delete your session cookie, the server starts you a new session (normally, if IP Stapling is not a feature), the SERVER has no control over you removing the input. Some PHPApps stable the session on IP Address and Browser Identification, so if your IP address changes during a session, you get a new one! Joomla code has this ability but its not implemented. There is also a timestamp/session length and if your session has expired, IT IS STILL LOADED but can have one of many states, one of 'inactive'|'active'|'expired'|'destroyed'|'error' !!!

So thats the input covered, in summary you only have control over some of the input.

So lets look at what PHPApp does with the request.

When you load a page, PHPApp takes your session cookie, and attempts to load that session id from the session handler. It does some checks to ensure its valid and then loads it as part of the $_SESSION which forms part of the INPUT to the PHPApp. If everything is ok then the output is what you expect. However if the INPUT is not ok, for example the session has expired, or doesnt contain the information (inputs) that the PHPApp code is written to expect then the output will be different to what you expect - for example you might get a message like "You are not permitted to use that link to directly access that page" because you no longer have a session with access permissions to that page.

In a PHPApp additional information (like the page id to edit) can be stored in the session. PHPApps like Joomla use this for persistence. In fact Joomla stores too much information to the session really, and this was the cause of the December 2015 security issues (in conjunction with old PHP versions).

If the PHPApp is so designed to accept an input that contains a valid "Active" session and a page id, and it doesn't get those in the input - then the output will be different to your expectation - a generic message like "You are not permitted to use that link to directly access that page" might be displayed which is factually and to the point - because you are not permitted because you did not provide the right inputs.

So lets turn our attention to after the page is loaded (whatever page, whatever state), the PHP Code then runs its shutdown and writes the session to the session handler, which in turn saves it somewhere. For legacy reasons Joomla by default uses a database to store its session data - arguably the slowest and most insecure place to save it. So long (relatively) after the page has been rendered in your browser, PHP is still working away to save your data ... depending on the speed of the session handler, the uncontrollable stack of the server loads, mysql loads, table locking etc... On a crap web host this will obviously take longer than on a good web host with low latency and low loads - the use of other session handlers, like memcache which write to RAM will be considerably faster.... but for now... imagine we are on a shared web host with above average loads and a slowish mysql server... like some have a mysql server that is separate from the web server so all data needs to be transmitted over a network to get to a mysql instance... further slowing the process...

"slow" here is a subjective term here, an increase of milliseconds, seconds...

Eventually your session data is written somewhere... Slowness of saving session data has been an issue for many years as evident from the comments in http://php.net/session_write_close

So lets see what happens if you continuously hit refresh on a page. Your session data is loaded from the session handler (database) each time, and written back to the database at the end of the page load... If this were a "page counter" PHPApp then the PageCount loaded from the database if you quickly refreshed the page 5 times could be the same number, or ir could be a greater number. It could be the SAME NUMBER because you have READ the session information from the session handler before the previous request has finished WRITING the last requests session data - but this is the way sessions work! A simple read/write.

So obviously this is not wonderful - but the whole of the internet works this simple way. But then again we are not expecting to sit here hitting refresh every 0.1 milliseconds just to get invalid INPUT to our code.

This is an example of a race condition. This is explored in the comments of the http://php.net/session_write_close article as well.

So lets look now at expiry (Pre Joomla 3.4.8) of a session. If you open a page, login, your session starts and a time and expiration is set. If you walk away from your computer and return after that time - your page is still loaded in your browser, but your session has (or more accurately, will on next access, or on garbage collection) expired. So if you click an Edit Article link you quite rightly get an access denied type message as the INPUT to the PHPApp doesn't contain the required information it expects (for example the users id or acl level). This is quite right.

Prior to Joomla 3.4.8 there was a bug, which was finally identified (Ironically by me, but explored and fixed by David) in 3.4.7 Admin Console, but could well be in ALL JOOMLA VERSIONS EVER, that when the state of a session expired (or more accurately was not "active"), that the Admin console would still refresh as if you were logged in, but you would get the generic error message "You are not permitted to use that link to directly access that page" - this was REPRODUCIBLE EVERY TIME by the select few of the Joomla Security Strike team and was fully addressed in Joomla 3.4.8, now if you open up Joomla admin and leave it open on a non-edit page beyond the session lifetime, on your next interaction with the admin console you are redirected to the login page. Again a massive hat tip to @tampe125 for giving up his christmas eve for that! It is wholly possible that this is part of the bug that effected the front end and is now resolved there and that the issue you are reporting for 10 years is indeed resolved. Time will tell.

However Ive lost my chain of thought here...

The SiteGround platform (and platforms that cache in general) are also another good point to think about here. For example, ON YOUR DEMO SITE it is IMPOSSIBLE to login if you (starting from a clean slate browser, no cookies, no active login session) go direct to http://durian808.demojoomla.com/index.php/login before any other page, you will see that no session cookie gets written, and so attempt to login, you will find you get generic INVALID_TOKEN - this is reproducible EVERY TIME ON YOUR DEMO SITE and is CAUSED by SiteGround caching the login page, which has a session based token in the source code! This further highlights the fact that not everything is under the PHPApp control when it comes to INPUT and that an invalid input will result in an unexpected output - this is a GREAT example of how a WEB HOST has destroyed a core, working, bug free feature of Joomla! But users will complain that this is a bug in Joomla when it is not!

(Note SiteGround Caching might have cached the login page from a prior interaction)

HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sat, 30 Jan 2016 07:02:15 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Cache-Enabled: True
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Last-Modified: Sat, 30 Jan 2016 06:52:24 GMT
Host-Header: 192fc2e7e50945beb8231a492d6a8024
X-Proxy-Cache: HIT

However if you go to the same page with a query string http://durian808.demojoomla.com/index.php/login?Asd

Then you get


HTTP/1.1 200 OK
Server: nginx/1.7.9
Date: Sat, 30 Jan 2016 07:02:48 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Vary: Accept-Encoding
X-Cache-Enabled: True
Expires: Mon, 1 Jan 2001 00:00:00 GMT
Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
Set-Cookie: ecc01c8447c94640ccd59a7d1aa40801=nvjmpf0si9qhh65i4stljae7c0; path=/; HttpOnly
Last-Modified: Sat, 30 Jan 2016 07:02:48 GMT
Host-Header: 192fc2e7e50945beb8231a492d6a8024
X-Proxy-Cache: MISS

and you can login.

I had a whole paragraph here about session management when opening several tabs to the site, but I just deleted it by mistake! as Ive now had no sleep all night! However the same is true of tabs, if you logout in one tab, this effects the sessions in other tabs to the same site... This can cause unexpected concequenses as well.

So my point is... that you might like to mark this out as a 10 year Joomla bug, when infact there is a lot more at play - the inputs to Joomla are uncontrolable when it comes to session data, race conditions and even recent session data migrations while users are logged in, can all cause session data to be different than what Joomla expects and Garbage in Garbage out, the GENERIC error message shown is appropriate as the information contained in the request, merged with the session data, is not acceptable and what is needed to be able to access that resource.

As we have seen there are many ways this session data can be changed/corrupted/deleted which are totally out of Joomla's control. And other times, like Joomla 3.4.6/7/8 upgrade path where Joomla specifically changed the structure of the session...

We have not even began to explore the impact that custom extensions can have on the session... including those that manipulate $_SESSION object directly without using the Joomla API.. We have not explored the myriad of settings there are in php.ini or in individual session handlers that can inpact session...

This "issue" is far greater than blaming Joomla for not addressing a 10 year old "bug" that is unreproduceable - a bug that is 100% un-reproidueable doesnt exist. On the too few occassions you get your GENERIC ERROR MESSAGE it instantly goes away on the next page load, or session creation, and the people reporting the "bug" dont have enough information about the INPUTS to the PHPApp in order to make educated decisions, or accurate bug reports... For example, you have not even told us which session handler you use, the session configuration options chosen in Joomla Global Configuration, your full PHP version and configuration settings for each site that you can replicate this on.

In summary - for a given set of INPUTS Joomla will result in a given OUTPUT. You dont have control over all the inputs, or the knowledge or experience to capture them at the point the "bug" occurs for you - which is understandable and acceptable - but calling this a 10 year Joomla bug is simply not accurate in the slightest.

Im not pissing on anyone, but I have - repeatedly - looked into this over many many years and understand the issues better than most, certainly a hell of a lot more than the average punter who says "it doesnt work - its joomla's fault!"

avatar PhilETaylor
PhilETaylor - comment - 30 Jan 2016

Here is a good example of session race conditions on YOUR DEMO SITE - when you repeatedly click the save button quickly without the browser moving to the next page, you are firing off requests, the article IS BEING SAVED and the session written,and then then on the next request read, and written and the messages appended) so racing ahead (in the nonconventional sense) of the browser, leading to unexpected messages

AGAIN this is NOT a Joomla bug - but expected behavior of use of the http protocol, and PHP sessions.

screen shot 2016-01-30 at 07 34 29

avatar durian808
durian808 - comment - 30 Jan 2016

OK @PhilETaylor, I appreciate that a great deal and I think maybe you set a record on that one. I am also looking at this issue right now, sniffing around in the code, looking at call stacks, when I should definitely be working on a bunch of other things. But I've always had a soft spot for tracking issues down. As we used to say in my software engineering days, sometimes it helps to have another set of eyes take a look at it. I'll take the red pill on this one and see how far down the rabbit hole it goes. :)


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 30 Jan 2016

Here is the point at which the edit task is about to be executed:

./components/com_content/content.php:38
    $user  = JFactory::getUser();
    $controller = JControllerLegacy::getInstance('Content');
    $controller->execute($input->get('task'));

@PhilETaylor or someone, please give us a detailed explanation of how this function works...

/**
     * Get an user object.
     *
     * Returns the global {@link JUser} object, only creating it if it doesn't already exist.
     *
     * @param   integer  $id  The user to load - Can be an integer or string - If string, it is converted to ID automatically.
     *
     * @return  JUser object
     *
     * @see     JUser
     * @since   11.1
     */
    public static function getUser($id = null)
    {
        $instance = self::getSession()->get('user');

        if (is_null($id))
        {
            if (!($instance instanceof JUser))
            {
                $instance = JUser::getInstance();
            }
        }
        // Check if we have a string as the id or if the numeric id is the current instance
        elseif (!($instance instanceof JUser) || is_string($id) || $instance->id !== $id)
        {
            $instance = JUser::getInstance($id);
        }

        return $instance;
    }

thanks!


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar Webdongle
Webdongle - comment - 30 Jan 2016

@durian808
Logged in
Clicked the edit icon
Added some text
Saved
FULL SUCCESS
works as expected
screen shot 2016-01-30 at 02 57 00


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 30 Jan 2016

@Webdongle let's say 97% of the time (maybe more) it will work as expected. This is not the point, which I will explain here.

I am trying to reproduce the conditions that produced the error during the test that I reported at the top of this tracker item. As I said before, it was very easy to set up the demo site, add a blog, add some articles, and then run the test on the top blog article – the error showed up immediately. Why it ceased to show up after that is a complete mystery, but I am not giving up. It is true that doing a test like this, in a loop, is not "normal use," but nonetheless it is revealing a shortcoming, and one that has been seen in the field by many people over the years. It occurs randomly. I need to reproduce it again, with my demo site at SiteGround (J3.4.8), because I'd like to get a better sense how sensitive this effect is to the rate at which the edit icon is clicked after the article is saved. The reason being, this has a direct bearing on how likely the issue is to arise during normal use. That is the key. But consider this scenario: An end user is editing an article, they make some changes and then save the article. When the screen reloads, they realize they forgot something during the edit and they immediately click the edit icon again. They get an error they've never seen before, which says "You are not permitted to use that link to directly access that page...". Not understanding the message, and not knowing why it is being shown, they notify their webmaster. The webmaster responds, "Oh sorry, yes this is a known issue in Joomla... you can ignore it". This is why I'm working on this issue. But this is only one scenario, and for ALL WE KNOW, this issue is not even timing related – in the sense that it will ONLY show up when the edit icon is clicked very quickly after the article was saved. At this point, it APPEARS that this particular timing-related situation – or other timing-related situations that delay the completion of the previous operation (edit) – ARE the ONLY cause of the problem, or the de facto root cause, but without knowing exactly what's going on in the code, coupled with the fact that it's a very difficult thing to reproduce, makes arriving at a definitive explanation very difficult.

I have 25 live Joomla sites being used by clients in the real world. Some are 3.4.8, some are 2.5.28, and some are in the queue for upgrade. I am seeking a fix for this issue so that it can be incorporated in 3.4.X, but also for a hot patch on the older sites.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

Humour me here... how many browser tabs did you have open when you were testing and replicated - that once - the problem you are reporting?

avatar durian808
durian808 - comment - 31 Jan 2016

Do you mean tabs open to the demo site, or just total tabs open? If you mean the former, just one for the front end and one for the back end. I believe I had also logged into the back end during the test. If you mean the later, maybe something like 15... around the same number as now when I am trying and failing to reproduce the problem. FireFox 43.0.4 on Mac.

I'm going to try a couple options soon... one is creating another SiteGround demo site, basically exactly like the first one, and then test again just as I did before. I'm feeling like there must be something going on with the initial state of the site. Another option is to do the test on a vanilla 3.4.8 on my server, already set up. I'm feeling like the issue happens on blog articles more than other articles, from past experience.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar ggppdk
ggppdk - comment - 31 Jan 2016

@durian808 , this is not a issue unless

  • it is either persistent ! or at least a little often (which is not at all with J3.4.8 !)
  • or it is very rare (like our case) but it leads to data loss, (please don't argue mean about how rare it is don't care to prove anything about how rare it is)

Persistent case , is the case of a (server issue) that a server is configured to send DEFAULT cache-ON HTTP headers for HTML content to the browsers,

  • in which case user is unable to edit the until he relogins ! (i suggested a workaround / fix for this #8757)

If you are worried of a user not retrying the edit-link, and contacting support, then

  • you may suggest here to modify the default message, asking user to retry once, or some more appropriate message / more informational message
  • you can use Joomla language manager to override it too (please don't argue me about adding the language string you have spent a more time here, regardless of how many sites you have), use the language overrides of strings to suit your needs
avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

I believe I know what is happening, I will write it up shortly as its 8am on a Sunday but the tl;dr; is that this is not a bug, its by design , but needs to be explained and users educated why it is such. I'll write up my findings shortly.

tl;dr; is that this is not a bug, its by design

avatar durian808
durian808 - comment - 31 Jan 2016

@ggppdk occasional random meaningless message by prominent CMS to end user is not good, maybe a little ok because not critical error, but really not cool to poor end user if there is reasonable easy way to fix by smart developer. Suggested new default language string: "Sorry, Joomla was not designed to respond 100% properly to what you just attempted. Please try again in a little while."

@PhilETaylor that's awesome... please humor me. I need some humor. If it's by design, then may I suggest the design be enhanced.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

@durian808 It is BECAUSE Joomla is designed 100% to handle the invalid input that is why you are seeing this message! Please stop calling this out as a Joomla bug - its not.

avatar durian808
durian808 - comment - 31 Jan 2016

@PhilETaylor OK, don't call it a bug... let's call it a design shortcoming that impacts the UI. Right? Come on people! Also, why are you calling it an "invalid input"? You want to tell that to the end user? That makes no sense at all. Anything the user can do on that front end of Joomla is fair game, especially if it's just something like clicking the frickin' edit button. Just because this is a web application doesn't change the fundamental requirements of a user interface.

Why are you Joomla developers so defensive about this? Is it because there are too many other "real bugs" that need to be fixed?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

Listen, when you can replicate this then come back and post - until then - stop replying and focus your energy on that... you are doing nothing to help here...

avatar durian808
durian808 - comment - 31 Jan 2016

Sure, I won't respond when people stop saying there is no issue. I am now eagerly awaiting your explanation of why it is happening. Or should we just bypass that and say that it's because there is no lock on the resource (article)? That's what it looks like to me... one edit request on top of an unfinished one, causing session data corruption, followed by a 403 error because Joomla gives up.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

I have already - repeatedly - told you what the problem is - yet you continue to ignore the simple facts. The problem is documented over 10 years in the PHP.net comments.

*## This is a hosting issue and not a Joomla issue. *

The problem is a race condition where the session is not stored fast enough before the redirect to the form takes place. You may not know but when you click the edit button it loads one request, and then that 303 redirects to the form.. the fact is that the session is not saved fast enough between these page loads. The first request sets an edit context (or a "lock" as you put it) and the second page load looks for that context.

The first request loads:
index.php?task=article.edit&a_id=1&return=aHR0cDovL3JlbW90ZTM0Lw==
which 303 redirects to
index.php?view=form&layout=edit&a_id=1&return=aHR0cDovL3JlbW90ZTM0Lw==

If the first request has not yet closed and written the session then the second request will quite rightly give a 403 error.

To see this context locking as it should be - open the same article in two tabs, click the edit button on both articles (one in each tab) now save the article in one tab, and you will see that when you save the article in the other tab you REPEATEDLY get the 403 error message as you no longer are holding open the article in a session context. This again is by design and is not a bug.

Joomla is designed to prevent direct linking to forms for security purposes, this is not going to change. If you do not like the way that then I'm sorry - its not about to change unless you write the code that replaces ALL the forms in Joomla.

So yes this COULD be changed in Joomla so that the edit process doesn't require a redirect - but would be a MASSIVE change breaking backward compatibility for such little win - for a problem that is not reproducible on decent web hosts.

I say if you are having these kinds of sessions issues - repeatably - then you should change web hosts.

HOWEVER - I believe that I have found a simple change to Joomla that might fix the race condition on a redirect as explained above, and I'll submit a pull request in a moment. At the moment the code just exit() after a redirect, instead of writing the session specifically with sesison_write_close() the PR will change that to explicitly save the session.

avatar PhilETaylor PhilETaylor - reference | c1d60b0 - 31 Jan 16
avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

PR #9035 now opened - recommend this issue is closed as a PR is available. As this cannot be reproduced, it is hard to test - but apply the PR and test as much as possible. This will not be released for sometime anyway.

avatar durian808
durian808 - comment - 31 Jan 2016

One thing stands out...

@PhilETaylor:
"To see this context locking as it should be - open the same article in two tabs, click the edit button on both articles (one in each tab) now save the article in one tab, and you will see that when you save the article in the other tab you REPEATEDLY get the 403 error message as you no longer are holding open the article in a session context. This again is by design and is not a bug."

Why does Joomla need to allow two editing sessions to be open simultaneously for the same user?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

Because its been that way since mambo days... when you open an article to edit it - it is "checked out" until you save or cancel on it.

If you just closed the tab and had coffee, and came back, you can still then click edit to get back into the edit screen for that article - if you are the person that checked it out.

If Joomla did not allow the article to be edited twice, as you say, then there would be no way back into the article at all until it was checked back in - a function only available to admins

avatar durian808
durian808 - comment - 31 Jan 2016

Oh, this is getting rich... been that way since the mambo days. You know, I was using mambo too back then!! Rock on!

Well, it's not acceptable. This is the problem... we've found the problem. When the user "checks out" an article, NOBODY should be able to edit that article, including the same user, until the article is saved or otherwise closed. If the user goes away and has coffee (or smokes something), so be it. If they have a Super User login (many do), then they can go into the back end and unlock the article (many do this already for various reasons... it's part of Joomla life.) If they don't have a Super User login – the webmaster does – email the webmaster and s/he will fix the problem. I happen to have a client, God bless him, who has an online magazine that gets a jillion hits per month. He is editing articles all the time... likes to have many tabs open... never turns off his computer... total chaos. It would be to his distinct advantage if he can't edit the same article from multiple tabs (which is just the kind of thing he would try, without knowing it).

Lock the article. If it times out because our friend left the room, Joomla can save and close it (make a article manager setting, save and close or just abort, whatever). Or, if the user comes back, opens a new tab, browses to the page... they should see a "Checked Out" status... that will be pleasant clue to them (feedback, good) that what they did was silly and then, they can click "Recover" and Joomla will do some magic to ensure that the user is re-opening one single editing session – and not opening a second editing session.

Done. Fixed. Let's go to the pub.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

We now have a PR - please test it - if you want massive changes then you should present that to the leadership, not as a note in a github issue.

avatar durian808
durian808 - comment - 31 Jan 2016

"...browses to the page... they should see a "Checked Out" status..."

Should be:
"...browses to the page, clicks edit, they should see a "Checked Out" status..."


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 31 Jan 2016

It's not a freakin' note in the github issue... It's the solution to a design problem that has been around since Mambo. The github issue reveals a design shortcoming that causes an annoying UI issue – the root cause is the design shortcoming. We started this in the Github because of that UI issue... is the Github not for UI issues? Why didn't you spot this on your first response to me? Because it took all these exchanges to arrive at this realization?

Look, man. I don't know your protocol here for what goes where, and frankly, that is beyond me at this point. I have gone way out of my way to help out here. What do we need to do to get this fixed?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

No - it is your suggestion for a change. Many will disagree with your concept.

For the record, I told you what the problem was here #9013 (comment) but you went on to shout about all kinds of things, and personal attacks...

I suggest you step away now. If you would like to request major changes and present those to the Production Leadership Team and/or have these added to the Development Strategy and Roadmap then this issue is not the place to do that.

There is a PR ready for testing that may or may not assist in SOME of the error messages caused by race conditions.

Quite frankly there is nothing else to see here... This issue should be closed.

avatar gwsdesk
gwsdesk - comment - 31 Jan 2016

I agree with closing. Makes no further sense to spend any effort here I
do believe

Leo

On 1/31/2016 5:30 PM, Phil Taylor wrote:

No - it is /your/ suggestion for a change. Many will disagree with
your concept.

I suggest you step away now.

There is a PR ready for testing that may or may not assist in SOME of
the error messages caused by race conditions.

Quite frankly there is nothing else to see here... This issue should
be closed.


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar brianteeman brianteeman - change - 31 Jan 2016
Status New Closed
Closed_Date 0000-00-00 00:00:00 2016-01-31 11:00:26
Closed_By brianteeman
avatar brianteeman
brianteeman - comment - 31 Jan 2016

Following our standard procedures this is now closed as we have a PR ( #9035)

Please test that PR and comment there if it works for you (and if it doesnt)


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar brianteeman brianteeman - close - 31 Jan 2016
avatar brianteeman brianteeman - close - 31 Jan 2016
avatar N6REJ
N6REJ - comment - 31 Jan 2016

@phil Not that my word carries much weight but I've seen this happen a
lot of time in the backend... as Michael says, I just log out and back
in to solve it. The most common time I've seen it is when going direct
to a extension to edit and it prompts me to log in then when I do so it
will throw the error.
Bear

On 1/29/2016 21:35, Phil Taylor wrote:

I'm out - I'll feed the troll no longer

Your words show you know nothing about software for the web or respect
those that do

Mutual exclusivity has nothing to do with this problem at all!! You
can't just make up stuff to fit your ideas! It's clear you don't
understand the underlying issue here in the slightest.

Http is stateless - the use of sessions is the way to persist data,
but your session IS exclusive ....

Life is too short to debug one http request that failed per day - 4
people including you can no longer repeat that failure and I spent 39
mins straight on your site attempting to break it following your
provided instructions

I have spent a further 1.5 hours setting up 4 servers with different
session handlers to test a theory

I then spent 3 hours looking at code related to form editing in the
frontend

All while trying to replicate this issue repeatedly - and not a single
failure!

There are much better bugs that need resolving in joomla than the one
that happens on 0.0000000001% of requests for that one person (yes I
made that figure up!)

So ill concentrate on those - important show stoppers - rather than
this And leave you to your words

Im happy to bet this issue will never be resolved.

Sent from my iPhone

On 30 Jan 2016, at 03:05, durian808 notifications@github.com wrote:

In response to @PhilETaylor: You can, indeed, fix problems that you
cannot easily reproduce. It's just not commonly done. For some of the
most difficult-to-resolve issues, what other place do you suggest the
Joomla community come together to solve those problems, other than
here in the Github? Further, you didn't address anything I said in my
carefully-worded post about properly handling mutual exclusion for
access to the resource, in this case the article. (Can anyone direct
me to that code? I'll have a look for myself!) Yeah, simply providing
inroads to solving a problem isn't good enough – you actually have to
solve the problem, not complain about how bad things are or deny that
anything can be done, or say that it's too difficult. So what exactly
are we doing here, folks? I don't get it... to me, this is not the
Joomla spirit!

To others who have responded about not being able to reproduce the
problem: thank you for trying; however, just because you can't
reproduce it, doesn't mean that it can't be reproduced, and doesn't
mean it's not a problem. Others may be able to reproduce it, under
certain conditions. @mbabker and @brianteeman have seen instances of
this issue – it is indeed a known problem. It's an annoying problem,
and a high-quality CMS like Joomla shouldn't be spitting out this kind
of worthless message that don't mean anything to 99% of end users. I'm
a software engineer – I understand that something like this is
tolerable and benign, to a point, but to have it persist for years in
a leading CMS... I think it's just not acceptable.

@PhilETaylor also says, "decoupling the editing process from the
session is a MASSIVE undertaking." Can someone please explain to us
how the proper handling of mutual exclusion in the case of the edit
operation somehow implies "decoupling the editing process from the
session"? If the answer to that is no, then the right people are not
yet looking at this tracker issue.

This comment was created with the J!Tracker Application at
issues.joomla.org/joomla-cms/9013.

Reply to this email directly or view it on GitHub.


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

@N6REJ What you are describing is a session expiration, followed by a login which attempts to redirect you back to where you wanted to go, but cannot because you wanted to go direct to a form. What you are seeing is exactly what should happen. There were excellent changes made to the admin session expiration redirection by @tampe125 (he will hate me tagging him again haha) in Joomla 3.4.8 which will effect and fix this for the backend

avatar N6REJ
N6REJ - comment - 31 Jan 2016

Phil, thanks for not sleeping... I FINALLY have a tiny idea of what
sessions do and exactly what happens when we "load a app".
I believe Mr. Sandman has a message for you ....
Bear

On 1/30/2016 01:21, Phil Taylor wrote:

Ok so I have driving home for 3 hours on the motorway, giving this
issue my undivided attention - so much so I drove 30 miles in the
wrong direction after missing my junction.

I feel I need to explain this issue in full - because quite frankly Im
fed up with people blaming Joomla without a full knowledge of the
truth, facts or the "issue" they are reporting.

This "Issue" will not be resolved, as its not a "bug" but a whole
concept, and one that Joomla cannot fully control.

Firstly some maintenance, The linking and bundling of #8757
#8757 and #8731
#8731 to this one is
simply incorrect. The reason for this is that those issues directly
are related to the important security work conducted in later Joomla
versions in December 2015 to specifically address security issues with
the Joomla session and old PHP versions. The issues reported against
those Joomla versions with the old chestnut "You are not permitted to
use that link to dire ctly acc ess that page" were directly caused by,
and resolved by changes made in the Joomla Session layer as part of
the excellent work by @tampe125 https://github.com/tampe125 and the
Joomla Security Strike Team. These issues were CONTINUOUSLY
REPRODUCIBLE by the select few in the JSST that were involved in the
week before Christmas and were directly attributable to the changes
made to secure the session. Once identified this SPECIFIC issue was
addressed by @tampe125 https://github.com/tampe125 and Joomla 3.4.8
was released which SPECIFICALLY fixed the regression, and further
enhanced the upgrade process by migrating of sessions from the old
format to the new format. That SPECIFIC issue resulted in the GENERIC
"You are not permitted to use that link to directly access that page"
error message as the input to the request was invalid - more on that
in a moment. Furthermore, an additional issue was found during thi s
debugg ing that for some reason allowed an expired state session not
to redirect you to the logout/login page if you were logged in, and
your session expired, This would give the same error message but is
now fixed. For this reason, your reported issue cannot be bundled with
this one, as yours is unreproducible in the main, with only sporadic
reports of issues over the last 10 years. However Im going to address
that here now.

I am a full stack developer - I approach this from a complete
understanding of the Joomla code which I work with every single day of
my life since the 9/11 event when the code was still Mambo code, the
PHP Session Layer, and the storage backends for session storage, Im
not interested in fluffy "it doesn't work" or "well sometimes this
happens" - I work with code, and not people, because code doesn't
change! For a given set of "INPUT" code will always product the same
"OUTPUT" - which is the way my brain likes to work, I don't deal in
half truths, only in absolutes.

So lets start with your report.

You are reporting that occasionally, over the last 10 years, you
randomly, and unreproducible get an error from Joomla that states "You
are not permitted to use that link to directly access that page" or
generic messages to that effect.

So lets start your education here... for all of us...

Joomla works, at the most simple concept, of inputting a set of
inputs, letting the code do its job and that will output something.

The inputs to Joomla are many

  • Server Environment
  • Request Data & Method, including Cookie data & Browser Information (the URL, user agent etc)
  • Session Data
    *

    (and more but lets not go there...)

    Some of these inputs we have control over to a greater and lesser
    degree.

    Joomla (From now on called the PHPApp because THIS IS NOT A JOOMLA
    ISSUE) takes this information and then processes it the way it is
    designed to.

    Depending on the inputs, the output will be different, or the
    same. For a given state, for a given set of inputs the result WILL
    ALWAYS be the same, that is why code is so nice - its not a women!

    Most PHPApps use sessions. The http protocol is, by design, a
    stateless protocol and so to persist something (like a login)
    there needs to be a special way to do that. The way that PHP does
    it is with sessions. A session has two bits of information in two
    different places. A cookie in your browser, and the session data
    stored in the session handler on the remote server. There are many
    session handlers, Joomla for instance by default (rightly or
    wrongly in 2016!) uses the database to store sessions. This is
    mainly a legacy thing as more appropriate session handlers are now
    available. For example SiteGround allows uses of the Memcache and
    Memcached session handler to persist the session into the RAM of
    the server. Each session handler has its plus and minus points -
    the database handler obviously relies on a Mysql database, which
    in turn relies on the mysql service, which in turn relies on the
    loads on the server down the stack we go...

    When you load a page of a PHPApp your browser sends your cookies
    to the server, PHP takes that and loads up your session from the
    session handler - database in Joomla's default. THIS THEN BECOMES
    AN INPUT TO THE REQUEST. So lets say half way through the
    experience in your PHPApp Admin, the server admin restarts mysql,
    truncates the table, the mysql service stalls the request, the
    sessions table is locked or the session row is removed from the
    table, the PHPApp admin forces a user to be logged out using the
    User Manager - all these things can effect the INPUT TO THE
    REQUEST - and all of these things you have no control over. The
    same is true if you delete your session cookie, the server starts
    you a new session (normally, if IP Stapling is not a feature), the
    SERVER has no control over you removing the input. Some PHPApps
    stable the session on IP Address and Browser Identification, so if
    your IP address changes during a session, you get a new one!
    Joomla code has this abili ty but i ts not implemented. There is
    also a timestamp/session length and if your session has expired,
    IT IS STILL LOADED but can have one of many states, one of
    'inactive'|'active'|'expired'|'destroyed'|'error' !!!

    So thats the input covered, in summary you only have control over
    some of the input.

    So lets look at what PHPApp does with the request.

    When you load a page, PHPApp takes your session cookie, and
    attempts to load that session id from the session handler. It does
    some checks to ensure its valid and then loads it as part of the
    $_SESSION which forms part of the INPUT to the PHPApp. If
    everything is ok then the output is what you expect. However if
    the INPUT is not ok, for example the session has expired, or
    doesnt contain the information (inputs) that the PHPApp code is
    written to expect then the output will be different to what you
    expect - for example you might get a message like "You are not
    permitted to use that link to directly access that page" because
    you no longer have a session with access permissions to that page.

    In a PHPApp additional information (like the page id to edit) can
    be stored in the session. PHPApps like Joomla use this for
    persistence. In fact Joomla stores too much information to the
    session really, and this was the cause of the December 2015
    security issues (in conjunction with old PHP versions).

If the PHPApp is so designed to accept an input that contains a valid
"Active" session and a page id, and it doesn't get those in the input

  • then the output will be different to your expectation - a generic message like "You are not permitted to use that link to directly access that page" might be displayed which is factually and to the point - because you are not permitted because you did not provide the right inputs.

So lets turn our attention to after the page is loaded (whatever page,
whatever state), the PHP Code then runs its shutdown and writes the
session to the session handler, which in turn saves it somewhere. For
legacy reasons Joomla by default uses a database to store its session
data - arguably the slowest and most insecure place to save it. So
long (relatively) after the page has been rendered in your browser,
PHP is still working away to save your data ... depending on the speed
of the session handler, the uncontrollable stack of the server loads,
mysql loads, table locking etc... On a crap web host this will
obviously take longer than on a good web host with low latency and low
loads - the use of other session handlers, like memcache which write
to RAM will be considerably faster.... but for now... imagine we are
on a shared web host with above average loads and a slowish mysql
server... like some have a mysql server that is separate from the web
server so all data needs to be t ransmitted over a network to get to a
mysql instance... further slowing the process...

"slow" here is a subjective term here, an increase of milliseconds,
seconds...

Eventually your session data is written somewhere... Slowness of
saving session data has been an issue for many years as evident from
the comments in http://php.net/session_write_close

So lets see what happens if you continuously hit refresh on a page.
Your session data is loaded from the session handler (database) each
time, and written back to the database at the end of the page load...
If this were a "page counter" PHPApp then the PageCount loaded from
the database if you quickly refreshed the page 5 times could be the
same number, or ir could be a greater number. It could be the SAME
NUMBER because you have READ the session information from the session
handler before the previous request has finished WRITING the last
requests session data - but this is the way sessions work! A simple
read/write.

So obviously this is not wonderful - but the whole of the internet
works this simple way. But then again we are not expecting to sit here
hitting refresh every 0.1 milliseconds just to get invalid INPUT to
our code.

This is an example of a race condition. This is explored in the
comments of the http://php.net/session_write_close article as well.

So lets look now at expiry (Pre Joomla 3.4.8) of a session. If you
open a page, login, your session starts and a time and expiration is
set. If you walk away from your computer and return after that time -
your page is still loaded in your browser, but your session has (or
more accurately, will on next access, or on garbage collection)
expired. So if you click an Edit Article link you quite rightly get an
access denied type message as the INPUT to the PHPApp doesn't contain
the required information it expects (for example the users id or acl
level). This is quite right.

Prior to Joomla 3.4.8 there was a bug, which was finally identified
(Ironically by me, but explored and fixed by David) in 3.4.7 Admin
Console, but could well be in ALL JOOMLA VERSIONS EVER, that when the
state of a session expired (or more accurately was not "active"), that
the Admin console would still refresh as if you were logged in, but
you would get the generic error message "You are not permitted to use
that link to directly access that page" - this was REPRODUCIBLE EVERY
TIME by the select few of the Joomla Security Strike team and was
fully addressed in Joomla 3.4.8, now if you open up Joomla admin and
leave it open on a non-edit page beyond the session lifetime, on your
next interaction with the admin console you are redirected to the
login page. Again a massive hat tip to @tampe125
https://github.com/tampe125 for giving up his christmas eve for
that! It is wholly possible that this is part of the bug that effected
the front end and is now resolved there and that the issue you are
reporting for 10 years is indeed resolved. Time will tell.

However Ive lost my chain of thought here...

The SiteGround platform (and platforms that cache in general) are also
another good point to think about here. For example, ON YOUR DEMO SITE
it is IMPOSSIBLE to login if you (starting from a clean slate browser,
no cookies, no active login session) go direct to
http://durian808.demojoomla.com/index.php/login before any other page,
you will see that no session cookie gets written, and so attempt to
login, you will find you get generic INVALID_TOKEN - this is
reproducible EVERY TIME ON YOUR DEMO SITE and is CAUSED by SiteGround
caching the login page, which has a session based token in the source
code! This further highlights the fact that not everything is under
the PHPApp control when it comes to INPUT and that an invalid input
will result in an unexpected output - this is a GREAT example of how a
WEB HOST has destroyed a core, working, bug free feature of Joomla!
But users will complain that this is a bug in J oomla wh en it is not!

|HTTP/1.1 200 OK Server: nginx/1.7.9 Date: Sat, 30 Jan 2016 07:02:15
GMT Content-Type: text/html; charset=utf-8 Connection: keep-alive
Vary: Accept-Encoding X-Cache-Enabled: True Expires: Mon, 1 Jan 2001
00:00:00 GMT Cache-Control: no-store, no-cache, must-revalidate,
post-check=0, pre-check=0 Pragma: no-cache Last-Modified: Sat, 30 Jan
2016 06:52:24 GMT Host-Header: 192fc2e7e50945beb8231a492d6a8024
X-Proxy-Cache: HIT |

However if you go to the same page with a query string
http://durian808.demojoomla.com/index.php/login?Asd

Then you get

|HTTP/1.1 200 OK Server: nginx/1.7.9 Date: Sat, 30 Jan 2016 07:02:48
GMT Content-Type: text/html; charset=utf-8 Connection: keep-alive
Vary: Accept-Encoding X-Cache-Enabled: True Expires: Mon, 1 Jan 2001
00:00:00 GMT Cache-Control: no-store, no-cache, must-revalidate,
post-check=0, pre-check=0 Pragma: no-cache Set-Cookie:
ecc01c8447c94640ccd59a7d1aa40801=nvjmpf0si9qhh65i4stljae7c0; path=/;
HttpOnly Last-Modified: Sat, 30 Jan 2016 07:02:48 GMT Host-Header:
192fc2e7e50945beb8231a492d6a8024 X-Proxy-Cache: MISS |

and you can login.

I had a whole paragraph here about session management when opening
several tabs to the site, but I just deleted it by mistake! as Ive now
had no sleep all night! However the same is true of tabs, if you
logout in one tab, this effects the sessions in other tabs to the same
site... This can cause unexpected concequenses as well.

So my point is... that you might like to mark this out as a 10 year
Joomla bug, when infact there is a lot more at play - the inputs to
Joomla are uncontrolable when it comes to session data, race
conditions and even recent session data migrations while users are
logged in, can all cause session data to be different than what Joomla
expects and Garbage in Garbage out, the GENERIC error message shown is
appropriate as the information contained in the request, merged with
the session data, is not acceptable and what is needed to be able to
access that resource.

As we have seen there are many ways this session data can be
changed/corrupted/deleted which are totally out of Joomla's control.
And other times, like Joomla 3.4.6/7/8 upgrade path where Joomla
specifically changed the structure of the session...

We have not even began to explore the impact that custom extensions
can have on the session... including those that manipulate $_SESSION
object directly without using the Joomla API.. We have not explored
the myriad of settings there are in php.ini or in individual session
handlers that can inpact session...

This "issue" is far greater than blaming Joomla for not addressing a
10 year old "bug" that is unreproduceable - a bug that is 100%
un-reproidueable doesnt exist. On the too few occassions you get your
GENERIC ERROR MESSAGE it instantly goes away on the next page load, or
session creation, and the people reporting the "bug" dont have enough
information about the INPUTS to the PHPApp in order to make educated
decisions, or accurate bug reports... For example, you have not even
told us which session handler you use, the session configuration
options chosen in Joomla Global Configuration, your full PHP version
and configuration settings for each site that you can replicate this on.

In summary - for a given set of INPUTS Joomla will result in a given
OUTPUT. You dont have control over all the inputs, or the knowledge or
experience to capture them at the point the "bug" occurs for you -
which is understandable and acceptable - but calling this a 10 year
Joomla bug is simply not accurate in the slightest.

Im not pissing on anyone, but I have - repeatedly - looked into this
over many many years and understand the issues better than most,
certainly a hell of a lot more than the average punter who says "it
doesnt work - its joomla's fault!"


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar N6REJ
N6REJ - comment - 31 Jan 2016

I'm sorry durian but thats the stupidest error message I've ever seen...
that would instantly tell me joomla is poorly designed and i should go
elsewhere!
Bear

On 1/31/2016 02:24, durian808 wrote:

@ggppdk https://github.com/ggppdk occasional random meaningless
message by prominent CMS to end user is not good, maybe a little ok
because not critical error, but really not cool to poor end user if
there is reasonable easy way to fix by smart developer. Suggested new
default language string: "Sorry, Joomla was not designed to respond
100% properly to what you just attempted. Please try again in a little
while."

@PhilETaylor https://github.com/PhilETaylor that's awesome... please
humor me. I need some humor. If it's by design, then may I suggest the
design be enhanced.


_This comment was created with the J!Tracker Application
https://github.com/joomla/jissues at
issues.joomla.org/joomla-cms/9013
https://issues.joomla.org/tracker/joomla-cms/9013.


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar N6REJ
N6REJ - comment - 31 Jan 2016

yeah, i've not seen it lately that I can think of. The "debate"
definitely woke me up this morning :P
Bear

On 1/31/2016 05:40, Phil Taylor wrote:

@N6REJ https://github.com/N6REJ What you are describing is a session
expiration, followed by a login which attempts to redirect you back to
where you wanted to go, but cannot because you wanted to go direct to
a form. What you are seeing is exactly what should happen. There were
excellent changes made to the admin session expiration redirection by
@tampe125 https://github.com/tampe125 (he will hate me tagging him
again haha) in Joomla 3.4.8 which will effect and fix this for the
backend


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar mbabker
mbabker - comment - 31 Jan 2016

I say if you are having these kinds of sessions issues - repeatably - then you should change web hosts.

When I've run into the issue, it's mainly been on joomla.org properties, not on a local environment I could do any sort of debugging against. Are you saying the Joomla project needs to change hosts?

avatar PhilETaylor
PhilETaylor - comment - 31 Jan 2016

If the root problem is that your sessions are not being written fast enough - then I would say yes, get a decent webhost.

But who listens to me anyway...

avatar durian808
durian808 - comment - 31 Jan 2016

@PhilETaylor said
"No - it is your suggestion for a change. Many will disagree with your concept."

Many will agree with the fact that an edit session should not be allowed to be opened twice, simultaneously, by the same user. Try it in your favorite operating system... what happens? So, what I'm talking about, and many (most) will agree, is not a "concept". Just because the Joomla environment is web-based doesn't imply that this long-held, logical way of dealing with editing sessions needs to be abandoned and ignored.

"For the record, I told you what the problem was here #9013 (comment) but you went on to shout about all kinds of things, and personal attacks..."

No sir, for the record you didn't explain the root cause, which finally came out in the discussion when I asked this question: "Why does Joomla need to allow two editing sessions to be open simultaneously for the same user?" To which you replied, essentially, that it's because it's the way we've always done it. You, sir, are the one who stayed up all night and almost drove your car off the road rambling about "all kinds of things," when you could have simply stated the obvious, which I eventually helped you remember. As for personal attacks, you were the king right from the get go. For the record.

"I suggest you step away now. If you would like to request major changes and present those to the Production Leadership Team and/or have these added to the Development Strategy and Roadmap then this issue is not the place to do that."

For other more polite people who are listening to this, how do I "present those to the Production Leadership Team and/or have these added to the Development Strategy and Roadmap"? (I would actually like to see someone other than @PhilETaylor answer this.)

"There is a PR ready for testing that may or may not assist in SOME of the error messages caused by race conditions."

Why not just design-out the race condition(s)?! Never mind answering that.

Lastly, @PhilETaylor says, "get a decent webhost". To me, it is laughable to suggest this. Imagine the system requirements of Joomla stating that it must be installed on a "decent webhost" (define decent) otherwise certain race conditions may arise that may frustrate end users (and webmasters) with random and rare cryptic error messages. In the real world, all kinds of webhosts are being used, and further, you never know what situation may arise on a web host – even a decent one.

Please tell me how to register a formal request for this simple, "major change." I hereby give my consent that this tracker item (which I opened) be closed. Thank you.

avatar durian808
durian808 - comment - 31 Jan 2016

Sorry to be a party pooper. I have just recreated the problem again, very easily. I just retraced my steps and created a fresh SiteGround demo site. On the FIRST click to edit the top blog article, I added 3 characters to the content and then clicked save. I waited about 3 SECONDS and pressed the edit icon again. The copy of the content in the editor window is missing the three characters. I believe this is when session data becomes corrupted... soon after in my test, I get the 403 error. I am able to replicate it more than once in this session that I have open right now. I have waited as long as 5 seconds and the error still appears.

I will now make a screen movie.

avatar durian808
durian808 - comment - 31 Jan 2016

More info... logging out and back in on the front end does not clear this condition. I am waiting a least 5 seconds... up to 10 seconds... failure still happens. Will do more testing later... gotta step out.

avatar gwsdesk
gwsdesk - comment - 1 Feb 2016

This issue has been closed, Brian pointed to the PR so why keeping on posting which is useless?

I suggest to lock this topic and move on?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 1 Feb 2016

Should I post the replication information to the PR?

avatar PhilETaylor
PhilETaylor - comment - 1 Feb 2016

No - you should not - Im still unable to replicate this issue on any siteground server...

You should be testing the PR. Not testing a site without the code changes applied to it.

avatar gwsdesk
gwsdesk - comment - 1 Feb 2016

No you should test the PR.

On 2/1/2016 2:14 PM, durian808 wrote:

Should I post the replication information to the PR?


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar gwsdesk
gwsdesk - comment - 1 Feb 2016

Ha never thought I would shake hands again with @PhilETaylor (joking Phil)


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/9013.

avatar durian808
durian808 - comment - 1 Feb 2016

How do I test the PR?

avatar durian808
durian808 - comment - 1 Feb 2016

By the way, I am still mystified by why my subsequent tests failed on my first SiteGround demo site, and then creating a new site, I got failures right away. Maybe there is something going on there w/ the initial state of the site.

avatar PhilETaylor
PhilETaylor - comment - 1 Feb 2016

I have never been able to replicate your problem. Ever. Over 4 days now, from 4 different internet connections and 3 different computers,

avatar durian808
durian808 - comment - 1 Feb 2016

So you tried creating a SiteGround site and set it up the same way I did, with the category blog?

avatar PhilETaylor
PhilETaylor - comment - 1 Feb 2016

Yes. Many Times!

avatar durian808
durian808 - comment - 1 Feb 2016

wow. any ideas why I could be possibly seeing this?

avatar gwsdesk
gwsdesk - comment - 1 Feb 2016

I can join the chorus here. I have not been able to reproduce this
either in any way described and we use our own server park. We are
wasting valuable time here. The "non-issue" is non relevant to anybody
but Submitter so can't we just move on here?

On 2/1/2016 2:19 PM, Phil Taylor wrote:

I have never been able to replicate your problem. Ever. Over 4 days
now, from 4 different internet connections and 3 different computers,


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar durian808
durian808 - comment - 1 Feb 2016

Did anyone watch my screen movie? It's relevant.

avatar durian808
durian808 - comment - 1 Feb 2016

If it fails for me, it HAS to fail for other people... under some conditions we presently don't understand.

avatar durian808
durian808 - comment - 1 Feb 2016

Staging code here, https://github.com/joomla/joomla-cms, do I just install that and try my test?

avatar PhilETaylor
PhilETaylor - comment - 1 Feb 2016

It adds nothing to this conversation. It shows you have a problem but nothing else. You excluded the urls again I see. One cannot debug a screenshot or video!

avatar PhilETaylor
PhilETaylor - comment - 1 Feb 2016

Im sorry - Im ending my involvement here.. jog on...

avatar durian808
durian808 - comment - 1 Feb 2016

I thought you guys were all about replicating problems? I replicated it. You are ignoring me.

avatar infograf768
infograf768 - comment - 1 Feb 2016

I tested http://durian808.demojoomla.com and I can't reproduce either.
I guess the issue is on your PC.

I strongly suggest to stop posting here and you test on another machine/OS.
My first and last post.

avatar durian808
durian808 - comment - 1 Feb 2016

Issue is on my PC?!!!

avatar durian808
durian808 - comment - 1 Feb 2016

Hey @PhilETaylor, I want to test your fix from my end... how do I do that? can you just give me a .ZIP file that I can download? I'll install on my server and replicate my SiteGround steps.

avatar brianteeman
brianteeman - comment - 1 Feb 2016

Standard practice on this tracker is that as soon as we have a PR an issue
is closed so that ALL further discussion takes place on the PR. Please
follow the links already provided and apply the PR and test. If it works
for you then report a successful test there. If it doesnt work for you then
report a failed test there. Commenting here is a waste of everyones time -
including yours.

If necessary this will be locked

On 1 February 2016 at 07:27, durian808 notifications@github.com wrote:

I thought you guys were all about replicating problems? I replicated it.
You are ignoring me.


Reply to this email directly or view it on GitHub
#9013 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar durian808
durian808 - comment - 1 Feb 2016

I need to be walked through it. I'm the only person who has recreated this... need some help.

avatar durian808
durian808 - comment - 1 Feb 2016

Switching to PR item now. Bye.

avatar gwsdesk
gwsdesk - comment - 1 Feb 2016

Can't you read? What is difficult here
https://docs.joomla.org/Testing_Joomla%21_patches?

@Brianteeman kindly request to lock this ?

On 2/1/2016 2:33 PM, durian808 wrote:

I need to be walked through it. I'm the only person who has recreated
this... need some help.


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar gwsdesk
gwsdesk - comment - 1 Feb 2016

"monitor PR"

On 2/1/2016 2:34 PM, durian808 wrote:

Switching to PR item now. Bye.


Reply to this email directly or view it on GitHub
#9013 (comment).

avatar durian808
durian808 - comment - 1 Feb 2016

@Brianteeman please lock this. I'm tired of the endless insults.

avatar brianteeman brianteeman - locked - 1 Feb 16
avatar brianteeman
brianteeman - comment - 1 Feb 2016

This issue has now ben locked

Add a Comment

Login with GitHub to post a comment