[#5109] - Language Switcher: Remove the language code for the default language
- Closed
- 17 Nov 2014
- Medium
- Build: staging
- # 5109
- Diff
- smz:LanguageModule
- Success The Travis CI build passed Details
User tests: Successful: Unsuccessful:
This will remove the language code for the default language when needed, i.e. when using sef URLs and when the "Remove URL Language Code" option is set in the language filter
jissues-bot
- | Labels |
Added:
?
|
||
How to test:
Have a multilingual site, with at least two languages and two menu hierarchies, one for each language. At least some of the different languages menu items have to be associated.
First try with "Remove URL Language Code" option switched off.
Navigate throughout the site and inspect the links generated by the language switcher. Change language back and forth.
Now do the same with the "Remove URL Language Code" option switched on.
What will go wrong:
Once in the home page (and probably in every page whose alias is the same for the two languages) and you switch out of the default language, you'll be caught in: no way to go back to the default language!
WTF, why do you publish a buggy PR??
IMHO it is not my PR that is buggy, but something else (I'm putting my bet on the router...). Want a demonstration? Turn off this PR and do the same: navigate out off the default language (e.g. English) home page (http://example.com) and into the corresponding one in a different language (e.g Italian, http://example.com/it/). Now try to navigate back to the English page by typing its canonical address (http://example.com) in the address bar of your browser: no way! You are caught in!
But the original language switcher does work...
Yes, because it always add the language code, even when you have said no to that in the language filter and this allows you to go back. This is, IMHO, a kludge
Oh, well, and is this a problem?
Yes, because spiders find also the bogus URL for the default language and index them as well. Result: duplicated content in Google, which is a bad-bad thing. This is exacerbated by the fact that redirects to the canonical URL are done with a 303 status code and not a 301
Is there more?
Yes, when switching back from another language to the default language, we always get at least a redirect (and I swear I have seen cases with two redirects!). This does not happen when switching from the default to another language. The server is hit at least twice and (for small pages) it takes twice the time to load. Beware: don't think only at users interacting with your site: think at the spiders as well...
This does not work when switching to the default language home page because a cookie is set.
how about if the home page is assosiated?
Association will not change anyhting. It is done looking at the lang prefix (and its associated lang_code).
If the lang prefix is absent then the lang cookie takes over.
@infograf768 have you actually read my comment, and particularly the "What will go wrong:" paragraph? I know that "This does not work when switching to the default language home page because a cookie is set" (actually, in the "How to test" section, I forgot to mention that you must clear your cookies before switching from a test to another...)
The question now is: why do we need a bloody language cookie? An URI is an URI and should always bring to a specific resource independently of any set cookie. I suspect (but I'm not sure) spiders do not accept cookies. This can bring to the situation where the site structure as perceived by a spider differs from the structure as perceived by a human being. This is bad for SEO.
I don't know exactly why we need that cookie: the code is complicated enough that I'm unable to understand it, but my feeling is that the cookie is needed for when we also have the Language Selection for new Visitors option set to Browser Settings. Am I on the right way?
The cookie is also used so that if a visitor sets a specific language they
will be back on that language when they revisit the site
On 15 November 2014 15:38, Sergio Manzi (smz) notifications@github.com
wrote:
@infograf768 https://github.com/infograf768 have you actually read my
comment, and particularly the "What will go wrong:" paragraph? I know
that "This does not work when switching to the default language home
page because a cookie is set" (actually, in the "How to test" section, I
forgot to mention that you must clear your cookies before switching
from a test to another...)The question now is: why do we need a bloody language cookie? An URI is
an URI and should always bring to a specific resource independently of any
set cookie. I suspect (but I'm not sure) spiders do not accept cookies.
This can bring to the situation where the site structure as perceived by
a spider differs from the structure as perceived by a human being.
This is bad for SEO.I don't know exactly why we need that cookie: the code is complicated
enough that I'm unable to understand it, but my feeling is that the
cookie is needed for when we also have the Language Selection for new
Visitors option set to Browser Settings. Am I on the right way?—
Reply to this email directly or view it on GitHub
#5109 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
Fair enough, Brian: that kind of usage is acceptable, but this (if I'm not mistaken) is needed only if we have resources (URI) that responds differently depending on the set language. Is that correct?
What I mean is: if we had an option saying "Remove language code from every language, be it the default one or any other", then we surely need a cookie to differentiate. But (happily enough) we don't have that. The edge case is only the home page, I guess.
Think of this:
- I have two associated menu items with alias "test-page" (en) and "pagina-di-prova" (it).
- My default language is "en"
- I have the "Remove URL Language Code" option switched on
- in the following assume that when i say "go" I mean "type the URL in the address bar and hit Enter"
- I go to http://example.com/test-page - I see the English page
- I go to http://example.com/it/pagina-di-prova - I see the Italian page
- I go (again) to http://example.com/test-page - I'm stuck into the Italian page!
Guys, this is sick!
My 2c here: I agree with Sergio, that the only point of truth should ALWAYS be the url. Now about the preselected (or prefered if you like) language there should be some check (onAfterInitialize I guess) for the cookie and redirect accordingly. Only thing that needs to be mandatory, otherwise this won’t work, is to stet up properly the associations.
Can somebody please tell me where the cookie is set and where is it tested? I would like to give a look at the code...
I also agree with @smanzi. There is no actual need for language cookie, the url should be the answer.
@smanzi about the problem you have with same alias. How is it possible to have the same alias for associated items? Joomla doesn't allow me to create same alias, and for example in the Greek site i use /app/ and in English /apps/.
ABOUT SEO: I am not sure if this bug is related to this one. But google has index about 8 times, article urls with wrong language code. So it gave me at Webmaster Tools a 404 Error for this urls.
My Bad, sorry, was confused by categories.
@ManuxGR Ah, yes, that's correct, although I would say it would be nice if had to have the possibility to have different categories (and in general any "kind of item" at the same level) with the same alias provided they are associated to different languages. But this is another story...
@ManuxGR this is OT, but anyway... your problem (and probably many others) can be solved by creating two top level categories ("en" and "gr") and then have every category for a particular language created as a sub-category of the "language category" and associated to the corresponding language.
Maybe if we treat language associations just as "above-the-top-level" categories we can solve a lot of issues...
You know? "Maybe if we treat language associations just as "above-the-top-level" categories we can solve a lot of issues..." The more I think to this the more I feel I've just had a damn good bright idea. Myabe not for 3.4, but...
Prove me wrong and I'll eat quite a lot of "Humble pie"...
Ah, to be clear, in the above when I talk about association I mean the association of an item to a particular language or "All" languages. I don't mean the association between two different items belonging to different languages.
In this case the "All" language should be seen just as another language, with the restriction that its aliases cannot appear in any other language (otherwise we will have a routing nightmare...)
If you look at the multilingual does that @infograf768 woe you will see
that I exactly what he suggests you do.
On 15 Nov 2014 17:23, "Sergio Manzi (smz)" notifications@github.com wrote:
In this case the "All" language should be seen just as another language,
with the restriction that its aliases cannot appear in any other
language (otherwise we will have a routing nightmare...)—
Reply to this email directly or view it on GitHub
#5109 (comment).
@brianteeman Sorry, Brian, I'm not getting it... can you please rephrase/elaborate?
haha, don't know about that bro (categories). Yeah you are right about top level categories but to late now for me to apply. i have already figure out the solution for me at least.
See
http://multilingual.demojoomla.com/multi-lingual-steps-by-steps/17-site-contents-structure.html
On 15 November 2014 18:42, ManuxGR notifications@github.com wrote:
haha, don't know about that bro (categories). Yeah you are right about top
level categories but to late now for me to apply. i have already figure out
the solution for me at least.—
Reply to this email directly or view it on GitHub
#5109 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
@brianteeman @smanzi Thanks a lot once again. I applied the changes you advise me about categories. Now they look a lot more organised. :)
Always helps to Read The Fine Manuals before deciding something is broken
On 15 November 2014 20:50, ManuxGR notifications@github.com wrote:
@brianteeman https://github.com/brianteeman @smanzi
https://github.com/smanzi Thanks a lot once again. I applied the
changes you advise me about categories. Now they look a lot more organised.
:)—
Reply to this email directly or view it on GitHub
#5109 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
@brianteeman How can we access the http://multilingual.demojoomla.com/ site backend? There are links to it in the article you pointed to but I can't see where the username/password are documented...
With all due and sincere respect and not willing to be an asshole, but do you think that documenting a bad/convoluted architecture does it make it any better?
https://demo.joomla.org/multilingual
Seriously dude - spend a few minutes thinking and reading before posting
and wasting time. I wont be replying any more
On 15 November 2014 21:30, Sergio Manzi (smz) notifications@github.com
wrote:
@brianteeman https://github.com/brianteeman How can we access the
http://multilingual.demojoomla.com/ site backend? There are links to it
in the article you pointed to but I can't see where the username/password
are documented...With all due and sincere respect and not willing to be an asshole, but do
you think that documenting a bad/convoluted architecture does it make it
any better?—
Reply to this email directly or view it on GitHub
#5109 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
Thanks, that a nice link!
I am going to try to convince you why the URL should be the answer when redirecting and explain the bug from my experience with my live site.
Create a multilanguage site. (in this example Greek as default and English)
Enable "remove language code from url"
We have external backlinks from a Greek article.
Users trying to click the backlink url which does not contain the language code.
They get 404 error page because they have english cookie OR because they were using the english version of my site before.
((yes it can happen some of my english articles are global targeting and even Greece))
And the language code is /en/ instead of /el/.
This is where the "Language Selection for new Visitors" to "site language" comes to fix this problem BUT only for the new visitors. The bug remains for the rest.
Now imagine what will happen if we finally replace the 303 with 301... if we don't fix this. And 301 is the correct.
Also this could be a security whole for negative seo by bad competitors. (i am not sure about this)
Thanks for reading my post and even if this is not fixed i am happy with !joomla! :)
@ManuxGR, I am already convinced... 
Now, I'm playing with languagefilter.php to see how and where that cookie is set/used and you might be interested to follow me in my line of thought and do some test yourself, but please note, this is just tinkering with the code, so:
Don't do this in a production environment!
- Try commenting out lines 118-122, the ones under the "Create a cookie" comment.
- Set the "Remove URL Language Code" option switched on
- Install PR #5110 (not necessary but will help moving around with canonical URLs)
- Erase your browser cookies
- Play around and see how it goes...
I'm perfectly aware this is not a correct solution and will have side effects, first of all with users logged into the front-end. This is just a quick-and dirty proof of concept to see if removing the cookie has some benefit.
The real solution I think (at this time) would be to use the cookie just for the home page.
I'm unsure what will happen to logged in users and which negative side effects this could have.
@ManuxGR If you wish, give this a try (again not in a production environment!) and tell me if you discover anything that will be broken by that...
Thanks!
Sergio
zero-24
- | Category | ⇒ | Modules Multilanguage |
@brianteeman OK, I missed the obvious location of the usernames/password for the multilingual demo site: I'm sorry, really sorry about that and having wasted your time. But was this enough for publicly chastise me?
Or was that for the suggestion of creating the language-associated top level categories? Yes, I admit, I didn't read the "Multi-lingual Tutorial" before, but eventually I came to the same conclusion and solution... "Great minds think alike!" 
Whatever is the case, I apologize, I didn't mean to offend in any way.
Anyway I fail to see how the above affects the issues we are now dealing here. Or do you really think that there is no issue and everything is OK with Joomla Multilingual and we just didn't read the manual and this is why we are having issues?
I think there are issues and I'm here trying to find solutions: maybe I don't have your experience (after all you are one of the founding fathers...), but the simple fact that I'm here struggling to understand the code (btw, no one has given an advice about where to look...), well, I think this should be appreciated and not dismissed with an attitude that really seems to say "You kids, go play in your room and let the grown-up do their things!"
Sincerly,
Sergio
I have re-purposed #5110 as an experimental version of the Language Switcher to get rid of the language cookie: try testing this PR together with it.
I've passed a night awake thinking, coding and testing, but it seems to me that the thing is working.
I've tested only with SEF URLs and with Apache mod_rewrite enabled, so there is a lot still to be checked, but I find it promising...
Switching language in the home page now works and It seems to be the only instance generating a redirect.
#5109 #5110 is really crude: I just commented out the storage of the language cookie. If everything will work, I'll then clean it up.
Front end login, even when users have an associated default language doesn't seems to create issues, at least when the redirect page is the home page. Must be checked thoroughly in other situations...
... and, most important, every URI is now reachable from everywhere: no more "locked-in syndrome"!
See #5110 (comment)
Enter"
I go to http://example.com/test-page - I see the English page
I go to http://example.com/it/pagina-di-prova - I see the Italian page
I go (again) to http://example.com/test-page - I'm stuck into the Italian page!
This is wrong! We have solved that kind of issue for ages.
It is very easy to test: enter the link "http://example.com/test-page" in an article in another site.
Then on the multilingual test site, display "http://example.com/it/pagina-di-prova".
Whether you enter in the same browser window "http://example.com/test-page" or use the link from the other site in the same browser, you will get to "http://example.com/test-page"
Folks, we "could" implement my code ( #4911 (comment) , and NOT #5110 which is wrong) to get rid of the cookie when set such in the plugin.
Then if this condition is set, sef on, url language code off for the site default language, this PR here to the switcher implemented adding the new conditional for the cookie, a user will always be redirected to the default site language OR the Browser language (depending on the parameter for new visitors) when going to the bare url http://mysite.com.
Most of the time this is NOT the desired purpose of a multilanguage site. Thus the reason of the lang cookie.
At least, using adding my code in the filter will let people decide what they want and evidently lang cookie default should NOT be set to None.
Therefore, once again, the matter is a decision for 301 or 303 for most joomla multilingual users
Enter" I go to http://example.com/test-page - I see the English page I go to http://example.com/it/pagina-di-prova - I see the Italian page I go (again) to http://example.com/test-page - I'm stuck into the Italian page!This is wrong! We have solved that kind of issue for ages.
It is very easy to test: enter the link "http://example.com/test-page" in an article in another site.
Then on the multilingual test site, display "http://example.com/it/pagina-di-prova".Whether you enter in the same browser window "http://example.com/test-page" or use the link from the other site in the same browser, you will get to "http://example.com/test-page"
Apparently you are totally correct... I'm puzzled... I swear I've seen this behavior while testing... I don't know what to say... sorry! 
I just pushed a new commit because the test I introduced to check if the language_filter plugin was enabled was always returning false and thus breaking everything. Can you please give a look and tell me what was wrong with it? Also, is it true that if we get an object from JPluginHelper::getPlugin() that means not only that the plugin exists but also that it is enabled?
I'm now going to test this PR with #4911. Should I incorporate your proposed diff #4911 (comment), or use it "as it is"?
TBH, I think this is not the right solution.
So far this PR seems to work nicely with #4911
I'm going to close #5110, which (as I always stated) was just an opprobrium to support testing this one.
Now, for this PR:
How to test (amended):
- Have a multilingual site, with at least two languages and two menu hierarchies, one for each language. At least some of the different languages menu items have to be associated.
- Install
#4911#5126 (important!) - First try with the "Remove URL Language Code" option (in the Language Switcher) switched off.
- Navigate throughout the site and inspect the links generated by the language switcher. Change language back and forth and confirm that everything works as expected. The links generated by the language switcher should have the language code also for the default language.
- Now switch on the "Remove URL Language Code" option in the Language Switcher and set the "Cookie Lifetime" to "none" in the "Language Filter" plugin
- Just to play it safe, erase all your browser cookies (it shouldn't be necessary, but this is something yet to be checked)
- Again, navigate throughout the site and inspect the links generated by the language switcher. Change language back and forth and confirm that everything works as expected. The links generated by the language switcher for the default language should not have the language code, except for the "home page"
TBH, I think this is not the right solution.
Can you elaborate the reasons why?
What do you think will be the right one?
The only problem I see is that if the "Remove URL Language Code" option is switched on, then the Cookie Lifetime" in the "Language Filter" must be set (or forced to) "none"
Ah, OK, I see!
Point 1: Do you want to transform your proposed code in a PR so we can test it? Do you want me to include it into this one?
Point 2: As I said above, yes, you're right. I've also doubt about the possibility of setting the cookie lifetime to none without setting the "Remove URL Language Code" option. Will it work?
Do you have an advice for me about the testing of the language_filter plugin being enabled? (see my comment above, just after the last commit)
Thanks!
Sergio
I was thinking of this too:
As with this PR we now have correct canonical links for every URI and every URI either has a language code explicitly set or it is implicitly associated to the default language, wouldn't it possible (and better) to leave the language cookie as it is now but just check it only for the default home pages?
@infograf768 Thanks! 
Do you have an advice for me about the testing of the language_filter plugin being enabled? (see my comment above, just after the last commit)
if (JLanguageMultilang::isEnabled()) is already used line 99
OK! I'll move the removal of the default language code inside that if() {...}: where it is now it is also performed when unneeded. No harm as the regexp will not find it, but...
@infograf768 It seems to be alright... 
@infograf768
2 more points:
If to use 303 or 301 for redirects is something yet to be decided. With this and your PRs and the system set-up to remove default language code and use no cookie, I think we shouldn't have many (just for the
home pagesdefault language home page if I'm not mistaken...) and they can be 303, but in all other cases my personal opinion is that a 301 is better...To correctly set-up the system and achieving what was the original target, one must set-up two options. I'm wondering if forcing the "no cookie" when the "no default language code" is set would be a better option (more intuitive and will not create unexpected results), but if you think it is best to leave it as it is for B/C I will not object (too much!
)
Wait... there are issues... 
This does not works for logged-in users.
I think a language cookie is created at login time and is screwing things up.
Either we eliminate that too (subjected to the option in language filter), or we change the logic in language_filter.php, so that it ignores the language cookie except for the "naked" home page (this, IMHO and at first thought, seems a better and more robust solution). It will also allows returning users to be back at the language of their choice.
What do you think, @infograf768 ?
@infograf768
To be more specific, the issue happens when the language defined for a user is not the default language (browser language doesn't have any effect). In this case all URI for the default language (those without the language code) give a 404
I've seen that the "language" cookie is set only in languagefilter.php and never if lang_cookie == 2, so... I really don't understand! I've spent hours looking at languagefilter.php but I can't figure out what's wrong with it... 
And forget about changing the switcher and cookie. :)
OK, but:
- Is this going to have a SEO impact?
- Is this going to create problems to sitemap generators?
- Will this litter my Webmaster Tools?
I've asked on a Google SEO group about the situation (not citing J!...). I'll let you know if I get any answer...
I've think it over again (a good shower does magic in cases like this! ) and came to the conclusion that this PR is just a kludge and thus I'm retracting it.
Probably the right place where to fix this all is where the route is built and parsed and not trying to fake anything in something else. Unhappily this is something that at this time goes beyond my capabilities.
Partially unrelated, but I still think that having the possibility of not storing the 'language' cookie is an option we should offer.
Sorry to everybody if I wasted your time, but after all I think something good can come out of this (I mean discussing this, not the PR which is a real kludge!)
| Status | Pending | ⇒ | Closed |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2014-11-17 12:21:17 |
Sorry: there are some "spurious" commits above this one as I did some mistake in my local repo, but it seems to me that, having reverted the spurious ones, everything should be OK here.
Is there a way to get rid of that garbage I involuntary created? Thanks to whomever will help...