[#6361] - Optimization of the robots.txt for components's media files (JS, CSS ...)
- Closed
- 7 Mar 2015
- Low
- Build: master
- # 6361
Some components install their JS, CSS and images in their own /assets directory instead of using the media tags in the XML manifest.
It may sense to optimize the robots.txt to give the ability for GoogleBot to access theses files.
I suggest to modify the robots.txt with theses lines :
Replace the line :
Disallow: /components/
By theses 3 lines :
Allow: /components/*/assets/css/*.css$ Allow: /components/*/assets/js/*.js$ Disallow: /components/
This way, GoogleBot will be able to validate that Joomla sites are more 'Mobile Compatible'.
More infos in my article on Mobile site optimization (sorry, this is a french article so i give you the auto-translate url) :
https://translate.google.com/translate?hl=en&sl=fr&tl=en&u=http%3A%2F%2Fwww.garstud.com%2Fjoomla%2F12-joomla%2F25-optimisation-joomla-pour-la-compatibilite-mobile.html
We cant have rules for everything - we can only have rules for standard behaviour
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/6361.
In addition as Allow is not part of the standard for robots.txt I am closing this
(You can confirm this yourself using the robots.txt checker referred to in the file itself)
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/6361.
zero-24
- | Status | New | ⇒ | Closed |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2015-03-07 19:55:29 |
| Closed_By | ⇒ | brianteeman |
Ok, i understand and i agree to close this PR. Devs should used the media tag!
For the 'Allow' directive, you should notice that the most actives bots (google, bing, ...) support this non-standard directive, and that make sense to optimize the "mobile-fiendly" required by Google.
(see 'Allow' directive : http://www.garstud.com/images/stories/blog/joomla/google_compat_mobile4.png )
And I speak in my article about using many 'User-agent' to be sure to give correct directives to correct bots.
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/6361.
| Labels |
Added:
?
|
||
for me it is the components issue,
I would ask the component developers to move their /assets to /media
Joomla use this long time ago, and I wonder why they still use /assets
lazy people