User tests: Successful: Unsuccessful:
https://technicalseo.com/tools/robots-txt/
Pull Request resolves # .
I was adding my SITEMAP to the ROBOTS txt on Joomla 6; noticed recommended tool is dead link. I found pretty good tool https://technicalseo.com/tools/robots-txt/
No testing necessary; its commented link inside Robots Txt
Robots txt comments
Robots txt comments
Please select:
Documentation link for guide.joomla.org:
[ / ] No documentation changes for guide.joomla.org needed
Pull Request link for manual.joomla.org:
[ / ] No documentation changes for manual.joomla.org needed
| Status | New | ⇒ | Pending |
Where is the deadlink you are referring to as I don't see this pr removing anything. At a guess you are referring to a link already removed a very long time ago
On my version of Joomla I downloaded and Installed v6.0.2 (2026-02-08); it had following robots txt; the link http://tool.motoricerca.info/robots-checker.phtml is dead
# If the Joomla site is installed within a folder such as at # e.g. www.example.com/joomla/ the robots.txt file MUST be # moved to the site root at e.g. www.example.com/robots.txt # AND the joomla folder name MUST be prefixed to the disallowed# path, e.g. the Disallow rule for the /administrator/ folder # MUST be changed to read Disallow: /joomla/administrator/ # # For more information about the robots.txt standard, see: # http://www.robotstxt.org/orig.html # # For syntax checking, see: # http://tool.motoricerca.info/robots-checker.phtml User-agent: * Disallow: /administrator/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/
I see in the new version of Joomla v6.0.3; the link was removed, and nothing was put in its place.
# If the Joomla site is installed within a folder # eg www.example.com/joomla/ then the robots.txt file # MUST be moved to the site root # eg www.example.com/robots.txt # AND the joomla folder name MUST be prefixed to all of the # paths. # eg the Disallow rule for the /administrator/ folder MUST # be changed to read # Disallow: /joomla/administrator/ # # For more information about the robots.txt standard, see: # https://www.robotstxt.org/orig.html User-agent: * Disallow: /administrator/ Disallow: /api/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/
My pull is just suggestion for a replacement checker URL
On my version of Joomla I downloaded and Installed v6.2.0 (2026-02-08); it had following robots txt; the link http://tool.motoricerca.info/robots-checker.phtml is dead
# If the Joomla site is installed within a folder such as at
# e.g. www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://tool.motoricerca.info/robots-checker.phtml
```
User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
I see in the new version of Joomla v6.3.0; the link was removed, and nothing was put in its place.`
# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html
```
User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/
```
@killrawr Could there be some typos on the Joomla version numbers above?
You mention Joomla 6.3.0 but 6.2 isn't even out yet?
`On my version of Joomla I downloaded and Installed v6.2.0 (2026-02-08); it had following robots txt; the link http://tool.motoricerca.info/robots-checker.phtml is dead
If the Joomla site is installed within a folder such as at
e.g. www.example.com/joomla/ the robots.txt file MUST be
moved to the site root at e.g. www.example.com/robots.txt
AND the joomla folder name MUST be prefixed to the disallowed# path, e.g. the Disallow rule for the /administrator/ folder
MUST be changed to read Disallow: /joomla/administrator/
For more information about the robots.txt standard, see:
http://www.robotstxt.org/orig.html
For syntax checking, see:
http://tool.motoricerca.info/robots-checker.phtml
User-agent: * Disallow: /administrator/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/
I see in the new version of Joomla v6.3.0; the link was removed, and nothing was put in its place.`
If the Joomla site is installed within a folder
eg www.example.com/joomla/ then the robots.txt file
MUST be moved to the site root
eg www.example.com/robots.txt
AND the joomla folder name MUST be prefixed to all of the
paths.
eg the Disallow rule for the /administrator/ folder MUST
be changed to read
Disallow: /joomla/administrator/
For more information about the robots.txt standard, see:
https://www.robotstxt.org/orig.html
User-agent: * Disallow: /administrator/ Disallow: /api/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/ `
@killrawr Could there be some typos on the Joomla version numbers above?
You mention Joomla 6.3.0 but 6.2 isn't even out yet?
Correct I meant 6.0.2 and 6.0.3
Ohhhh I migrated from joomla 3 to joomla 6 and my robots hadn't changed
I dont see any need to add this checker and no guarantee that it won't disappear in the future. There are many options available and despite your statement there is nothing about this checker that is open source that I could find.
| Status | Pending | ⇒ | Closed |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2026-03-14 09:42:40 |
| Closed_By | ⇒ | killrawr | |
| Labels |
Added:
PR-5.4-dev
|
||
Here is direct URL to the tool https://technicalseo.com/tools/robots-txt/ I've tried it and it seems to work pretty well.