PR-5.4-dev Pending

User tests: Successful: Unsuccessful:

avatar killrawr
killrawr
13 Mar 2026

https://technicalseo.com/tools/robots-txt/

Pull Request resolves # .

  • [ / ] I read the Generative AI policy and my contribution is either not created with the help of AI or is compatible with the policy and GNU/GPL 2 or later.

Summary of Changes

I was adding my SITEMAP to the ROBOTS txt on Joomla 6; noticed recommended tool is dead link. I found pretty good tool https://technicalseo.com/tools/robots-txt/

Testing Instructions

No testing necessary; its commented link inside Robots Txt

Actual result BEFORE applying this Pull Request

Robots txt comments

Expected result AFTER applying this Pull Request

Robots txt comments

Link to documentations

Please select:

  • Documentation link for guide.joomla.org:

  • [ / ] No documentation changes for guide.joomla.org needed

  • Pull Request link for manual.joomla.org:

  • [ / ] No documentation changes for manual.joomla.org needed

avatar killrawr killrawr - open - 13 Mar 2026
avatar killrawr killrawr - change - 13 Mar 2026
Status New Pending
avatar killrawr
killrawr - comment - 13 Mar 2026

Here is direct URL to the tool https://technicalseo.com/tools/robots-txt/ I've tried it and it seems to work pretty well.

avatar brianteeman
brianteeman - comment - 13 Mar 2026

Where is the deadlink you are referring to as I don't see this pr removing anything. At a guess you are referring to a link already removed a very long time ago

avatar killrawr
killrawr - comment - 14 Mar 2026

On my version of Joomla I downloaded and Installed v6.0.2 (2026-02-08); it had following robots txt; the link http://tool.motoricerca.info/robots-checker.phtml is dead

# If the Joomla site is installed within a folder such as at
# e.g. www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://tool.motoricerca.info/robots-checker.phtml

User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

I see in the new version of Joomla v6.0.3; the link was removed, and nothing was put in its place.

# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html

User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

My pull is just suggestion for a replacement checker URL

avatar exlemor
exlemor - comment - 14 Mar 2026
On my version of Joomla I downloaded and Installed v6.2.0 (2026-02-08); it had following robots txt; the link http://tool.motoricerca.info/robots-checker.phtml is dead

# If the Joomla site is installed within a folder such as at
# e.g. www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://tool.motoricerca.info/robots-checker.phtml

```
User-agent: *
Disallow: /administrator/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

I see in the new version of Joomla v6.3.0; the link was removed, and nothing was put in its place.`

# If the Joomla site is installed within a folder
# eg www.example.com/joomla/ then the robots.txt file
# MUST be moved to the site root
# eg www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to all of the
# paths.
# eg the Disallow rule for the /administrator/ folder MUST
# be changed to read
# Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# https://www.robotstxt.org/orig.html

```
User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

```

@killrawr Could there be some typos on the Joomla version numbers above? 

You mention Joomla 6.3.0 but 6.2 isn't even out yet?
avatar killrawr
killrawr - comment - 14 Mar 2026

`On my version of Joomla I downloaded and Installed v6.2.0 (2026-02-08); it had following robots txt; the link http://tool.motoricerca.info/robots-checker.phtml is dead

If the Joomla site is installed within a folder such as at

e.g. www.example.com/joomla/ the robots.txt file MUST be

moved to the site root at e.g. www.example.com/robots.txt

AND the joomla folder name MUST be prefixed to the disallowed# path, e.g. the Disallow rule for the /administrator/ folder

MUST be changed to read Disallow: /joomla/administrator/

For more information about the robots.txt standard, see:

http://www.robotstxt.org/orig.html

For syntax checking, see:

http://tool.motoricerca.info/robots-checker.phtml

User-agent: * Disallow: /administrator/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/

I see in the new version of Joomla v6.3.0; the link was removed, and nothing was put in its place.`

If the Joomla site is installed within a folder

eg www.example.com/joomla/ then the robots.txt file

MUST be moved to the site root

eg www.example.com/robots.txt

AND the joomla folder name MUST be prefixed to all of the

paths.

eg the Disallow rule for the /administrator/ folder MUST

be changed to read

Disallow: /joomla/administrator/

For more information about the robots.txt standard, see:

https://www.robotstxt.org/orig.html

User-agent: * Disallow: /administrator/ Disallow: /api/ Disallow: /bin/ Disallow: /cache/ Disallow: /cli/ Disallow: /components/ Disallow: /includes/ Disallow: /installation/ Disallow: /language/ Disallow: /layouts/ Disallow: /libraries/ Disallow: /logs/ Disallow: /modules/ Disallow: /plugins/ Disallow: /tmp/ `

@killrawr Could there be some typos on the Joomla version numbers above?

You mention Joomla 6.3.0 but 6.2 isn't even out yet?

Correct I meant 6.0.2 and 6.0.3

avatar brianteeman
brianteeman - comment - 14 Mar 2026

That link was removed from Joomla in 2020 with Joomla 3.9.23 #31327

avatar killrawr
killrawr - comment - 14 Mar 2026

Ohhhh I migrated from joomla 3 to joomla 6 and my robots hadn't changed

avatar brianteeman
brianteeman - comment - 14 Mar 2026

I dont see any need to add this checker and no guarantee that it won't disappear in the future. There are many options available and despite your statement there is nothing about this checker that is open source that I could find.

avatar killrawr killrawr - change - 14 Mar 2026
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2026-03-14 09:42:40
Closed_By killrawr
Labels Added: PR-5.4-dev
avatar killrawr killrawr - close - 14 Mar 2026

Add a Comment

Login with GitHub to post a comment