Not index content that is linked to categories "uncategorised". This page is a duplicate.

How is this automatically duplicate content?

All these new lines are NOT robots.txt standard.

http://www.robotstxt.org/robotstxt.html
See remark from "Note also that globbing and regular expression are not supported in either the User-agent or Disallow lines...."

And I've never seen a plus sign in robots.txt standards. What is it good for?

@Kostelano Can you elaborate and answer the raised question?
To me it doesn't make much sense what you propose here.

• About the "uncategorised". Almost always the category "Uncategorized" is not renamed. Any articles that are tied to this category, and thus get a link from the menu (for example) have a double:

http://SITE.COM/content_1/
http://SITE.COM/content_1/2-uncategorised/

This almost always takes place.

• Sorry, copying accidentally added "+" symbol. Of course, it's not there! I'm sorry!

After listening to your opinions, I suggest to specify some strings so that it does not create problems with the menus (for example) - someone wrote above.
For example:

Disallow: /*?tmpl=component&print=*
Disallow: /*?start=*
Disallow: /*?limitstart=* // it duplicates the first page category with page navigation
Disallow: /*-uncategorised

Can be ignored "?start=" because the line is not a double. This page is just not interesting to users.

• Perhaps google and well takes two identical pages (for Bakual), but why do we? It is logical to have one such page is correct. With that, adding the exception string with /component/, we get rid of at least five more unnecessary pages in the index of search engines.

After all, even with the authorization module off the search engine spiders find links to the registration, login and password recovery, etc.

Thank you for your attention.

Perhaps google and well takes two identical pages (for Bakual), but why do we? It is logical to have one such page is correct. With that, adding the exception string with /component/, we get rid of at least five more unnecessary pages in the index of search engines.

Like I noted, your change with that line would effectively block search engines from not indexing ALL routes using the default Joomla SEF implementation (where routes are named /component/<component_name>/<view_name>) and I don't think that is a change that we should make to our default robots.txt file. For what you're suggesting to accomplish, I really think you need more specific rules that aren't going to cause potential issues with other valid routes.

As stated above by @bertmert (but ignored during further dicsussions), the globbing in the disallow lines is NOT robots.txt standard. It is a Google extension which is not standard and not supported by most other search engines.

Based on the overall feedback so far, I'm closing this PR.

Feel free to create a new one with the feedback included.