[#22368] - Add more search engine bots to robots list
- Fixed in Code Base
- 1 Nov 2018
- Medium
- Build: staging
- # 22368
- Diff
- jurihahn:add-bingbot-to-robots-list
User tests: Successful: Unsuccessful:
Pull Request for Issue # .
Summary of Changes
Add more robots to list for detection. List generated from:
https://github.com/monperrus/crawler-user-agents
Testing Instructions
Code review
Expected result
more robots in the robots list
Actual result
small robots list
Documentation Changes Required
No
| Status | New | ⇒ | Pending |
joomla-cms-bot
- | Category | ⇒ | Libraries |
There must be a more up to date list available somewhere that can be used - for example I know that majestic-12 is one of the top (if not the top) crawlers
Should i completely update the list with one PR?
If you can find a good list then it would make sense to do it all in one pr
i think this is good list http://www.useragentstring.com/pages/useragentstring.php?typ=Crawler
| Title |
|
||||||
I note that that list doesn't include bingpreview which is in this pr
Found this list from keycdn of the most popular - lets just make sure we have those
https://www.keycdn.com/blog/web-crawlers/
| Labels |
Added:
?
|
||
This is a decent resource https://github.com/monperrus/crawler-user-agents. If not overkill.
| Title |
|
||||||
i think it's ready to test/review
This is a decent resource https://github.com/monperrus/crawler-user-agents. If not overkill.
i generated robot list from it
@SharkyKZ @brianteeman it's now OK?
seems ok to me
I have tested this item
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/22368.
libraries/src/User/UserHelper.php, Line 760: Should be ->where($db->quoteName('user_id') . ' = ' . (int) $userId);
@jurihahn see, [#22079] - typecasting code privacy content plugin
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/22368.
This PR does not touch at this file.
Thanks @infograf768 and @jurihahn.
I got it.
There are more files in the Joomla core with this issue.
So it should be a new PR.
[EDIT] Some text.
I have tested this item
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/22368.
| Status | Pending | ⇒ | Ready to Commit |
RTC
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/22368.
mbabker
- | Status | Ready to Commit | ⇒ | Fixed in Code Base |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2018-11-01 02:33:07 |
| Closed_By | ⇒ | mbabker | |
| Labels |
Added:
?
|
||
There must be a more up to date list available somewhere that can be used - for example I know that majestic-12 is one of the top (if not the top) crawlers