René's URL Explorer Experiment


Title: ​robots.txt report - Search Console Help

Description: See whether Google can process your robots.txt filesThe robots.txt report shows which robots.txt files Google found for the top 20 hosts on your site, the last time they were crawled, and any warnings

direct link

Domain: support.google.com

format-detectionemail=no
None4
google-site-verificationnk1dShrJgCQtxaaJ6DHv707PkUgq-EuBsYjHWcaawmw

Links:

Skip to main contenthttps://support.google.com/webmasters/answer/6062598#search-form
https://support.google.com/?tab=uu
Search Console Helphttps://support.google.com/webmasters
https://www.google.com/intl/en/about/products?tab=uh
Sign inhttps://accounts.google.com/ServiceLogin?hl=en&passive=true&continue=http://support.google.com/webmasters/answer/6062598&ec=GAZAdQ
https://support.google.com/?tab=uu
Google Helphttps://support.google.com/
Help Centerhttps://support.google.com/webmasters/?hl=en
Communityhttps://support.google.com/webmasters/community?hl=en&help_center_link=CIaE8gISFOKAi3JvYm90cy50eHQgcmVwb3J0
Search Consolehttps://search.google.com/search-console
Privacy Policyhttps://www.google.com/intl/en/privacy.html
Terms of Servicehttps://www.google.com/accounts/TOS
Help Centerhttps://support.google.com/webmasters/?hl=en
Communityhttps://support.google.com/webmasters/community?hl=en&help_center_link=CIaE8gISFOKAi3JvYm90cy50eHQgcmVwb3J0
Search Consolehttps://search.google.com/search-console
Managing properties and users on Search Consolehttps://support.google.com/webmasters/topic/9455938?hl=en&ref_topic=4558844
Property and user settingshttps://support.google.com/webmasters/topic/9456376?hl=en&ref_topic=9455938
Property Settingshttps://support.google.com/webmasters/topic/9354552?hl=en&ref_topic=9456376
​robots.txt reporthttps://support.google.com/webmasters/answer/6062598
robots.txthttps://support.google.com/webmasters/answer/12818275
Use noindex if you want to prevent content from appearing in search resultshttps://developers.google.com/search/docs/advanced/crawling/block-indexing
Domain propertyhttps://support.google.com/webmasters/answer/10431861
URL-prefix propertyhttps://support.google.com/webmasters/answer/10432366
Open robots.txt reporthttps://search.google.com/search-console/settings/robots-txt?utm_source=support.google.com/webmasters/&utm_medium=referral&utm_campaign= 6062598
Location of robots.txt fileshttps://support.google.com/webmasters/answer/6062598#which_urls
inspecting the URLhttps://support.google.com/webmasters/answer/9012289#test_live_page
how Google behaves when there's a robots.txt errorhttps://developers.google.com/search/docs/crawling-indexing/robots/robots_txt#http-status-codes
List of indexing issueshttps://support.google.com/webmasters/answer/7440203#status_types
how Google behaves when there's a robots.txt errorhttps://developers.google.com/search/docs/crawling-indexing/robots/robots_txt#http-status-codes
robots.txt validatorhttps://www.google.com/search?q=robots.txt+validators
RFC 9309https://www.rfc-editor.org/rfc/rfc9309.html
create a domain propertyhttps://support.google.com/webmasters/answer/34592#domain_property&zippy=%2Cdomain-property-examplecom
URL inspection toolhttps://support.google.com/webmasters/answer/9012289
Google's open source robots.txt libraryhttps://github.com/google/robotstxt
What is a robots.txt filehttps://developers.google.com/search/docs/advanced/robots/intro
How to implement a robots.txt filehttps://developers.google.com/search/docs/advanced/robots/create-robots-txt
Post to the help community Get answers from community members https://support.google.com/webmasters/community?hl=en&help_center_link=CIaE8gISFOKAi3JvYm90cy50eHQgcmVwb3J0
Property Settingshttps://support.google.com/webmasters/topic/9354552?hl=en&ref_topic=9456376
Search Console Property Settings pagehttps://support.google.com/webmasters/answer/7687465?hl=en&ref_topic=9354552
Verify your site ownershiphttps://support.google.com/webmasters/answer/9008080?hl=en&ref_topic=9354552
Managing owners, users, and permissionshttps://support.google.com/webmasters/answer/7687615?hl=en&ref_topic=9354552
Associationshttps://support.google.com/webmasters/answer/9419894?hl=en&ref_topic=9354552
Change of Address toolhttps://support.google.com/webmasters/answer/9370220?hl=en&ref_topic=9354552
Export Search Console data to BigQueryhttps://support.google.com/webmasters/topic/12917674?hl=en&ref_topic=9354552
Search generative AI controlhttps://support.google.com/webmasters/answer/16908024?hl=en&ref_topic=9354552
Shipping and return settingshttps://support.google.com/webmasters/answer/14907594?hl=en&ref_topic=9354552
​robots.txt reporthttps://support.google.com/webmasters/answer/6062598
Crawl Stats reporthttps://support.google.com/webmasters/answer/9679690?hl=en&ref_topic=9354552
Remove a propertyhttps://support.google.com/webmasters/answer/9357768?hl=en&ref_topic=9354552
Opt out of display on local search results and other Google propertieshttps://support.google.com/webmasters/answer/3035947?hl=en&ref_topic=9354552
https://support.google.com/webmasters/answer/9128669?hl=en&ref_topic=9128571
New to Search Console?https://support.google.com/webmasters/answer/9128669?hl=en&ref_topic=9128571
Privacy Policyhttps://www.google.com/intl/en/privacy.html
Terms of Servicehttps://www.google.com/accounts/TOS

Viewport: width=device-width,initial-scale=1,minimum-scale=1,maximum-scale=3,user-scalable=yes

Robots: follow,index

URLs of crawlers that visited me.