René's URL Explorer Experiment

Title: 13.3. robotparser — Parser for robots.txt — Python 2.7.18 documentation

Links:

no longer supported	https://devguide.python.org/devcycle/#end-of-life-branches
Python documentation for the current stable release	https://docs.python.org/3/library/robotparser.html
index	https://docs.python.org/2/genindex.html
modules	https://docs.python.org/2/py-modindex.html
next	https://docs.python.org/2/library/netrc.html
previous	https://docs.python.org/2/library/configparser.html
Python	https://www.python.org/
Python 2.7.18 documentation	https://docs.python.org/2/index.html
The Python Standard Library	https://docs.python.org/2/library/index.html
13. File Formats	https://docs.python.org/2/library/fileformats.html
robotparser	https://docs.python.org/2/library/robotparser.html#module-robotparser
¶	https://docs.python.org/2/library/robotparser.html#module-robotparser
robotparser	https://docs.python.org/2/library/robotparser.html#module-robotparser
2to3	https://docs.python.org/2/glossary.html#term-2to3
RobotFileParser	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser
http://www.robotstxt.org/orig.html	http://www.robotstxt.org/orig.html
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser.set_url
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser.read
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser.parse
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser.can_fetch
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser.mtime
¶	https://docs.python.org/2/library/robotparser.html#robotparser.RobotFileParser.modified
13.2. ConfigParser — Configuration file parser	https://docs.python.org/2/library/configparser.html
13.4. netrc — netrc file processing	https://docs.python.org/2/library/netrc.html
Show Source	https://docs.python.org/2/_sources/library/robotparser.rst.txt
index	https://docs.python.org/2/genindex.html
modules	https://docs.python.org/2/py-modindex.html
next	https://docs.python.org/2/library/netrc.html
previous	https://docs.python.org/2/library/configparser.html
Python	https://www.python.org/
Python 2.7.18 documentation	https://docs.python.org/2/index.html
The Python Standard Library	https://docs.python.org/2/library/index.html
13. File Formats	https://docs.python.org/2/library/fileformats.html
Copyright	https://docs.python.org/2/copyright.html
Please donate.	https://www.python.org/psf/donations/
Found a bug	https://docs.python.org/2/bugs.html
Sphinx	http://sphinx.pocoo.org/

URLs of crawlers that visited me.