René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"robotparser doesn't support request rate and crawl delay parameters","articleBody":"BPO | [16099](https://bugs.python.org/issue16099)\n--- | :---\nNosy | @rhettinger, @orsenthil, @tiran, @berkerpeksag, @hynek\nFiles | \u003cli\u003e[robotparser.patch](https://bugs.python.org/file27373/robotparser.patch \"Uploaded as text/plain at 2012-10-01.12:58:24 by XapaJIaMnu\"): patch for robotparser.py\u003c/li\u003e\u003cli\u003e[robotparser.patch](https://bugs.python.org/file27374/robotparser.patch \"Uploaded as text/plain at 2012-10-01.13:37:51 by XapaJIaMnu\"): same patch for python3X\u003c/li\u003e\u003cli\u003e[robotparser.patch](https://bugs.python.org/file27476/robotparser.patch \"Uploaded as text/plain at 2012-10-07.18:20:52 by XapaJIaMnu\"): Changes + test cases + documentation\u003c/li\u003e\u003cli\u003e[robotparser_reformatted.patch](https://bugs.python.org/file27477/robotparser_reformatted.patch \"Uploaded as text/plain at 2012-10-07.19:56:11 by XapaJIaMnu\"): Changes, test cases, documentation, reformatted\u003c/li\u003e\u003cli\u003e[robotparser_v2.patch](https://bugs.python.org/file33071/robotparser_v2.patch \"Uploaded as text/plain at 2013-12-10.00:22:50 by XapaJIaMnu\"): V2 with fixes\u003c/li\u003e\u003cli\u003e[robotparser_v3.patch](https://bugs.python.org/file35377/robotparser_v3.patch \"Uploaded as text/plain at 2014-05-27.09:29:03 by XapaJIaMnu\"): V3 crawl delay and request rate patch\u003c/li\u003e\n\n\u003csup\u003e*Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.*\u003c/sup\u003e\n\n\u003cdetails\u003e\u003csummary\u003eShow more details\u003c/summary\u003e\u003cp\u003e\n\nGitHub fields:\n```python\nassignee = 'https://github.com/berkerpeksag'\nclosed_at = \u003cDate 2015-10-08.09:34:21.553\u003e\ncreated_at = \u003cDate 2012-10-01.12:58:25.141\u003e\nlabels = ['easy', 'type-feature', 'library']\ntitle = \"robotparser doesn't support request rate and crawl delay parameters\"\nupdated_at = \u003cDate 2015-10-08.09:34:21.551\u003e\nuser = 'https://bugs.python.org/XapaJIaMnu'\n```\n\nbugs.python.org fields:\n```python\nactivity = \u003cDate 2015-10-08.09:34:21.551\u003e\nactor = 'berker.peksag'\nassignee = 'berker.peksag'\nclosed = True\nclosed_date = \u003cDate 2015-10-08.09:34:21.553\u003e\ncloser = 'berker.peksag'\ncomponents = ['Library (Lib)']\ncreation = \u003cDate 2012-10-01.12:58:25.141\u003e\ncreator = 'XapaJIaMnu'\ndependencies = []\nfiles = ['27373', '27374', '27476', '27477', '33071', '35377']\nhgrepos = []\nissue_num = 16099\nkeywords = ['patch', 'easy', 'needs review']\nmessage_count = 17.0\nmessages = ['171711', '171712', '171715', '171719', '172327', '172338', '205567', '205641', '205755', '205761', '208721', '219212', '223099', '225916', '252483', '252521', '252525']\nnosy_count = 7.0\nnosy_names = ['rhettinger', 'orsenthil', 'christian.heimes', 'python-dev', 'berker.peksag', 'hynek', 'XapaJIaMnu']\npr_nums = []\npriority = 'normal'\nresolution = 'fixed'\nstage = 'resolved'\nstatus = 'closed'\nsuperseder = None\ntype = 'enhancement'\nurl = 'https://bugs.python.org/issue16099'\nversions = ['Python 3.6']\n```\n\n\u003c/p\u003e\u003c/details\u003e\n","author":{"url":"https://github.com/XapaJIaMnu","@type":"Person","name":"XapaJIaMnu"},"datePublished":"2012-10-01T12:58:25.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":17},"url":"https://github.com/60303/cpython/issues/60303"}