René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"3.12: tokenize adds a newline when it is not there","articleBody":"\u003c!--\r\n  If you're new to Python and you're not sure whether what you're experiencing is a bug, the CPython issue tracker is not\r\n  the right place to seek help. Consider the following options instead:\r\n\r\n  - reading the Python tutorial: https://docs.python.org/3/tutorial/\r\n  - posting in the \"Users\" category on discuss.python.org: https://discuss.python.org/c/users/7\r\n  - emailing the Python-list mailing list: https://mail.python.org/mailman/listinfo/python-list\r\n  - searching our issue tracker (https://github.com/python/cpython/issues) to see if\r\n    your problem has already been reported\r\n--\u003e\r\n\r\n# Bug report\r\n\r\ntokenize adds a concrete newline character when it is not present.  this breaks any sort of roundtrip source generation and pycodestyle's end-of-file checker\r\n\r\nhere's an example of a file containing a single byte (generated via `echo -n 1 \u003e t.py`)\r\n\r\n```console\r\n# hd -c t.py \r\n00000000  31                                                |1|\r\n0000000   1                                                            \r\n0000001\r\n# python3.12 -m tokenize t.py\r\n0,0-0,0:            ENCODING       'utf-8'        \r\n1,0-1,1:            NUMBER         '1'            \r\n1,1-1,2:            NEWLINE        '\\n'           \r\n2,0-2,0:            ENDMARKER      ''             \r\n# python3.11 -m tokenize t.py\r\n0,0-0,0:            ENCODING       'utf-8'        \r\n1,0-1,1:            NUMBER         '1'            \r\n1,1-1,2:            NEWLINE        ''             \r\n2,0-2,0:            ENDMARKER      '' \r\n```\r\n\r\n# Your environment\r\n\r\n- CPython versions tested on: 3.12 dbd7d7c8e1\r\n- Operating system and architecture: ubuntu 22.04 LTS x86_64\r\n\r\ncc @pablogsal\r\n\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-105364\n* gh-105367\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/asottile","@type":"Person","name":"asottile"},"datePublished":"2023-06-02T22:32:02.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":8},"url":"https://github.com/105259/cpython/issues/105259"}