René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Heap buffer overflow in `_PyTokenizer_ensure_utf8`","articleBody":"# Bug report\n\n### Bug description:\n\nOSS-Fuzz has found a heap buffer overflow in `_PyTokenizer_ensure_utf8`. [Link to OSS-Fuzz bug report](https://issues.oss-fuzz.com/issues/451112368).\n\nThe root cause is that `valid_utf8()` in `Parser/tokenizer/helpers.c` checks continuation bytes in reverse order thus reader `s[expected]` before `s[1]` on these lines:\n\nhttps://github.com/python/cpython/blob/8b7b5a994602824a5e41cf2516691212fcdfa25e/Parser/tokenizer/helpers.c#L497-L499\n\nWhen a multi-byte UTF-8 sequence is truncated - such as a 3-byte lead `\\xEA` followed immediately by a null terminator - the backward loop reads past the end of the valid data before encountering the null byte that would stop it.\n\nThis is not a security-critical issue.\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\n_No response_\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-144807\n* gh-145287\n* gh-145441\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/AdamKorcz","@type":"Person","name":"AdamKorcz"},"datePublished":"2026-02-16T12:13:49.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":1},"url":"https://github.com/144872/cpython/issues/144872"}