René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Heap-buffer-overflow in the `hz` incremental decoder on a trailing `~`","articleBody":"# Bug report\n\n### Bug description:\n\n```python\nimport codecs\nd = codecs.getincrementaldecoder('hz')()\nd.decode(b'~', False)   \nd.decode(b'', True)\n```\n\n```\n=================================================================\n==1==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x5020000040d1 at pc 0xffff7feff068 bp 0xffffd608df50 sp 0xffffd608df40\nREAD of size 1 at 0x5020000040d1 thread T0\n    #0 0xffff7feff064 in hz_decode Modules/cjkcodecs/_codecs_cn.c:417\n    #1 0xffff7fe7426c in decoder_feed_buffer Modules/cjkcodecs/multibytecodec.c:921\n    #2 0xffff7fe76238 in _multibytecodec_MultibyteIncrementalDecoder_decode_impl Modules/cjkcodecs/multibytecodec.c:1214\n    #3 0xffff7fe76238 in _multibytecodec_MultibyteIncrementalDecoder_decode Modules/cjkcodecs/clinic/multibytecodec.c.h:395\n    #4 0xaaaad3656320 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:144\n    #5 0xaaaad3656320 in PyObject_Vectorcall Objects/call.c:327\n    #6 0xaaaad3ca1bc8 in _Py_VectorCallInstrumentation_StackRefSteal Python/ceval.c:768\n    #7 0xaaaad33d1de8 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1906\n    #8 0xaaaad3cc0b1c in _PyEval_EvalFrame Include/internal/pycore_ceval.h:122\n    #9 0xaaaad3cc0b1c in _PyEval_Vector Python/ceval.c:2174\n    #10 0xaaaad3cc0b1c in PyEval_EvalCode Python/ceval.c:679\n    #11 0xaaaad3ebfeb4 in run_mod Python/pythonrun.c:1509\n    #12 0xaaaad3ec0d38 in _PyRun_File Python/pythonrun.c:1332\n    #13 0xaaaad3ec4ffc in _PyRun_SimpleFile Python/pythonrun.c:544\n    #14 0xaaaad3ec8dd0 in _PyRun_AnyFile Python/pythonrun.c:92\n    #15 0xaaaad3f9390c in pymain_run_file_obj Modules/main.c:478\n    #16 0xaaaad3f9390c in pymain_run_file Modules/main.c:494\n    #17 0xaaaad3f9390c in pymain_run_python Modules/main.c:804\n    #18 0xaaaad3f9390c in Py_RunMain Modules/main.c:891\n    #19 0xaaaad3f95c38 in pymain_main Modules/main.c:921\n    #20 0xaaaad3f95c38 in Py_BytesMain Modules/main.c:945\n    #21 0xffff82c484c0  (/lib/aarch64-linux-gnu/libc.so.6+0x284c0) (BuildId: 27027b96e5b8c475fc327aa445bea1c71d37b4e2)\n    #22 0xffff82c48594 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x28594) (BuildId: 27027b96e5b8c475fc327aa445bea1c71d37b4e2)\n    #23 0xaaaad343beec in _start (/src/cpython/python+0x183beec) (BuildId: eba909a3db0adabacf151d7dc3081dda7999191a)\n\n0x5020000040d1 is located 0 bytes after 1-byte region [0x5020000040d0,0x5020000040d1)\nallocated by thread T0 here:\n    #0 0xffff835e76d0 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69\n    #1 0xffff7fe76090 in _multibytecodec_MultibyteIncrementalDecoder_decode_impl Modules/cjkcodecs/multibytecodec.c:1201\n    #2 0xffff7fe76090 in _multibytecodec_MultibyteIncrementalDecoder_decode Modules/cjkcodecs/clinic/multibytecodec.c.h:395\n    #3 0xaaaad3656320 in _PyObject_VectorcallTstate Include/internal/pycore_call.h:144\n    #4 0xaaaad3656320 in PyObject_Vectorcall Objects/call.c:327\n    #5 0xaaaad3ca1bc8 in _Py_VectorCallInstrumentation_StackRefSteal Python/ceval.c:768\n    #6 0xaaaad33d1de8 in _PyEval_EvalFrameDefault Python/generated_cases.c.h:1906\n    #7 0xaaaad3cc0b1c in _PyEval_EvalFrame Include/internal/pycore_ceval.h:122\n    #8 0xaaaad3cc0b1c in _PyEval_Vector Python/ceval.c:2174\n    #9 0xaaaad3cc0b1c in PyEval_EvalCode Python/ceval.c:679\n    #10 0xaaaad3ebfeb4 in run_mod Python/pythonrun.c:1509\n    #11 0xaaaad3ec0d38 in _PyRun_File Python/pythonrun.c:1332\n    #12 0xaaaad3ec4ffc in _PyRun_SimpleFile Python/pythonrun.c:544\n    #13 0xaaaad3ec8dd0 in _PyRun_AnyFile Python/pythonrun.c:92\n    #14 0xaaaad3f9390c in pymain_run_file_obj Modules/main.c:478\n    #15 0xaaaad3f9390c in pymain_run_file Modules/main.c:494\n    #16 0xaaaad3f9390c in pymain_run_python Modules/main.c:804\n    #17 0xaaaad3f9390c in Py_RunMain Modules/main.c:891\n    #18 0xaaaad3f95c38 in pymain_main Modules/main.c:921\n    #19 0xaaaad3f95c38 in Py_BytesMain Modules/main.c:945\n    #20 0xffff82c484c0  (/lib/aarch64-linux-gnu/libc.so.6+0x284c0) (BuildId: 27027b96e5b8c475fc327aa445bea1c71d37b4e2)\n    #21 0xffff82c48594 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x28594) (BuildId: 27027b96e5b8c475fc327aa445bea1c71d37b4e2)\n    #22 0xaaaad343beec in _start (/src/cpython/python+0x183beec) (BuildId: eba909a3db0adabacf151d7dc3081dda7999191a)\n\nSUMMARY: AddressSanitizer: heap-buffer-overflow Modules/cjkcodecs/_codecs_cn.c:417 in hz_decode\nShadow bytes around the buggy address:\n  0x502000003e00: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd\n  0x502000003e80: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fa\n  0x502000003f00: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa\n  0x502000003f80: fa fa fd fa fa fa fd fa fa fa fd fa fa fa fd fa\n  0x502000004000: fa fa fd fa fa fa fd fd fa fa fd fa fa fa fd fa\n=>0x502000004080: fa fa fd fa fa fa fd fa fa fa[01]fa fa fa fa fa\n  0x502000004100: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa\n  0x502000004180: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa\n  0x502000004200: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa\n  0x502000004280: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa\n  0x502000004300: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa\nShadow byte legend (one shadow byte represents 8 application bytes):\n  Addressable:           00\n  Partially addressable: 01 02 03 04 05 06 07\n  Heap left redzone:       fa\n  Freed heap region:       fd\n  Stack left redzone:      f1\n  Stack mid redzone:       f2\n  Stack right redzone:     f3\n  Stack after return:      f5\n  Stack use after scope:   f8\n  Global redzone:          f9\n  Global init order:       f6\n  Poisoned by user:        f7\n  Container overflow:      fc\n  Array cookie:            ac\n  Intra object redzone:    bb\n  ASan internal:           fe\n  Left alloca redzone:     ca\n  Right alloca redzone:    cb\n==1==ABORTING\n```\n\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-157333\n* gh-158161\n* gh-158162\n* gh-158163\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/djarfluka","@type":"Person","name":"djarfluka"},"datePublished":"2026-09-11T15:03:15Z","url":"https://github.com/python/cpython/issues/157325"}