Title: Several C API functions do not return NUL-terminated buffers · Issue #8685 · RustPython/RustPython · GitHub
Open Graph Title: Several C API functions do not return NUL-terminated buffers · Issue #8685 · RustPython/RustPython
X Title: Several C API functions do not return NUL-terminated buffers · Issue #8685 · RustPython/RustPython
Description: Summary The C API documents that the buffers returned by PyBytes_AsString(), PyByteArray_AsString(), and PyUnicode_AsUTF8AndSize() to contain an extra trailing NUL byte. RustPython instead returns pointers to the objects' exact-length Ru...
Open Graph Description: Summary The C API documents that the buffers returned by PyBytes_AsString(), PyByteArray_AsString(), and PyUnicode_AsUTF8AndSize() to contain an extra trailing NUL byte. RustPython instead returns ...
X Description: Summary The C API documents that the buffers returned by PyBytes_AsString(), PyByteArray_AsString(), and PyUnicode_AsUTF8AndSize() to contain an extra trailing NUL byte. RustPython instead returns ...
Opengraph URL: https://github.com/RustPython/RustPython/issues/8685
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Several C API functions do not return NUL-terminated buffers","articleBody":"## Summary\n\nThe C API documents that the buffers returned by `PyBytes_AsString()`, `PyByteArray_AsString()`, and `PyUnicode_AsUTF8AndSize()` to contain an extra trailing NUL byte. RustPython instead returns pointers to the objects' exact-length Rust buffers. Reading the documented byte at `data[len]` is therefore a heap-buffer-overflow.\n\nThe relevant implementations return raw pointers without adding a terminator: [`PyBytes_AsString()`](https://github.com/RustPython/RustPython/blob/8195385d6176a6ef94c4653f8093aeca6c7ca6c5/crates/capi/src/bytesobject.rs#L41-L45), [`PyByteArray_AsString()`](https://github.com/RustPython/RustPython/blob/8195385d6176a6ef94c4653f8093aeca6c7ca6c5/crates/capi/src/bytearrayobject.rs#L58-L62), and [`PyUnicode_AsUTF8AndSize()`](https://github.com/RustPython/RustPython/blob/8195385d6176a6ef94c4653f8093aeca6c7ca6c5/crates/capi/src/unicodeobject.rs#L73-L92).\n\nThe bytearray case was [raised during review of PR #8009](https://github.com/RustPython/RustPython/pull/8009#discussion_r3340166727), but dismissed as intentional. However, I'm not sure whether this being a soundness issue was raised at that time.\n\n## Reproducer\n\nSave as `capi_trailing_null.c`:\n\n```c\n#include \u003cPython.h>\n#include \u003cstdio.h>\n\nint main(void)\n{\n PyObject *bytes, *bytearray, *unicode;\n char *bytes_data, *bytearray_data;\n const char *unicode_data;\n Py_ssize_t bytes_len, bytearray_len, unicode_len;\n unsigned char bytes_trailing, bytearray_trailing, unicode_trailing;\n\n Py_Initialize();\n\n bytes = PyBytes_FromStringAndSize(\"abc\", 3);\n bytearray = PyByteArray_FromStringAndSize(\"abc\", 3);\n unicode = PyUnicode_FromStringAndSize(\"abc\", 3);\n if (!bytes || !bytearray || !unicode)\n return 2;\n\n bytes_data = PyBytes_AsString(bytes);\n bytearray_data = PyByteArray_AsString(bytearray);\n unicode_data = PyUnicode_AsUTF8AndSize(unicode, &unicode_len);\n bytes_len = PyBytes_Size(bytes);\n bytearray_len = PyByteArray_Size(bytearray);\n if (!bytes_data || !bytearray_data || !unicode_data ||\n bytes_len \u003c 0 || bytearray_len \u003c 0)\n return 2;\n\n bytes_trailing = (unsigned char)bytes_data[bytes_len];\n bytearray_trailing = (unsigned char)bytearray_data[bytearray_len];\n unicode_trailing = (unsigned char)unicode_data[unicode_len];\n printf(\"bytes: data[len] = 0x%02x\\n\", bytes_trailing);\n printf(\"bytearray: data[len] = 0x%02x\\n\", bytearray_trailing);\n printf(\"unicode UTF-8: data[len] = 0x%02x\\n\", unicode_trailing);\n\n Py_Finalize();\n return bytes_trailing || bytearray_trailing || unicode_trailing;\n}\n```\n\n\n## Expected\n\nWith CPython:\n\n```console\n$ cc -fsanitize=address -fno-omit-frame-pointer $(python3-config --cflags) capi_trailing_null.c $(python3-config --embed --ldflags) -o trailing-null-cpython\n$ ASAN_OPTIONS=detect_leaks=0 ./trailing-null-cpython\nbytes: data[len] = 0x00\nbytearray: data[len] = 0x00\nunicode UTF-8: data[len] = 0x00\n```\n\n## Actual\n\nFrom a RustPython checkout on macOS (use `.so` instead of `.dylib` on Linux):\n\n```console\n$ cargo build -p rustpython-capi\n$ cc -fsanitize=address -fno-omit-frame-pointer $(python3-config --includes) capi_trailing_null.c target/debug/librustpython_capi.dylib -o trailing-null-rustpython\n$ ASAN_OPTIONS=detect_leaks=0 ./trailing-null-rustpython\n==...==ERROR: AddressSanitizer: heap-buffer-overflow\n...\n0x... is located 0 bytes after 3-byte region [0x...,0x...)\n```\n\n## Python Documentation\n\nCPython documents the trailing byte for [`PyBytes_AsString()`](https://docs.python.org/3/c-api/bytes.html#c.PyBytes_AsString), [`PyByteArray_AsString()`](https://docs.python.org/3/c-api/bytearray.html#c.PyByteArray_AsString), and [`PyUnicode_AsUTF8AndSize()`](https://docs.python.org/3/c-api/unicode.html#c.PyUnicode_AsUTF8AndSize).\n\n","author":{"url":"https://github.com/ngoldbaum","@type":"Person","name":"ngoldbaum"},"datePublished":"2026-09-10T15:22:01Z","url":"https://github.com/RustPython/RustPython/issues/8685"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:fb543c82-6930-5f8c-e4a1-abc1ca15cf2f |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | DD3A:A70A9:E8828D:133DADF:6B01D40E |
| html-safe-nonce | c9ba0fa352dfad1af00ceb1b2dd9398947001a5d86116ed146eef1e1c8dc5dd7 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJERDNBOkE3MEE5OkU4ODI4RDoxMzNEQURGOjZCMDFENDBFIiwidmlzaXRvcl9pZCI6IjYyODc0MzA4OTU4ODQ0MjQyMDYiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | 375f50383ada44b834bc9cc98aaadc0f6dd72cf91c4397ef2a8306d6e2a04239 |
| hovercard-subject-tag | issue:5414073907 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/RustPython/RustPython/8685/issue_layout |
| twitter:image | https://opengraph.githubassets.com/92acd434161022e6cb9611d9f4772423453ec7f73eb09acbcba050a8378bc8f6/RustPython/RustPython/issues/8685 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/92acd434161022e6cb9611d9f4772423453ec7f73eb09acbcba050a8378bc8f6/RustPython/RustPython/issues/8685 |
| og:image:alt | Summary The C API documents that the buffers returned by PyBytes_AsString(), PyByteArray_AsString(), and PyUnicode_AsUTF8AndSize() to contain an extra trailing NUL byte. RustPython instead returns ... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | ngoldbaum |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width