Title: Concurrent compilation changes marshal output in free-threaded builds · Issue #156504 · python/cpython · GitHub
Open Graph Title: Concurrent compilation changes marshal output in free-threaded builds · Issue #156504 · python/cpython
X Title: Concurrent compilation changes marshal output in free-threaded builds · Issue #156504 · python/cpython
Description: Bug description: Concurrent compilation on free-threaded CPython builds can produce marshal output that differs byte-for-byte from serial compilation of the same source. This occurs both with the GIL disabled and with the GIL explicitly ...
Open Graph Description: Bug description: Concurrent compilation on free-threaded CPython builds can produce marshal output that differs byte-for-byte from serial compilation of the same source. This occurs both with the G...
X Description: Bug description: Concurrent compilation on free-threaded CPython builds can produce marshal output that differs byte-for-byte from serial compilation of the same source. This occurs both with the G...
Opengraph URL: https://github.com/python/cpython/issues/156504
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Concurrent compilation changes marshal output in free-threaded builds","articleBody":"### Bug description:\n\nConcurrent compilation on free-threaded CPython builds can produce marshal output that differs byte-for-byte from serial compilation of the same source.\n\nThis occurs both with the GIL disabled and with the GIL explicitly re-enabled. It reproduces on Windows and Linux/WSL. Standard CPython builds used as controls did not reproduce the difference.\n\nThis was discovered while implementing threaded bytecode compilation in [pypa/pip#14286](https://github.com/pypa/pip/pull/14286).\n\n## Reproducer\n\nA standalone standard-library-only reproducer is available here:\n\n- [Reproducer script](https://github.com/khmyznikov/pip/blob/review/windows-bytecode-threads/tools/repro_threaded_pyc_determinism.py)\n- [Methodology and results](https://github.com/khmyznikov/pip/blob/review/windows-bytecode-threads/tools/repro_threaded_pyc_determinism.md)\n\nIt does not import or execute pip. It:\n\n- creates 64 copies of a minimized source fixture;\n- compiles corresponding files serially and with a `ThreadPoolExecutor`, in fresh child interpreters;\n- fixes logical filenames, `PYTHONHASHSEED`, and `SOURCE_DATE_EPOCH`;\n- compares checked-hash `.pyc` files or raw `marshal.dumps(compile(...))` payloads;\n- unmarshals differing outputs and recursively compares documented code-object fields.\n\nExample using the raw marshal backend:\n\n```console\npython3.14t repro_threaded_pyc_determinism.py --gil off --backend marshal --workers 4 --output repro-ft-off-marshal --require-difference\npython3.14t repro_threaded_pyc_determinism.py --gil on --backend marshal --workers 4 --output repro-ft-on-marshal --require-difference\n```\n\nThe output directory must be new or empty.\n\n## Expected behavior\n\nSerial and concurrent compilation of the same source, using the same logical filename and deterministic environment settings, should produce byte-identical marshal output.\n\n## Actual behavior\n\nOn free-threaded builds, some concurrently compiled files have different serialized bytes from their serial counterparts.\n\nThe differing payloads have:\n\n- equal lengths;\n- equal `.pyc` headers when using `py_compile`;\n- equal values for all recursively inspected code-object fields after unmarshalling, including bytecode, constants, names, filenames, line tables, exception tables, and nested code objects.\n\nThe raw marshal backend reproduces the behavior without `py_compile` or `.pyc` headers. Its differing offsets are exactly 16 bytes earlier than the corresponding `.pyc` offsets, confirming that the `.pyc` header is not responsible.\n\nAn empty `semanticDifferences` result does not establish that the object identity or alias graph is identical. The precise cause has not yet been determined.\n\n## Observed results\n\n### Windows\n\nWindows 11 prerelease VM, eight virtual CPUs, CPython 3.15.0rc1 free-threaded build and corresponding standard CPython 3.15 build:\n\n| Build | GIL | Backend | Result |\n| --- | --- | --- | --- |\n| Free-threaded | Off | `py_compile` | Reproduced; 49 differing bytes in a 3,447-byte `.pyc` |\n| Free-threaded | On | `py_compile` | Reproduced; 49 differing bytes in a 3,447-byte `.pyc` |\n| Free-threaded | Off | `marshal` | Reproduced; 49 differing bytes in a 3,431-byte payload |\n| Free-threaded | On | `marshal` | Reproduced; 49 differing bytes in a 3,431-byte payload |\n| Standard | On/default | Both | No difference in 10 attempts |\n\n### Linux under WSL2\n\nUbuntu 26.04 LTS, kernel `6.18.33.2-microsoft-standard-WSL2`, native ext4 storage, four available CPUs:\n\n| Build | GIL | Backend | Result |\n| --- | --- | --- | --- |\n| CPython 3.14.7 free-threaded | Off | `py_compile` | Reproduced on attempt 1 |\n| CPython 3.14.7 free-threaded | On | `py_compile` | Reproduced on attempt 1 |\n| CPython 3.14.7 free-threaded | Off | `marshal` | Reproduced on attempt 1 |\n| CPython 3.14.7 free-threaded | On | `marshal` | Reproduced on attempt 1 |\n| CPython 3.14.4 standard | On/default | Both | No difference in 10 attempts |\n\nThe WSL differences contained 50 differing bytes per output. All inspected decoded code-object field values were equal.\n\n## Impact\n\nAlthough the decoded code objects appear semantically equivalent, byte-different outputs affect:\n\n- reproducible builds;\n- artifact hashing;\n- build caches;\n- deterministic CI output;\n- tools that record `.pyc` hashes.\n\nRe-enabling the GIL on a free-threaded build does not prevent the behavior. As a temporary precaution, pip’s proposed threaded compilation path detects `Py_GIL_DISABLED` at build time and remains serial for free-threaded builds.\n\n### CPython versions tested on:\n\n3.15\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-156862\n* gh-159192\n* gh-159290\n\u003c!-- /gh-linked-prs -->","author":{"url":"https://github.com/khmyznikov","@type":"Person","name":"khmyznikov"},"datePublished":"2026-08-27T23:02:22Z","url":"https://github.com/python/cpython/issues/156504"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:56acc3c0-0cbe-ca16-8dde-06ca98814ee1 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | EABC:2270F4:377E0E4:4992087:6B020AD9 |
| html-safe-nonce | d55038b038458fa2a7abb7a1d34038f44e5e69c704d1eff45f255fda1564bb64 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQUJDOjIyNzBGNDozNzdFMEU0OjQ5OTIwODc6NkIwMjBBRDkiLCJ2aXNpdG9yX2lkIjoiMzIxMDU5MjQ4NzY4NTA5ODIwMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | e7427c222933f168ff8a38db5fc2c498a0f7c6a183508ea7474a7319ac0fcd9e |
| hovercard-subject-tag | issue:5272910191 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/156504/issue_layout |
| twitter:image | https://opengraph.githubassets.com/73377fcc07c8111c430297c304f828c7073be1f1bca6b68901457742618d41ca/python/cpython/issues/156504 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/73377fcc07c8111c430297c304f828c7073be1f1bca6b68901457742618d41ca/python/cpython/issues/156504 |
| og:image:alt | Bug description: Concurrent compilation on free-threaded CPython builds can produce marshal output that differs byte-for-byte from serial compilation of the same source. This occurs both with the G... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | khmyznikov |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width