Title: `test_capi.test_unicode.test_check_interned` fails on a default build that reads a `.pyc` written by a free-threaded build · Issue #162092 · python/cpython · GitHub
Open Graph Title: `test_capi.test_unicode.test_check_interned` fails on a default build that reads a `.pyc` written by a free-threaded build · Issue #162092 · python/cpython
X Title: `test_capi.test_unicode.test_check_interned` fails on a default build that reads a `.pyc` written by a free-threaded build · Issue #162092 · python/cpython
Description: Bug report I was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (#159718) with overnight stress tests and had a default build and a free-threaded build configured out of tree from one checkout, as a reference. The de...
Open Graph Description: Bug report I was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (#159718) with overnight stress tests and had a default build and a free-threaded build configured out of tree f...
X Description: Bug report I was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (#159718) with overnight stress tests and had a default build and a free-threaded build configured out of tree f...
Opengraph URL: https://github.com/python/cpython/issues/162092
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`test_capi.test_unicode.test_check_interned` fails on a default build that reads a `.pyc` written by a free-threaded build","articleBody":"### Bug report\n\nI was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (https://github.com/python/cpython/pull/159718) with overnight stress tests and had a default build and a free-threaded build configured out of tree from one checkout, as a reference. The default build then fails one test every time, also when it runs alone:\n\n```\n$ ./python -m test test_capi -m test_check_interned\n File \"Lib/test/test_capi/test_unicode.py\", line 1896, in test_check_interned\n self.assertEqual(check_interned(s), SSTATE_INTERNED_MORTAL)\nAssertionError: 2 != 1\n```\n\nIt looks like an assumption of the test (added in https://github.com/python/cpython/pull/157798, also in 3.14 and 3.15) rather than a runtime bug. Main at e8c0584877 (2026-11-10) on Linux arm64, and at 7c59816cb4 (2026-11-16) on macOS arm64. An AI assistant helped with the analysis.\n\n### What happens\n\nWhether it fails depends on which build wrote `Lib/test/test_capi/__pycache__/test_unicode.cpython-316.pyc`:\n\n| the `.pyc` was written by | test run by the default build | test run by the free-threaded build |\n|---|---|---|\n| the default build | passes | passes |\n| the free-threaded build | fails | passes |\n\nBoth builds use the cache tag `cpython-316`, so they read each other's `.pyc` files. The free-threaded build interns every string constant of a code object ([`_PyCode_ShouldInternString()`](https://github.com/python/cpython/blob/e8c0584877e0/Objects/codeobject.c#L116-L137)), and marshal [writes](https://github.com/python/cpython/blob/e8c0584877e0/Python/marshal.c#L1043-L1046) an interned string as `TYPE_INTERNED`. When the default build loads such a file, marshal [interns](https://github.com/python/cpython/blob/e8c0584877e0/Python/marshal.c#L1801-L1806) every string marked that way as immortal. The test's literal `'long string unlikely to be used by Python'` is a constant of the test's own code, so it is already interned and immortal by then, and `sys.intern()` of an equal fresh string returns that one.\n\nWith `PYTHONPYCACHEPREFIX` pointing to a directory of its own, the default build passes, and so does its full test suite.\n\n### Without the test suite\n\n```python\n# interned_mod.py\nimport sys, _testcapi\n\ndef fresh_string(text):\n return text.encode().decode()\n\ns = sys.intern(fresh_string('long string unlikely to be used by Python'))\nprint(\"free-threaded\" if \"t\" in sys.abiflags else \"default (GIL)\", \"build: PyUnicode_CHECK_INTERNED() of an interned copy is\", _testcapi.unicode_check_interned(s), \"(1 = mortal, 2 = immortal)\")\n```\n\n```\n$ rm -rf __pycache__; default/python -c \"import interned_mod\"\ndefault (GIL) build: PyUnicode_CHECK_INTERNED() of an interned copy is 1 (1 = mortal, 2 = immortal)\n$ rm -rf __pycache__; free-threaded/python -c \"import interned_mod\"\nfree-threaded build: PyUnicode_CHECK_INTERNED() of an interned copy is 2 (1 = mortal, 2 = immortal)\n$ default/python -c \"import interned_mod\"\ndefault (GIL) build: PyUnicode_CHECK_INTERNED() of an interned copy is 2 (1 = mortal, 2 = immortal)\n```\n\n### A possible change\n\nThe test could intern a text that is put together at run time, so that it can't be a constant of its own code. Other tests build such strings at run time too, to get one that isn't interned yet, for example `test_capi/test_object.py` (`''.join(['decref', '_delayed', '_immortal'])`) and `test_sys.test_intern` (`\"never interned before\" + str(random.randrange(0, 10**9))`). In `Lib/test/test_capi/test_unicode.py`:\n\n```diff\n- s = fresh_string('long string unlikely to be used by Python')\n+ # Built at run time: a constant of this module may already be\n+ # interned (and immortal) when the .pyc file was written by a\n+ # free-threaded build.\n+ s = fresh_string(''.join(['long string ', 'unlikely to be used by Python']))\n```\n\nWith that change the test passes on the default build with a `.pyc` written by the free-threaded build, and the other way round, and `test_capi.test_unicode` passes on both builds. I'll open a PR with this change.\n\nOne thing beyond the test, in case it is of interest: a default build that loads `.pyc` files written by a free-threaded build keeps every string constant of those modules immortal. That only happens where both builds import the same source files, for example two builds from one checkout, or a project's own modules run with both `python3.16` and `python3.16t` (the installed standard libraries are separate). I haven't measured what it costs, and whether the two builds should share `.pyc` files at all is a separate question; the test change doesn't depend on it.\n\ncc @vstinner\n\n\n### CPython versions tested on:\n\n3.14, 3.15, CPython main branch\n\n### Operating systems tested on:\n\nmacOS, Linux\n\n\u003c!-- gh-linked-prs -->\n### Linked PRs\n* gh-162099\n* gh-162118\n* gh-162119\n\u003c!-- /gh-linked-prs -->\n","author":{"url":"https://github.com/maikschulze","@type":"Person","name":"maikschulze"},"datePublished":"2026-11-16T08:30:56Z","url":"https://github.com/python/cpython/issues/162092"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:33f7520c-2428-5208-5cfe-64334b20f623 |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 94DA:28A483:11DA929:17AB1DF:6B01E84A |
| html-safe-nonce | 7fccebaa665c484c4d75d9993580d4573fe454858761959ad10c17b0a07ad77f |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5NERBOjI4QTQ4MzoxMURBOTI5OjE3QUIxREY6NkIwMUU4NEEiLCJ2aXNpdG9yX2lkIjoiNjE2NDAwNzg0NTYwNjEzNDUwIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | c282afa18864956025f3f7c5516bd66858cc4488659fee8b6dfd2d3aaedbf19f |
| hovercard-subject-tag | issue:6417328431 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/python/cpython/162092/issue_layout |
| twitter:image | https://opengraph.githubassets.com/f3dbc1476c2d69345db698a7128798d6cf40df2166a965006411132d8344c9e1/python/cpython/issues/162092 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/f3dbc1476c2d69345db698a7128798d6cf40df2166a965006411132d8344c9e1/python/cpython/issues/162092 |
| og:image:alt | Bug report I was exploring peak RSS gaps of the free-threaded build with mimalloc v3 (#159718) with overnight stress tests and had a default build and a free-threaded build configured out of tree f... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | maikschulze |
| hostname | github.com |
| expected-hostname | github.com |
| None | cc5dfdd32f62ebd3223cab3af3aa2b268d3605218237326a48a99d1efd89495c |
| turbo-cache-control | no-preview |
| go-import | github.com/python/cpython git https://github.com/python/cpython.git |
| octolytics-dimension-user_id | 1525981 |
| octolytics-dimension-user_login | python |
| octolytics-dimension-repository_id | 81598961 |
| octolytics-dimension-repository_nwo | python/cpython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 81598961 |
| octolytics-dimension-repository_network_root_nwo | python/cpython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 8c77c9e8439031ad54c75a250694842892f693e1 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width