Title: Snapshot startup filesystem encoding on Linux by youknowdot · Pull Request #9689 · RustPython/RustPython · GitHub
Open Graph Title: Snapshot startup filesystem encoding on Linux by youknowdot · Pull Request #9689 · RustPython/RustPython
X Title: Snapshot startup filesystem encoding on Linux by youknowdot · Pull Request #9689 · RustPython/RustPython
Description: Summary Initialize the Linux command-line locale with CPython 3.15's environment precedence and PEP 538 coercion. Keep embedded startup non-mutating; the executable explicitly opts into the unsafe, single-threaded process-startup contract. Snapshot filesystem encoding in interpreter configuration. Share the fixed configuration with subinterpreters, keep Python codec objects local to each interpreter, and preserve filesystem behavior after later LC_CTYPE changes. Reuse native UTF-8, ASCII and Latin-1 codecs. GNU Linux uses an owned locale snapshot for wider-codec bootstrap, then switches to the initialized codec registry. Make filesystem decoding fallible throughout startup and OS-facing callers. Preserve Python exceptions, retain accepted sockets until address conversion succeeds, and close received ancillary descriptors if conversion fails. Stop using frozen filesystem encoding for the internal current-locale conversion helper. This addresses the runtime cause behind the Linux test_utf8_mode.UTF8ModeTests.test_cmd_line expected failure. No test code or decorators are changed because a RustPython runtime pass is not yet available. CPython reference Compared against CPython v3.15.0: Locale preconfiguration PEP 538 startup coercion Native locale conversion Filesystem codec initialization Validation Passed cargo check -p rustpython --lib --bin rustpython. Passed strict Clippy: cargo clippy -p rustpython-host_env -p rustpython-vm -p rustpython-stdlib -p rustpython --lib --bin rustpython -- -D warnings. Passed no-host VM compilation: cargo check -p rustpython-vm --no-default-features --features compiler. Normal commit hooks passed. An out-of-tree, bounded host-helper probe passed 40 decode/encode roundtrips across C, C.UTF-8, ISO-8859-1, ISO-8859-5 and GB2312, including embedded NUL and malformed/incomplete input. Snapshot stability after changing LC_CTYPE and typed encoding errors passed. Fifteen host-helper startup cases matched CPython 3.15.0 output and warning stderr exactly, including environment precedence, ignored Python controls, coercion warnings, invalid locale settings and LC_NUMERIC behavior. Unmodified CPython 3.15.0 test_cmd_line and test_filesystemencoding reference cases passed. Separate source review completed. No repository test additions, skips or expected-failure removals. The host-helper probe does not execute RustPython's VM or interpreter. Local RustPython runtime and aggregate workspace/C-API suites were not run because no verified runtime guard is available. This draft needs hosted CI and runtime review. Only the GNU Linux Rust target is installed; Windows/WASI call sites were source-reviewed but not cross-compiled. Scope and integration Non-Linux filesystem behavior is preserved. Non-GNU Linux supports existing native fast codecs and retains its previous UTF-8 behavior for wider CODESETs. This does not add arbitrary embedded filesystem-codec overrides, Windows legacy mbcs support, complete raw -m parsing, or non-UTF-8 getpath configuration strings. VirtualMachine::fsdecode and dependent Rust path-result helpers now return PyResult; embedding callers must propagate errors. check_initialization lets embedded callers inspect startup failures without diagnostics or process exit. This overlaps #9688 (native script filenames). A disposable composition of current main 9d657b02e4, #9688 and this patch passed cargo check -p rustpython --lib --bin rustpython after resolving the cant_open_file overlap and propagating four additional fsdecode results in src/lib.rs and crates/vm/src/vm/python_run.rs. Those merge-time adjustments are prepared separately; neither standalone branch includes the other PR. AI assistance This is maintainer-directed Python 3.15 upgrade work. OpenAI Codex assisted with investigation, implementation, source review and validation. Model version was not exposed. No human runtime validation is claimed. Summary by CodeRabbit Bug Fixes Filesystem path and locale-based text conversion errors are now reported across file, socket, and system operations instead of being silently ignored. Interpreter and subinterpreter startup now report initialization failures rather than continuing with incomplete setup. Socket message handling closes received file descriptors if address conversion fails. New Features Added locale-aware filesystem encoding support on Linux, including optional locale initialization during startup and improved handling of legacy locales.
Open Graph Description: Summary Initialize the Linux command-line locale with CPython 3.15's environment precedence and PEP 538 coercion. Keep embedded startup non-mutating; the executable explicitly opts into the un...
X Description: Summary Initialize the Linux command-line locale with CPython 3.15's environment precedence and PEP 538 coercion. Keep embedded startup non-mutating; the executable explicitly opts into th...
Opengraph URL: https://github.com/RustPython/RustPython/pull/9689
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:33289c32-190d-a8d4-a1ac-295e67b55c84 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | ECC0:25F5AB:EDF69:130092:6B01FE27 |
| html-safe-nonce | f66f91893e48bd30e0c9c90020bbebc320863382c1b2744ef96a4da5e70daf1d |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJFQ0MwOjI1RjVBQjpFREY2OToxMzAwOTI6NkIwMUZFMjciLCJ2aXNpdG9yX2lkIjoiOTg5OTEyODkwMzM1MTY1OTkxIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 3c2ee5679813a5b557e1ee74002682c4f3b747c0acb779ebc9e58796e94c0dd2 |
| hovercard-subject-tag | pull_request:5147864643 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/9689/files |
| twitter:image | https://avatars.githubusercontent.com/u/336763669?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/336763669?s=400&v=4 |
| og:image:alt | Summary Initialize the Linux command-line locale with CPython 3.15's environment precedence and PEP 538 coercion. Keep embedded startup non-mutating; the executable explicitly opts into the un... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width