Title: Implement basic REPL execution and fix piped stdin handling by youknowdot · Pull Request #9163 · RustPython/RustPython · GitHub
Open Graph Title: Implement basic REPL execution and fix piped stdin handling by youknowdot · Pull Request #9163 · RustPython/RustPython
X Title: Implement basic REPL execution and fix piped stdin handling by youknowdot · Pull Request #9163 · RustPython/RustPython
Description: Implement basic REPL execution and fix piped stdin handling What changes, and where to review Ordinary piped stdin runs as a script. The basic interactive REPL instead keeps reading until one input record is complete, executes it, reports its result or exception, and continues. This PR moves that evaluation into the VM and shares it with the CLI shell; the shell keeps editing/history and input transport. The EINTR follow-up lets a Python signal handler run while the basic reader is blocked on an open pipe. Suggested reading order: Lib/test/test_repl.py for input/traceback behavior, then crates/vm/src/vm/python_run.rs (run_interactive, run_basic_interactive, run_stdin_file). Script-stdin exit/error coverage also lives in test_cmd_line.py and test_runpy.py. extra_tests/snippets/stdlib_signal.py:check_native_stdin_eintr, then crates/host_env/src/readline.rs for the new interrupt regression and transport boundary. Lib/test/test_codeop.py and compiler interactive-input handling for warnings/future behavior. These existing tests do not directly cover every native record-boundary transaction. Review CAPI main-VM lifetime and unraisable-hook handling separately from stdin parsing, then the canonical Lib/updater changes. Summary Implement native stdin evaluation behind sys._baserepl and share it with the root shell. The shell retains line editing/history and input transport; the VM owns interactive record boundaries, compilation, execution and exception reporting. Preserve finite input boundaries, prompts, encoding, source locations and linecache identities. Collect interactive warnings and preserve callback exception ownership. Publish accepted future flags before later codegen errors, without publishing incomplete parse probes. Track mutable effective-inspect state while preserving coherent immutable sys.flags snapshots. Classify already-reported exceptions at the host boundary without printing twice; direct _baserepl SystemExit remains Python-catchable. Preserve the original main VM through C API execution/finalization and harden unraisable-hook fallback handling. Base and scope This validated candidate was built on historical main fd5b30eb5 plus the exact #9161 benchmark import fix bf076f10c. #9161 has since merged into main. The evaluator work is a narrow extraction related to #8954; it preserves main's AST/source-context architecture and initialization policy. Ten reviewable commits cover original-main CAPI lifetime, unraisable handling, warning collection, native evaluation, canonical REPL tests, compiler dev-dependency placement, explicit conditional-marker retirement, canonical stdin/audit tests, runpy attribution/capability markers, and native-reader EINTR propagation. Delta against #9161: 29 files, +2,058/-565. Including #9161 against the stated main: 30 files, +2,059/-566. CI repair and canonical tests Initial head 2955e91 had four failed CI jobs: Cargo-shear found the root compiler dependency was used only by benchmarks/examples. It is now a dev-dependency. Linux, macOS and Windows Python jobs all reported the same four unexpected successes: audit excepthook, two stdin tokenizer/decoding tests, and runpy stdin exit handling. Their obsolete markers are retired. These four tests now pass normally locally. Lib files were refreshed through the actual updater from pinned CPython 3.15.0 source, preserving applicable customizations. The closure includes the audit helper/empty fixtures and import helper. A small whole-file runpy refresh supplies the compile module name, making the new run_path warning-filter test pass; it also adopts canonical removal of __cached__ assignment. The import helper retains defaults for new parameters and adopts canonical cache-cleanup/import changes. Six newly imported tests remain expected failures and still execute their bodies: Three import-audit tests require absent _testcapi. Their conditional markers turn off whenever the module's spec is discoverable; these results do not establish import-audit semantics. test_dump_path_config: explicit PYTHONHOME startup selection/fatal missing-encodings behavior is unimplemented. test_import_time_unencodable_module_name: import-time tracing and surrogate-name import exception behavior remain unsupported. test_run_module_filter_syntax_warnings_by_module: frozen SourceLoader still omits the compile module name. Its whole canonical update requires coordinated importlib/abc/tests changes, deferred here. Validation Initial published candidate: c5f9224c22e1d6f0914d7ef7a605d05a6244c5b2; its hosted CI completed successfully with all 25 jobs. Follow-up candidate: 65cb4cb8d4917b82fcf98422d88a04e6b062e5b3. All commits ran normal hooks. Initial candidate results (follow-up gates are listed below): Default/release and workspace/CAPI test builds; cargo-shear; strict workspace and separate ABI3T CAPI Clippy passed. Workspace doctests: 24 passed, 18 existing ignored. CAPI doctest gate passed with zero runnable tests. Release regression packet: 27 tests, 20 normal passes and seven expected failures (six above plus one retained invalid-UTF8 marker). All four original unexpected successes and the new run_path warning-filter test pass normally. Actual release repository test_codeop: 13 tests, SUCCESS. Six exact finite-pipe test_repl filters: SUCCESS, including the retained return/yield expected failure. Sealed workspace native tests: 22 executables, 1,358 passed. Separate ABI3T CAPI: 123 passed and four existing ignored. No failures or timeouts. Fresh external canonical module admissions passed; 17/19 selected methods passed, with the two limitations below. Earlier feature checks passed on 2955e91; after the dependency move, c1f4e3 default, development-target, no-default and root-CAPI checks passed. Subsequent changes through c5f9224 affected Python/updater sources only; the EINTR follow-up changes the native reader and its caller. Builds use two jobs and CARGO_INCREMENTAL=0, with the repository release profile. Comparison and limits The fresh fixed admission plus 27 observations have clean integrity checks and agree with CPython 3.15.0 final on stdout/status. Six stderr differences remain: three incomplete-record parser diagnostics, and three suite cases affected by main's older traceback keyword-suggestion rendering. Fresh external canonical results were 17/19 selected methods passing: return/yield parser diagnostics and three duplicate-parameter wording subcases remained different. Existing repository codeop expectations use “duplicate argument” and pass. No manual assertion weakening, string postprocessing, vendor-parser workaround or partial Lib backport is included. A coherent traceback refresh is deferred. This validates finite pipes, not full terminal/PTY or Windows parity. Terminal signal delivery, explicit fork lifecycle, real memory exhaustion and excluded PTY/asyncio paths were not tested; the follow-up below adds bounded pipe-EINTR signal coverage. Existing startup helper limitations and main's absent relative-lazy-import warning remain outside this scope. The official reference reports CPython 3.15.0 final, with executable/libpython hashes pinned. Its exact build commit is unproven; canonical source is separately pinned to d220409a13b77de7a6c30abe16ea90db01528edc. Native reader EINTR follow-up A review found that Rust's BufRead::read_until retries Interrupted internally, preventing the basic reader from reaching VM signal dispatch while stdin remains open. The follow-up uses the same shared stdin buffer through fill_buf/consume, returning interrupts before retrying and releasing the input lock before Python handlers run. Its EOF guard preserves Rust's already accumulated bytes; this is not a claim of CPython partial-fgets parity. Three deterministic transport tests cover delimiter/remainder preservation, per-call byte counts, partial EOF, interruptions/retries and ordinary errors. A regression in the existing signal snippet uses an explicit native REPL over empty pipes: a returning SIGINT handler must acknowledge before any input is sent, then a complete sentinel line executes once and EOF exits zero. The exact bounded test passes on official CPython 3.15.0 and the fixed candidate; the prior published head times out before acknowledgment. Separate diagnostics remain nonpassing or outside this narrow fix: Default KeyboardInterrupt now promptly recovers and executes subsequent input, but the candidate exits zero whereas CPython remembers the reported interrupt and terminates with SIGINT (-2 as a POSIX subprocess return code). This finalization-policy difference is not fixed here. Direct sys._baserepl() on a noninteractive pipe selects file mode in CPython, whereas RustPython currently enters its basic interactive loop. The comparison explicitly supplies -i to both. CPython may discard bytes consumed by an interrupted partial libc fgets; the public regression intentionally supplies no input before handler acknowledgment. No terminal/PTY, Windows signal, or continuation-prompt coverage is claimed. Follow-up validation on 65cb4cb8: strict workspace and ABI3T CAPI Clippy; workspace/CAPI test builds; 24 workspace doctests (18 existing ignored); CAPI doctest gate (zero runnable tests); release build; 1,361 workspace native tests; 123 CAPI tests (four existing ignored); actual release codeop (13 tests) and four finite-pipe REPL methods; and the complete RustPython signal-snippet pytest node all passed. Each runtime gate used bounded, owned-process cleanup. Other review findings Two suggested behavior changes were not applied: Preserving the earlier command failure after a successful -i session would conflict with pinned CPython pymain_repl, which replaces that status with the REPL result. Raising OSError for every ordinary stdin read failure would conflict with the canonical raw tokenizer's EOF-like read-error handling and accumulated-input handling. Codec-backed input behaves differently; full codec/error parity remains unverified. AI assistance Implementation, review and validation were assisted by OpenAI Codex. Commits include Assisted-by: OpenAI Codex:model version unavailable trailers. Summary by CodeRabbit New Features Added interactive stdin execution with multiline input, continuation prompts, and support for future imports. Interactive startup commands and hooks run before the REPL when configured. Terminal sessions can use the enhanced REPL, and inspection mode can launch a REPL after execution. Bug Fixes Improved syntax-error locations, incomplete-input detection, and interactive warning locations. Improved uncaught-exception and traceback display, exit-code handling, and keyboard-interrupt behavior. Script file open failures now produce a clear error message and exit status.
Open Graph Description: Implement basic REPL execution and fix piped stdin handling What changes, and where to review Ordinary piped stdin runs as a script. The basic interactive REPL instead keeps reading until one input...
X Description: Implement basic REPL execution and fix piped stdin handling What changes, and where to review Ordinary piped stdin runs as a script. The basic interactive REPL instead keeps reading until one input...
Opengraph URL: https://github.com/RustPython/RustPython/pull/9163
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:bd161e32-50b0-fd62-d76f-b3e408df229a |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | C140:2C4BED:3912A21:4880BDC:6B01F65D |
| html-safe-nonce | 5bec09f7a28cce71723d87ffae8895463fe1435ab0e613bba94725914ec3b2ca |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJDMTQwOjJDNEJFRDozOTEyQTIxOjQ4ODBCREM6NkIwMUY2NUQiLCJ2aXNpdG9yX2lkIjoiODI0MDIxODI3MjI2NTY0MTg5IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 16ae616d759e4d333fd98b4255099f18beebdc9369b4c0166c476fe1aa0f6f43 |
| hovercard-subject-tag | pull_request:4884721945 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/9163/files |
| twitter:image | https://avatars.githubusercontent.com/u/336763669?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/336763669?s=400&v=4 |
| og:image:alt | Implement basic REPL execution and fix piped stdin handling What changes, and where to review Ordinary piped stdin runs as a script. The basic interactive REPL instead keeps reading until one input... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width