Title: common: port the lzma engine to xz-core by youknowone · Pull Request #8639 · RustPython/RustPython · GitHub
Open Graph Title: common: port the lzma engine to xz-core by youknowone · Pull Request #8639 · RustPython/RustPython
X Title: common: port the lzma engine to xz-core by youknowone · Pull Request #8639 · RustPython/RustPython
Description: Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Rust xz-core port of liblzma rather than the xz / xz-sys bindings to the C library, and drop the android / wasm32 cfg that the C dependency carried. Three commits: lzma: export FILTERS_MAX and raise ValueError — LZMA_FILTERS_MAX becomes pub const FILTERS_MAX, so parse_filter_chain stops restating the value in a local const of its own. The over-long filter chain is reported as ValueError rather than LZMAError; parse_filter_chain_spec in Modules/_lzmamodule.c raises PyExc_ValueError, and CPython 3.14 agrees: $ python3 -c "import lzma; lzma.LZMACompressor(format=lzma.FORMAT_RAW, filters=[{'id':999}]*5)" ValueError: Too many filters - liblzma supports a maximum of 4 That measurement also pins the ordering: five specs each carrying an invalid id report the length, not the id, so the length check stays ahead of the parse loop. extra_tests/snippets/stdlib_lzma.py now covers both. common: port the lzma engine to xz-core — the port itself. test_lzma: drop three passing expectedFailure markers — test_decompressor_chunks_empty, test_decompressor_chunks_maxsize and test_issue21872 pass on the ported engine. An unexpected success ends the suite as FAILED, so their markers go with the port. The other seven TODO: RUSTPYTHON markers in that file still fail and stay. test_issue21872 is worth calling out, because its marker read AssertionError: True is not false. Modules/_lzmamodule.c splits the avail_in == 0 case in two: else if (lzs->avail_in == 0) { lzs->next_in = NULL; if (lzs->avail_out == 0) { /* (avail_in==0 && avail_out==0) Maybe lzs's internal state still have a few bytes can be output, try to output them next time. */ d->needs_input = 0; } else { d->needs_input = 1; } } The ported engine carries that second arm, which is what the test pins. The [patch.crates-io] pin xz-core 0.1.0-rc.0 as published registers a lzma12_optmap static initializer. On MSVC it lands in .CRT$XIB, which _initterm_e walks as int (__cdecl *)(void) while the initializer's Rust signature returns (), so a Windows build exits 255 with empty stdout and stderr. That release also overflows the alone_decoder dictionary-size check in a debug build for dict_size == 0. simnalamburt/xz-rs#21 fixes both and has merged, so the pin is the upstream commit carrying it, 5bf9541, and not a fork. It drops once a release carrying that commit reaches crates.io. Validation Run on macOS aarch64: cargo check -p rustpython-common --features lzma cargo check -p rustpython-stdlib cargo test -p rustpython-common --features lzma — 76 passed, 0 failed cargo clippy --all-targets -- -D warnings cargo build --release ./target/release/rustpython -m test test_lzma -v — run=121, skipped=1, 7 expected failures, 0 unexpected successes, SUCCESS ./target/release/rustpython extra_tests/snippets/stdlib_lzma.py The Cargo.lock entry is worth checking on review: xz-core must resolve to git+https://github.com/simnalamburt/xz-rs?rev=5bf9541..., and [[patch.unused]] must be absent. If either is not so, the published crate is being built and the Windows failure above is still present. AI disclosure The engine port was written by Grok (grok-4.6) from a written specification. Review covered the Decompressor state transitions against Modules/_lzmamodule.c, the lc/lp/pb validation and FORMAT_ALONE filter forwarding added in #8631 (both carried over by the port), the exception type and message for every catch_lzma_error return value, and the checks listed above. Summary by CodeRabbit New Features LZMA compression and decompression are now available on Android and WebAssembly targets when enabled. Improved compatibility and reliability across supported platforms. Bug Fixes Fixed Windows MSVC initialization issues and debug-build dictionary sizing. Improved handling of LZMA filter chains, including clearer errors when too many filters are provided. Updated filter-chain limits to remain consistent with the compression backend. Tests Added coverage for invalid and oversized raw LZMA filter chains.
Open Graph Description: Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Ru...
X Description: Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Ru...
Opengraph URL: https://github.com/RustPython/RustPython/pull/8639
X: @github
Domain: github.com
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository/pull/:id/files(.:format) |
| route-controller | pull_requests |
| route-action | files |
| fetch-nonce | v2:6bccbdad-86c9-009d-e410-d739eebc9f35 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | B38A:317608:3BF4E89:4BE79A9:6B01FE49 |
| html-safe-nonce | 2392fbddc97bac2684cf399a533d79597a8306df6aaaeea58541a10fde546e11 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMzhBOjMxNzYwODozQkY0RTg5OjRCRTc5QTk6NkIwMUZFNDkiLCJ2aXNpdG9yX2lkIjoiNzIxNjcxNTk5NjAwOTIwMTIyNSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 83ec45c9829fc0d030de3162f2175710d82a263195d2f84e820bd6692b029fa1 |
| hovercard-subject-tag | pull_request:4421241383 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/8639/files |
| twitter:image | https://avatars.githubusercontent.com/u/69878?s=400&v=4 |
| twitter:card | summary_large_image |
| og:image | https://avatars.githubusercontent.com/u/69878?s=400&v=4 |
| og:image:alt | Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Ru... |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| diff-view | unified |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive full-width |
| disable-turbo | true |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width