René's URL Explorer Experiment


Title: common: port the lzma engine to xz-core by youknowone · Pull Request #8639 · RustPython/RustPython · GitHub

Open Graph Title: common: port the lzma engine to xz-core by youknowone · Pull Request #8639 · RustPython/RustPython

X Title: common: port the lzma engine to xz-core by youknowone · Pull Request #8639 · RustPython/RustPython

Description: Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Rust xz-core port of liblzma rather than the xz / xz-sys bindings to the C library, and drop the android / wasm32 cfg that the C dependency carried. Three commits: lzma: export FILTERS_MAX and raise ValueError — LZMA_FILTERS_MAX becomes pub const FILTERS_MAX, so parse_filter_chain stops restating the value in a local const of its own. The over-long filter chain is reported as ValueError rather than LZMAError; parse_filter_chain_spec in Modules/_lzmamodule.c raises PyExc_ValueError, and CPython 3.14 agrees: $ python3 -c "import lzma; lzma.LZMACompressor(format=lzma.FORMAT_RAW, filters=[{'id':999}]*5)" ValueError: Too many filters - liblzma supports a maximum of 4 That measurement also pins the ordering: five specs each carrying an invalid id report the length, not the id, so the length check stays ahead of the parse loop. extra_tests/snippets/stdlib_lzma.py now covers both. common: port the lzma engine to xz-core — the port itself. test_lzma: drop three passing expectedFailure markers — test_decompressor_chunks_empty, test_decompressor_chunks_maxsize and test_issue21872 pass on the ported engine. An unexpected success ends the suite as FAILED, so their markers go with the port. The other seven TODO: RUSTPYTHON markers in that file still fail and stay. test_issue21872 is worth calling out, because its marker read AssertionError: True is not false. Modules/_lzmamodule.c splits the avail_in == 0 case in two: else if (lzs->avail_in == 0) { lzs->next_in = NULL; if (lzs->avail_out == 0) { /* (avail_in==0 && avail_out==0) Maybe lzs's internal state still have a few bytes can be output, try to output them next time. */ d->needs_input = 0; } else { d->needs_input = 1; } } The ported engine carries that second arm, which is what the test pins. The [patch.crates-io] pin xz-core 0.1.0-rc.0 as published registers a lzma12_optmap static initializer. On MSVC it lands in .CRT$XIB, which _initterm_e walks as int (__cdecl *)(void) while the initializer's Rust signature returns (), so a Windows build exits 255 with empty stdout and stderr. That release also overflows the alone_decoder dictionary-size check in a debug build for dict_size == 0. simnalamburt/xz-rs#21 fixes both and has merged, so the pin is the upstream commit carrying it, 5bf9541, and not a fork. It drops once a release carrying that commit reaches crates.io. Validation Run on macOS aarch64: cargo check -p rustpython-common --features lzma cargo check -p rustpython-stdlib cargo test -p rustpython-common --features lzma — 76 passed, 0 failed cargo clippy --all-targets -- -D warnings cargo build --release ./target/release/rustpython -m test test_lzma -v — run=121, skipped=1, 7 expected failures, 0 unexpected successes, SUCCESS ./target/release/rustpython extra_tests/snippets/stdlib_lzma.py The Cargo.lock entry is worth checking on review: xz-core must resolve to git+https://github.com/simnalamburt/xz-rs?rev=5bf9541..., and [[patch.unused]] must be absent. If either is not so, the published crate is being built and the Windows failure above is still present. AI disclosure The engine port was written by Grok (grok-4.6) from a written specification. Review covered the Decompressor state transitions against Modules/_lzmamodule.c, the lc/lp/pb validation and FORMAT_ALONE filter forwarding added in #8631 (both carried over by the port), the exception type and message for every catch_lzma_error return value, and the checks listed above. Summary by CodeRabbit New Features LZMA compression and decompression are now available on Android and WebAssembly targets when enabled. Improved compatibility and reliability across supported platforms. Bug Fixes Fixed Windows MSVC initialization issues and debug-build dictionary sizing. Improved handling of LZMA filter chains, including clearer errors when too many filters are provided. Updated filter-chain limits to remain consistent with the compression backend. Tests Added coverage for invalid and oversized raw LZMA filter chains.

Open Graph Description: Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Ru...

X Description: Closes #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Ru...

Opengraph URL: https://github.com/RustPython/RustPython/pull/8639

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:6bccbdad-86c9-009d-e410-d739eebc9f35
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-idB38A:317608:3BF4E89:4BE79A9:6B01FE49
html-safe-nonce2392fbddc97bac2684cf399a533d79597a8306df6aaaeea58541a10fde546e11
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCMzhBOjMxNzYwODozQkY0RTg5OjRCRTc5QTk6NkIwMUZFNDkiLCJ2aXNpdG9yX2lkIjoiNzIxNjcxNTk5NjAwOTIwMTIyNSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac83ec45c9829fc0d030de3162f2175710d82a263195d2f84e820bd6692b029fa1
hovercard-subject-tagpull_request:4421241383
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/RustPython/RustPython/pull/8639/files
twitter:imagehttps://avatars.githubusercontent.com/u/69878?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/69878?s=400&v=4
og:image:altCloses #xxxx One of checkbox below must be checked. I did not use AI to write the code of this patch. This PR follows our AI policy Summary Drive common/compression/lzma.rs through the pure-Ru...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git
octolytics-dimension-user_id39710557
octolytics-dimension-user_loginRustPython
octolytics-dimension-repository_id135201145
octolytics-dimension-repository_nwoRustPython/RustPython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id135201145
octolytics-dimension-repository_network_root_nwoRustPython/RustPython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release423dbb54c5e7bfecc838eb387c1435eb3d0e760d
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/RustPython/RustPython/pull/8639/files#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F8639%2Ffiles
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2FRustPython%2FRustPython%2Fpull%2F8639%2Ffiles
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=RustPython%2FRustPython
Reloadhttps://github.com/RustPython/RustPython/pull/8639/files
Reloadhttps://github.com/RustPython/RustPython/pull/8639/files
Reloadhttps://github.com/RustPython/RustPython/pull/8639/files
RustPythonhttps://github.com/RustPython
RustPythonhttps://github.com/RustPython/RustPython
Codehttps://github.com/RustPython/RustPython
Issues195 (195)https://github.com/RustPython/RustPython/issues
Pull requests133 (133)https://github.com/RustPython/RustPython/pulls
Discussionshttps://github.com/RustPython/RustPython/discussions
Actionshttps://github.com/RustPython/RustPython/actions
Projectshttps://github.com/RustPython/RustPython/projects
Wikihttps://github.com/RustPython/RustPython/wiki
Security and qualityhttps://github.com/RustPython/RustPython/security
Insightshttps://github.com/RustPython/RustPython/insights
Sign up for GitHub https://github.com/signup?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2FRustPython%2FRustPython%2Fissues%2Fnew%2Fchoose
youknowonehttps://github.com/youknowone
RustPython:mainhttps://github.com/RustPython/RustPython/tree/main
youknowone:lzma-xz-corehttps://github.com/youknowone/RustPython/tree/lzma-xz-core
Conversation 7 https://github.com/RustPython/RustPython/pull/8639
Commits 3 https://github.com/RustPython/RustPython/pull/8639/commits
Checks 1 https://github.com/RustPython/RustPython/pull/8639/checks
Files changed https://github.com/RustPython/RustPython/pull/8639/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/8639/files
common: port the lzma engine to xz-core https://github.com/RustPython/RustPython/pull/8639/files#top
Show all changes 3 commits https://github.com/RustPython/RustPython/pull/8639/files
c413d01 lzma: export FILTERS_MAX and raise ValueError youknowone Sep 1, 2026 https://github.com/RustPython/RustPython/pull/8639/commits/c413d0121d4f029032c6b349319e7d04121f818b
0474123 common: port the lzma engine to xz-core youknowone Sep 1, 2026 https://github.com/RustPython/RustPython/pull/8639/commits/0474123269721313f6aee2b415d71228921b5940
4854c86 test_lzma: drop three passing expectedFailure markers youknowone Sep 1, 2026 https://github.com/RustPython/RustPython/pull/8639/commits/4854c8666af8b1ecce1e3bef6c010e4ba22d652c
Clear filters https://github.com/RustPython/RustPython/pull/8639/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/8639/files
Please reload this pagehttps://github.com/RustPython/RustPython/pull/8639/files
Cargo.lock https://github.com/RustPython/RustPython/pull/8639/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87e
Cargo.toml https://github.com/RustPython/RustPython/pull/8639/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
test_lzma.py https://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
Cargo.toml https://github.com/RustPython/RustPython/pull/8639/files#diff-67449309128bb90a86efa4567e238536ea0f347ae18098f8dbcd747de5b7b4ca
lzma.rs https://github.com/RustPython/RustPython/pull/8639/files#diff-83341d9b777a346e4c3fb79c3c2cb0165ca319852150f8bc3842aa4469b1f102
mod.rs https://github.com/RustPython/RustPython/pull/8639/files#diff-f2d3c2ab43fccf4482a1f36427b611c4f7df39bc55f8cbaeaa1cc63e6f3e4fd8
lzma.rs https://github.com/RustPython/RustPython/pull/8639/files#diff-a6ce94ca49cec269e515f26f28731b5be2cf4b61942a9063906f23b16d9ef171
stdlib_lzma.py https://github.com/RustPython/RustPython/pull/8639/files#diff-c7832ad440298a8cb1f7970398c409df25ae1898f8d3d4096c8559881de033e0
Cargo.lockhttps://github.com/RustPython/RustPython/pull/8639/files#diff-13ee4b2252c9e516a0547f2891aa2105c3ca71c6d7a1e682c69be97998dfc87e
View file https://github.com/RustPython/RustPython/blob/4854c8666af8b1ecce1e3bef6c010e4ba22d652c/Cargo.lock
Open in desktop https://desktop.github.com
how customized files appear on GitHubhttps://docs.github.com/github/administering-a-repository/customizing-how-changed-files-appear-on-github
Please reload this pagehttps://github.com/RustPython/RustPython/pull/8639/files
Cargo.tomlhttps://github.com/RustPython/RustPython/pull/8639/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
View file https://github.com/RustPython/RustPython/blob/4854c8666af8b1ecce1e3bef6c010e4ba22d652c/Cargo.toml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/8639/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/8639/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
https://github.com/RustPython/RustPython/pull/8639/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
https://github.com/RustPython/RustPython/pull/8639/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
https://github.com/RustPython/RustPython/pull/8639/files#diff-2e9d962a08321605940b5a657135052fbcef87b5e360662bb527c96d9a615542
Lib/test/test_lzma.pyhttps://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
View file https://github.com/RustPython/RustPython/blob/4854c8666af8b1ecce1e3bef6c010e4ba22d652c/Lib/test/test_lzma.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/8639/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
https://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
https://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
https://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
https://github.com/RustPython/RustPython/pull/8639/files#diff-c0d354f43bb17a1ab769897c04ad70a9d5364b876cde66a0fa6ebf50f7db1fb6
crates/common/Cargo.tomlhttps://github.com/RustPython/RustPython/pull/8639/files#diff-67449309128bb90a86efa4567e238536ea0f347ae18098f8dbcd747de5b7b4ca
View file https://github.com/RustPython/RustPython/blob/4854c8666af8b1ecce1e3bef6c010e4ba22d652c/crates/common/Cargo.toml
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/RustPython/RustPython/pull/8639/{{ revealButtonHref }}
https://github.com/RustPython/RustPython/pull/8639/files#diff-67449309128bb90a86efa4567e238536ea0f347ae18098f8dbcd747de5b7b4ca
https://github.com/RustPython/RustPython/pull/8639/files#diff-67449309128bb90a86efa4567e238536ea0f347ae18098f8dbcd747de5b7b4ca
https://github.com/RustPython/RustPython/pull/8639/files#diff-67449309128bb90a86efa4567e238536ea0f347ae18098f8dbcd747de5b7b4ca
Please reload this pagehttps://github.com/RustPython/RustPython/pull/8639/files
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.