René's URL Explorer Experiment


Title: bpo-32728: Add compresslevel support for zipfile and LZMA by bbayles · Pull Request #5534 · python/cpython · GitHub

Open Graph Title: bpo-32728: Add compresslevel support for zipfile and LZMA by bbayles · Pull Request #5534 · python/cpython

X Title: bpo-32728: Add compresslevel support for zipfile and LZMA by bbayles · Pull Request #5534 · python/cpython

Description: This PR extends the compresslevel support added to zipfile.ZipFile in #5385 to the LZMA compression method. Currently ZIP_DEFLATED and ZIP_BZ2 can be used with compresslevel, but ZIP_LZMA can't. zipfile currently defines a custom LZMACompressor instead of using lzma.LZMACompressor. This is so the parameters needed for the "LZMA Properties Header" in the ZIP file can be be written. This custom compressor sets the LZMA dict_size to 8 MiB, but leaves the other compression parameters (those determined by LZMA's presets) unspecified. Here I have added a dictionary that maps the presets (0 through 9, plus 0 through 9 OR-ed with lzma.PRESET_EXTREME (see these docs) to their compression parameters. See here for what each compression parameter means. The values can be be mostly assembled from the xz man page (example), but see also liblzma's lzma_lzma_preset function. Once we have the compression parameters for the selected preset, then we write the header such that the ZIP file can be decoded later. I've added a few comments about this in the code; the current process is a bit unclear if you don't have the ZIP spec memorized. One can see from this script that the compression results are different when using the keyword (default behavior does not change). I've also added a test that checks that the correct property header is written to the ZIP file; this we can do with LZMA much more easily than with zlib or bzip2. It may be noted that tarfile has a way of setting an LZMA archive's preset with a preset keyword (instead of compresslevel), but I don't think this is worth bringing here - (a) that capability is currently undocumented in tarfile; (b) The zipfile and tarfile modules already differ substantially (e.g., write() vs. add() methods); (c) it seems a bit wrong to make people remember that zlib and bz2 use compresslevel but lzma uses preset in this module. https://bugs.python.org/issue32728

Open Graph Description: This PR extends the compresslevel support added to zipfile.ZipFile in #5385 to the LZMA compression method. Currently ZIP_DEFLATED and ZIP_BZ2 can be used with compresslevel, but ZIP_LZMA can't...

X Description: This PR extends the compresslevel support added to zipfile.ZipFile in #5385 to the LZMA compression method. Currently ZIP_DEFLATED and ZIP_BZ2 can be used with compresslevel, but ZIP_LZMA can&#...

Opengraph URL: https://github.com/python/cpython/pull/5534

X: @github

direct link

Domain: github.com

route-pattern/:user_id/:repository/pull/:id/files(.:format)
route-controllerpull_requests
route-actionfiles
fetch-noncev2:7c75220e-d7a8-09c2-8853-46ce1403bb26
current-catalog-service-hashae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b
request-id9B46:322E35:2816E57:37A616C:6A4F5503
html-safe-nonce8d36940f6797cb3e03aae008b163c6cc7631fc3c5879f04c15bf17a7fda9b986
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5QjQ2OjMyMkUzNToyODE2RTU3OjM3QTYxNkM6NkE0RjU1MDMiLCJ2aXNpdG9yX2lkIjoiNTEzMzMxMTkxMTQzNDA4MTUzOSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac458ed3be881618014027059e94055b906df11569c8a2b6f63613de076b5acb03
hovercard-subject-tagpull_request:167025928
github-keyboard-shortcutsrepository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///pull_requests/show/files
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/python/cpython/pull/5534/files
twitter:imagehttps://avatars.githubusercontent.com/u/1922815?s=400&v=4
twitter:cardsummary_large_image
og:imagehttps://avatars.githubusercontent.com/u/1922815?s=400&v=4
og:image:altThis PR extends the compresslevel support added to zipfile.ZipFile in #5385 to the LZMA compression method. Currently ZIP_DEFLATED and ZIP_BZ2 can be used with compresslevel, but ZIP_LZMA can't...
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
Noneb92d11c0aa4a77d54ef4af1078b6a15fb5a70a215b30c4ecf28889d5a8e656d9
turbo-cache-controlno-preview
diff-viewunified
go-importgithub.com/python/cpython git https://github.com/python/cpython.git
octolytics-dimension-user_id1525981
octolytics-dimension-user_loginpython
octolytics-dimension-repository_id81598961
octolytics-dimension-repository_nwopython/cpython
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forkfalse
octolytics-dimension-repository_network_root_id81598961
octolytics-dimension-repository_network_root_nwopython/cpython
turbo-body-classeslogged-out env-production page-responsive full-width
disable-turbotrue
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release4b249b445842943ed31549e027f57a8ade9881ed
ui-targetfull
theme-color#1e2327
color-schemelight dark

Links:

Skip to contenthttps://github.com/python/cpython/pull/5534/files#start-of-content
https://github.com/
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F5534%2Ffiles
GitHub CopilotWrite better code with AIhttps://github.com/features/copilot
GitHub Copilot appDirect agents from issue to mergehttps://github.com/features/ai/github-app
MCP RegistryNewIntegrate external toolshttps://github.com/mcp
ActionsAutomate any workflowhttps://github.com/features/actions
CodespacesInstant dev environmentshttps://github.com/features/codespaces
IssuesPlan and track workhttps://github.com/features/issues
Code ReviewManage code changeshttps://github.com/features/code-review
GitHub Advanced SecurityFind and fix vulnerabilitieshttps://github.com/security/advanced-security
Code securitySecure your code as you buildhttps://github.com/security/advanced-security/code-security
Secret protectionStop leaks before they starthttps://github.com/security/advanced-security/secret-protection
Why GitHubhttps://github.com/why-github
Documentationhttps://docs.github.com
Bloghttps://github.blog
Changeloghttps://github.blog/changelog
Marketplacehttps://github.com/marketplace
View all featureshttps://github.com/features
Enterpriseshttps://github.com/enterprise
Small and medium teamshttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
App Modernizationhttps://github.com/solutions/use-case/app-modernization
DevSecOpshttps://github.com/solutions/use-case/devsecops
DevOpshttps://github.com/solutions/use-case/devops
CI/CDhttps://github.com/solutions/use-case/ci-cd
View all use caseshttps://github.com/solutions/use-case
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
View all industrieshttps://github.com/solutions/industry
View all solutionshttps://github.com/solutions
AIhttps://github.com/resources/articles?topic=ai
Software Developmenthttps://github.com/resources/articles?topic=software-development
DevOpshttps://github.com/resources/articles?topic=devops
Securityhttps://github.com/resources/articles?topic=security
View all topicshttps://github.com/resources/articles
Customer storieshttps://github.com/customer-stories
Events & webinarshttps://github.com/resources/events
Ebooks & reportshttps://github.com/resources/whitepapers
Business insightshttps://github.com/solutions/executive-insights
GitHub Skillshttps://skills.github.com
Documentationhttps://docs.github.com
Customer supporthttps://support.github.com
Community forumhttps://github.com/orgs/community/discussions
Trust centerhttps://github.com/trust-center
Partnershttps://github.com/partners
View all resourceshttps://github.com/resources
GitHub SponsorsFund open source developershttps://github.com/open-source/sponsors
Security Labhttps://securitylab.github.com
Maintainer Communityhttps://maintainers.github.com
Acceleratorhttps://github.com/open-source/accelerator
GitHub Starshttps://stars.github.com
Archive Programhttps://archiveprogram.github.com
Topicshttps://github.com/topics
Trendinghttps://github.com/trending
Collectionshttps://github.com/collections
Enterprise platformAI-powered developer platformhttps://github.com/enterprise
GitHub Advanced SecurityEnterprise-grade security featureshttps://github.com/security/advanced-security
Copilot for BusinessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Pricinghttps://github.com/pricing
Search syntax tipshttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
documentationhttps://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax
Sign in https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fpython%2Fcpython%2Fpull%2F5534%2Ffiles
Sign up https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fpull_requests%2Fshow%2Ffiles&source=header-repo&source_repo=python%2Fcpython
Reloadhttps://github.com/python/cpython/pull/5534/files
Reloadhttps://github.com/python/cpython/pull/5534/files
Reloadhttps://github.com/python/cpython/pull/5534/files
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
python https://github.com/python
cpythonhttps://github.com/python/cpython
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
Notifications https://github.com/login?return_to=%2Fpython%2Fcpython
Fork 34.8k https://github.com/login?return_to=%2Fpython%2Fcpython
Star 73.6k https://github.com/login?return_to=%2Fpython%2Fcpython
Code https://github.com/python/cpython
Issues 5k+ https://github.com/python/cpython/issues
Pull requests 2.3k https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects https://github.com/python/cpython/projects
Security and quality 0 https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Code https://github.com/python/cpython
Issues https://github.com/python/cpython/issues
Pull requests https://github.com/python/cpython/pulls
Actions https://github.com/python/cpython/actions
Projects https://github.com/python/cpython/projects
Security and quality https://github.com/python/cpython/security
Insights https://github.com/python/cpython/pulse
Sign up for GitHub https://github.com/signup?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
terms of servicehttps://docs.github.com/terms
privacy statementhttps://docs.github.com/privacy
Sign inhttps://github.com/login?return_to=%2Fpython%2Fcpython%2Fissues%2Fnew%2Fchoose
bbayleshttps://github.com/bbayles
python:masterhttps://github.com/python/cpython/tree/master
bbayles:zipfile-lzma-presetshttps://github.com/bbayles/cpython/tree/zipfile-lzma-presets
Conversation 12 https://github.com/python/cpython/pull/5534
Commits 11 https://github.com/python/cpython/pull/5534/commits
Checks 0 https://github.com/python/cpython/pull/5534/checks
Files changed https://github.com/python/cpython/pull/5534/files
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
bpo-32728: Add compresslevel support for zipfile and LZMA https://github.com/python/cpython/pull/5534/files#top
Show all changes 11 commits https://github.com/python/cpython/pull/5534/files
63a9350 Add compresslevel support for LZMA bbayles Jan 30, 2018 https://github.com/python/cpython/pull/5534/commits/63a9350c973816b9fcbfc9671926d488e229acb8
4ca4ba6 Add NEWS entry bbayles Feb 4, 2018 https://github.com/python/cpython/pull/5534/commits/4ca4ba64b7c6df33c2b67800984aca635d921c46
47d7dbc Add docs entry for lzma compresslevel bbayles Feb 4, 2018 https://github.com/python/cpython/pull/5534/commits/47d7dbc8b1b936796d4c23798dff3670f4414cd0
c97397a Merge branch 'master' into zipfile-lzma-presets bbayles Sep 1, 2018 https://github.com/python/cpython/pull/5534/commits/c97397a071468780027a2682d95dfaa4e493da1e
3218ba6 Merge remote-tracking branch 'upstream/master' into zipfile-lzma-presets bbayles Aug 5, 2019 https://github.com/python/cpython/pull/5534/commits/3218ba6da53b9b176ee350ee2ea45f6d8c5bbd22
916be26 Merge remote-tracking branch 'upstream/master' into zipfile-lzma-presets bbayles Aug 5, 2019 https://github.com/python/cpython/pull/5534/commits/916be2600ca1aaeeaa3db0727dbbb67c5f6a5d4e
37b8945 Update Doc/library/zipfile.rst bbayles Aug 5, 2019 https://github.com/python/cpython/pull/5534/commits/37b8945188930f8e825e6bcfb4a07e4d14c92202
3ed3f24 Update Misc/NEWS.d/next/Library/2018-02-04-13-18-14.bpo-32728.2u0pLO.rst bbayles Aug 5, 2019 https://github.com/python/cpython/pull/5534/commits/3ed3f2495d6386f30cd3101059c51b0c997ae734
b7499b0 Adjust punctuation bbayles Aug 5, 2019 https://github.com/python/cpython/pull/5534/commits/b7499b06da2dab7c83edbf1b2174df93c4348a5f
978bf3f Remove collections import bbayles Aug 6, 2019 https://github.com/python/cpython/pull/5534/commits/978bf3feca99e5d54453236b41a4ac0baae36715
538e956 Prettify dict_size bbayles Aug 6, 2019 https://github.com/python/cpython/pull/5534/commits/538e9566f2f6fb02a2d4621c34469ad2ee14a0fc
Clear filters https://github.com/python/cpython/pull/5534/files
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
zipfile.rst https://github.com/python/cpython/pull/5534/files#diff-8a20c37fb313d7b188ae15bb57b351cc2ab73cbdcb6b2aee44cf2f8451768086
test_zipfile.py https://github.com/python/cpython/pull/5534/files#diff-88215aededc3f81c55ba2432fe67617be9fc3faff9cb10e0c01f710adfcba426
zipfile.py https://github.com/python/cpython/pull/5534/files#diff-b5cf7374d79aae191c1e38d0959527e0bbb7a95e0df5b125a8b1056cc2c54851
2018-02-04-13-18-14.bpo-32728.2u0pLO.rst https://github.com/python/cpython/pull/5534/files#diff-6076fa9d16f489b670e03043d2a3751cd8d28a35017e1c96aff10a01f508ae5f
Doc/library/zipfile.rsthttps://github.com/python/cpython/pull/5534/files#diff-8a20c37fb313d7b188ae15bb57b351cc2ab73cbdcb6b2aee44cf2f8451768086
View file https://github.com/bbayles/cpython/blob/538e9566f2f6fb02a2d4621c34469ad2ee14a0fc/Doc/library/zipfile.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/5534/{{ revealButtonHref }}
https://github.com/python/cpython/pull/5534/files#diff-8a20c37fb313d7b188ae15bb57b351cc2ab73cbdcb6b2aee44cf2f8451768086
https://github.com/python/cpython/pull/5534/files#diff-8a20c37fb313d7b188ae15bb57b351cc2ab73cbdcb6b2aee44cf2f8451768086
https://github.com/python/cpython/pull/5534/files#diff-8a20c37fb313d7b188ae15bb57b351cc2ab73cbdcb6b2aee44cf2f8451768086
epicfaacehttps://github.com/epicfaace
Aug 27, 2019https://github.com/python/cpython/pull/5534/files#r317887561
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
https://github.com/python/cpython/pull/5534/files#diff-8a20c37fb313d7b188ae15bb57b351cc2ab73cbdcb6b2aee44cf2f8451768086
Lib/test/test_zipfile.pyhttps://github.com/python/cpython/pull/5534/files#diff-88215aededc3f81c55ba2432fe67617be9fc3faff9cb10e0c01f710adfcba426
View file https://github.com/bbayles/cpython/blob/538e9566f2f6fb02a2d4621c34469ad2ee14a0fc/Lib/test/test_zipfile.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/5534/{{ revealButtonHref }}
https://github.com/python/cpython/pull/5534/files#diff-88215aededc3f81c55ba2432fe67617be9fc3faff9cb10e0c01f710adfcba426
ZackerySpytzhttps://github.com/ZackerySpytz
Aug 5, 2019https://github.com/python/cpython/pull/5534/files#r310813438
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
bbayleshttps://github.com/bbayles
Aug 5, 2019https://github.com/python/cpython/pull/5534/files#r310828879
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
https://github.com/python/cpython/pull/5534/files#diff-88215aededc3f81c55ba2432fe67617be9fc3faff9cb10e0c01f710adfcba426
Lib/zipfile.pyhttps://github.com/python/cpython/pull/5534/files#diff-b5cf7374d79aae191c1e38d0959527e0bbb7a95e0df5b125a8b1056cc2c54851
View file https://github.com/bbayles/cpython/blob/538e9566f2f6fb02a2d4621c34469ad2ee14a0fc/Lib/zipfile.py
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/5534/{{ revealButtonHref }}
https://github.com/python/cpython/pull/5534/files#diff-b5cf7374d79aae191c1e38d0959527e0bbb7a95e0df5b125a8b1056cc2c54851
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
ZackerySpytzhttps://github.com/ZackerySpytz
Aug 5, 2019https://github.com/python/cpython/pull/5534/files#r310815088
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
bbayleshttps://github.com/bbayles
Aug 5, 2019https://github.com/python/cpython/pull/5534/files#r310829138
Learn morehttps://docs.github.com/articles/managing-disruptive-comments/#hiding-a-comment
Please reload this pagehttps://github.com/python/cpython/pull/5534/files
https://github.com/python/cpython/pull/5534/files#diff-b5cf7374d79aae191c1e38d0959527e0bbb7a95e0df5b125a8b1056cc2c54851
https://github.com/python/cpython/pull/5534/files#diff-b5cf7374d79aae191c1e38d0959527e0bbb7a95e0df5b125a8b1056cc2c54851
https://github.com/python/cpython/pull/5534/files#diff-b5cf7374d79aae191c1e38d0959527e0bbb7a95e0df5b125a8b1056cc2c54851
Misc/NEWS.d/next/Library/2018-02-04-13-18-14.bpo-32728.2u0pLO.rsthttps://github.com/python/cpython/pull/5534/files#diff-6076fa9d16f489b670e03043d2a3751cd8d28a35017e1c96aff10a01f508ae5f
View file https://github.com/bbayles/cpython/blob/538e9566f2f6fb02a2d4621c34469ad2ee14a0fc/Misc/NEWS.d/next/Library/2018-02-04-13-18-14.bpo-32728.2u0pLO.rst
Open in desktop https://desktop.github.com
https://github.co/hiddenchars
https://github.com/python/cpython/pull/5534/{{ revealButtonHref }}
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width

URLs of crawlers that visited me.