Title: `zlib.decompressobj`: `decompress(b"", n)` and `flush()` never return output that inflate still holds (raw deflate loses its last bytes under `max_length`) · Issue #8907 · RustPython/RustPython · GitHub
Open Graph Title: `zlib.decompressobj`: `decompress(b"", n)` and `flush()` never return output that inflate still holds (raw deflate loses its last bytes under `max_length`) · Issue #8907 · RustPython/RustPython
X Title: `zlib.decompressobj`: `decompress(b"", n)` and `flush()` never return output that inflate still holds (raw deflate loses its last bytes under `max_length`) · Issue #8907 · RustPython/RustPython
Description: Environment RustPython d31ccae04d039ca3ab294742f2493aa5f2b63ff7, host build (x86_64-unknown-linux-gnu, debug), zlib.ZLIB_VERSION = 1.3.0-zlib-rs-0.6.8. crates/common/src/compression/zlib.rs on main (a7d75d25a7b2c291720411dc65445accd972e6...
Open Graph Description: Environment RustPython d31ccae04d039ca3ab294742f2493aa5f2b63ff7, host build (x86_64-unknown-linux-gnu, debug), zlib.ZLIB_VERSION = 1.3.0-zlib-rs-0.6.8. crates/common/src/compression/zlib.rs on main...
X Description: Environment RustPython d31ccae04d039ca3ab294742f2493aa5f2b63ff7, host build (x86_64-unknown-linux-gnu, debug), zlib.ZLIB_VERSION = 1.3.0-zlib-rs-0.6.8. crates/common/src/compression/zlib.rs on main...
Opengraph URL: https://github.com/RustPython/RustPython/issues/8907
X: @github
Domain: github.com
{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`zlib.decompressobj`: `decompress(b\"\", n)` and `flush()` never return output that inflate still holds (raw deflate loses its last bytes under `max_length`)","articleBody":"### Environment\n\n- RustPython `d31ccae04d039ca3ab294742f2493aa5f2b63ff7`, host build (x86_64-unknown-linux-gnu,\n debug), `zlib.ZLIB_VERSION` = `1.3.0-zlib-rs-0.6.8`.\n- `crates/common/src/compression/zlib.rs` on `main` (`a7d75d25a7b2c291720411dc65445accd972e69d`,\n 2026-09-29) is byte-identical to `d31ccae`; the line numbers below hold for both.\n- Reference: CPython 3.13.5.\n\n### Reproducer\n\n```python\nimport zlib\nc = zlib.compressobj(6, zlib.DEFLATED, -15) # raw deflate, no trailer\nraw = c.compress(b\"x\" * 1168) + c.flush()\nd = zlib.decompressobj(-15)\nout = d.decompress(raw, 100)\nwhile d.unconsumed_tail:\n out += d.decompress(d.unconsumed_tail, 100)\nprint(\"all input consumed:\", len(out), \"of 1168, eof\", d.eof)\nmore = d.decompress(b\"\", 100)\nprint(\"decompress(b'', 100):\", len(more))\nrest = d.flush()\nprint(\"flush():\", len(rest), \"total:\", len(out + more + rest))\n```\n\nCPython 3.13.5:\n\n```text\nall input consumed: 1100 of 1168, eof False\ndecompress(b'', 100): 68\nflush(): 0 total: 1168\n```\n\nRustPython `d31ccae`:\n\n```text\nall input consumed: 1100 of 1168, eof False\ndecompress(b'', 100): 0\nflush(): 0 total: 1100\n```\n\nThe last 68 bytes stay inside the inflate state. The input is all consumed, so `unconsumed_tail` is\nempty, and neither `decompress(b\"\", n)` nor `flush()` runs inflate again.\n\nFrequency, the same drain loop plus `flush()`, sizes `n` in `range(1000, 3000, 7)`, `max_length`\nin `(64, 100, 256, 1024)` (1 144 cases):\n\n| | raw deflate (`wbits=-15`) short | zlib (`15`) and gzip (`31`) short |\n|---|---|---|\n| CPython 3.13.5 | 0 / 1 144 | 0 / 2 288 |\n| RustPython `d31ccae` | 196 / 1 144 | 0 / 2 288 |\n\nWith a zlib or gzip wrapper the trailer bytes are still unconsumed while output is pending, so\n`unconsumed_tail` is non-empty and the loop calls inflate again. Raw deflate has no trailer, so\nthe pending output can outlive the last input byte.\n\n### Source\n\n`crates/common/src/compression/zlib.rs`, `decompress_chunks`, lines 353–355:\n\n```rust\nif data.is_empty() {\n return Ok((Vec::new(), false));\n}\n```\n\n`Decompressor::decompress` (line 690) passes `b\"\"` through `decompress_inner` →\n`decompress_all` → `decompress_chunks`, which returns before it calls inflate.\n`Decompressor::flush` (lines 701–709) takes `unconsumed_tail`, which is empty here, and reaches\nthe same early return, so the `InflateFlush::Finish` call never runs either. CPython calls inflate\nfor an empty input and returns the pending output.\n\nA fix has to keep the existing unit test `empty_input_does_not_finish_a_stream` (line 1000):\nan empty input before any compressed data MUST NOT set `eof`. #8752 aligned lzma to this same early\nreturn for that reason.\n\n### Impact\n\nAny consumer that decodes raw deflate with a bounded `max_length` and drains with\n`decompress(b\"\", n)` or `flush()` gets a truncated result with no error. One example is HTTP\n`Content-Encoding: deflate` sent as raw deflate, which urllib3's `DeflateDecoder` accepts after its\nfirst zlib-header attempt fails. I have not measured urllib3 end to end for this case.\n\nThe reproducers above were run locally on RustPython and CPython.\n\n---\n\n🤖 Written by Claude Code (2.1.283) using model claude-opus-5-5\n","author":{"url":"https://github.com/JMLX42","@type":"Person","name":"JMLX42"},"datePublished":"2026-09-29T15:51:37Z","url":"https://github.com/RustPython/RustPython/issues/8907"}
| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /_view_fragments/issues/show/:user_id/:repository/:id/issue_layout(.:format) |
| route-controller | voltron_issues_fragments |
| route-action | issue_layout |
| fetch-nonce | v2:1a999ca0-1411-0371-b5ac-6d12a3c7385e |
| current-catalog-service-hash | 81bb79d38c15960b92d99bca9288a9108c7a47b18f2423d0f6438c5b7bcd2114 |
| request-id | 84C6:1DC4CC:1634563:1D8D08E:6B020691 |
| html-safe-nonce | bc03b931adde865a6182b022a398c212266fe5ed318d50864a360338eed05e86 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4NEM2OjFEQzRDQzoxNjM0NTYzOjFEOEQwOEU6NkIwMjA2OTEiLCJ2aXNpdG9yX2lkIjoiNTI1NjM4MTkzMTU2OTk0MDExMyIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 5a3a31cecc07c3fd48d7ac4443a01b9d97dc50c3883e4455a3c1e0de8d9ae4ae |
| hovercard-subject-tag | issue:5633265806 |
| github-keyboard-shortcuts | repository,issues,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/issues/show/RustPython/RustPython/8907/issue_layout |
| twitter:image | https://opengraph.githubassets.com/76c8a63efbde66c6dacd24673fc2068e57babb6eaead088c775f540d2709325e/RustPython/RustPython/issues/8907 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/76c8a63efbde66c6dacd24673fc2068e57babb6eaead088c775f540d2709325e/RustPython/RustPython/issues/8907 |
| og:image:alt | Environment RustPython d31ccae04d039ca3ab294742f2493aa5f2b63ff7, host build (x86_64-unknown-linux-gnu, debug), zlib.ZLIB_VERSION = 1.3.0-zlib-rs-0.6.8. crates/common/src/compression/zlib.rs on main... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | JMLX42 |
| hostname | github.com |
| expected-hostname | github.com |
| None | 87e774ee7c6b3bee0490394e4d3e57ec9dbc956f74b87ea685dc8996c1676456 |
| turbo-cache-control | no-preview |
| go-import | github.com/RustPython/RustPython git https://github.com/RustPython/RustPython.git |
| octolytics-dimension-user_id | 39710557 |
| octolytics-dimension-user_login | RustPython |
| octolytics-dimension-repository_id | 135201145 |
| octolytics-dimension-repository_nwo | RustPython/RustPython |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 135201145 |
| octolytics-dimension-repository_network_root_nwo | RustPython/RustPython |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 423dbb54c5e7bfecc838eb387c1435eb3d0e760d |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| react-app-name | issues |
Links:
Viewport: width=device-width