René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"GzipFile.flush doesn't flush compressor in 3.12 beta","articleBody":"\u003c!--\r\n  If you're new to Python and you're not sure whether what you're experiencing is a bug, the CPython issue tracker is not\r\n  the right place to seek help. Consider the following options instead:\r\n\r\n  - reading the Python tutorial: https://docs.python.org/3/tutorial/\r\n  - posting in the \"Users\" category on discuss.python.org: https://discuss.python.org/c/users/7\r\n  - emailing the Python-list mailing list: https://mail.python.org/mailman/listinfo/python-list\r\n  - searching our issue tracker (https://github.com/python/cpython/issues) to see if\r\n    your problem has already been reported\r\n--\u003e\r\n\r\n# Bug report\r\n\r\nThe change to add buffering to `GzipFile.write` (gh-89550, #101251) broke the `GzipFile.flush` method. The flush method previously called `self.compress.flush`, but now it only flushes the IO objects and not the compressor (as a side effect, the zlib_mode argument is now ignored, although in my case I only use the default). Flushing the compressor is necessary to create synchronization points that can be used to decompress part of the stream. \r\n\r\nHere is a test script, reduced from Tornado's use of GzipFile (see tornadoweb/tornado#3278 for the way this manifests in Tornado's test suite):\r\n\r\n```python\r\nimport io\r\nimport gzip\r\nimport zlib\r\n\r\n# Write two chunks to the same compressed stream. In real usage\r\n# I send these chunks as two separate network messages, but in this\r\n# test I just save them to two local variables.\r\ndata = io.BytesIO()\r\ngzip_file = gzip.GzipFile(fileobj=data, mode=\"wb\")\r\ngzip_file.write(b\"Hello World\")\r\ngzip_file.flush()\r\nmessage1 = data.getvalue()\r\ndata.truncate(0)\r\ndata.seek(0)\r\ngzip_file.write(b\"Goodbye World\")\r\ngzip_file.close()\r\nmessage2 = data.getvalue()\r\n\r\n# Decode the two messages. Each one should decode separately,\r\n# but in Python 3.12b2 the compressor was not flushed with\r\n# Z_SYNC_FLUSH so the second message produces no output on its own\r\n# and both messages are emitted when the second message is added\r\n# to the decompressor's input.\r\n#\r\n# This results in the error\r\n#   AssertionError: [b'', b'Hello WorldGoodbye World']\r\ndecompressor = zlib.decompressobj(16 + zlib.MAX_WBITS)\r\nmessages = [decompressor.decompress(message1), decompressor.decompress(message2)]\r\nassert messages == [b\"Hello World\", b\"Goodbye World\"], messages\r\n```\r\n\r\n# Your environment\r\n\r\n\u003c!-- Include as many relevant details as possible about the environment you experienced the bug in --\u003e\r\n\r\n- CPython versions tested on: The bug is present in 3.12b2; the above script passes on 3.11 and earlier\r\n- Operating system and architecture: macOS and Linux\r\n\r\n\u003c!--\r\nYou can freely edit this text. Remove any lines you believe are unnecessary.\r\n--\u003e\r\n\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-105910\n* gh-105920\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/bdarnell","@type":"Person","name":"bdarnell"},"datePublished":"2023-06-15T01:20:58.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/105808/cpython/issues/105808"}