René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Unable to cancel Server.serve_forever() if a reader is blocked while reading (3.12 regression)","articleBody":"# Bug report\r\n\r\n### Bug description:\r\n\r\nConsider this code, slightly simplifying the documentation's TCPServer example code:\r\n```python\r\nimport asyncio\r\n\r\nasync def handle_echo(reader, writer):\r\n    print(\"Reading\")\r\n    data = await reader.read(100)\r\n    message = data.decode()\r\n    print(f\"Received '{message!r}'\")\r\n\r\n    print(\"Closing the connection\")\r\n    writer.close()\r\n    await writer.wait_closed()\r\n\r\nasync def main():\r\n    server = await asyncio.start_server(handle_echo, \"127.0.0.1\", 8888)\r\n    print(\"Serving forever...\")\r\n    async with server:\r\n        try:\r\n            await server.serve_forever()  \r\n        except asyncio.CancelledError:\r\n            print(\"Cancelled by Ctrl+C\")\r\n            server.close()\r\n\r\nasyncio.run(main())\r\n```\r\n(My code is closer to a `while True: await reader.readline(); ...`, but the above probably suffices as a demonstration)\r\n\r\nRunning this results in a _Serving forever_, and hitting Ctrl+C, results in a _Cancelled by Ctrl+C_, and a normal exit.\r\n\r\nHowever, if in another window we `nc 127.0.0.1 8888`, and leave the connection open, Ctrl+C (SIGINT) does nothing, and a second Ctrl+C is required to terminate. (This however breaks out of the asyncio loop by raising `KeyboardInterrupt()` , as documented).\r\n\r\nSo basically clients can prevent the server from cleanly exiting by just keeping their connection open.\r\n\r\nThis is a regression: this fails with 3.12.5 and 3.13.0-rc1 but works with 3.11.9.\r\n\r\nThis is because (TTBOMK) of this code in `base_events.py`:\r\n```python\r\n        try:\r\n            await self._serving_forever_fut\r\n        except exceptions.CancelledError:\r\n            try:\r\n                self.close()\r\n                await self.wait_closed()\r\n            finally:\r\n                raise\r\n        finally:\r\n            self._serving_forever_fut = None\r\n```\r\n\r\nI believe this to be related to the `wait_closed()` changes, 5d09d11aa0b89aeba187f4f520728ccaf4fc5ac1, 26553695592ad399f735d4dbaf32fd871d0bb1e1 etc. (Cc @gvanrossum). Related issues #104344 and #113538.\r\n\r\nPer @gvanrossum in https://github.com/python/cpython/issues/113538#issuecomment-1874724821:  \"_In 3.10 and before, server.wait_closed() was a no-op, unless you called it before server.close(), in a task (asyncio.create_task(server.wait_closed())). The unclosed connection was just getting abandoned._\"\r\n\r\n`CancelledError()` is caught here, which spawns `wait_closed()` before re-raising the exception. In 3.12+, `wait_closed()`... actually waits for the connection to close, as intended. However, while this prevents the reader task from being abandoned, it does not allow neither _the callers_ of `reader.read()` or `serve_forever()` to catch `CancelledError()` and clean up (such as actually closing the connection, potentially after e.g. a signaling to the client a server close through whatever protocol is implemented here).\r\n\r\nBasically no user code is executed until the client across the network drops the connection.\r\n\r\nAs far as I know, it's currently impossible to handle SIGINTs cleanly with clients blocked in a `read()` without messing with deep asyncio/selector internals, which seems like a pretty serious limitation? Have I missed something?\r\n\r\n### CPython versions tested on:\r\n\r\n3.11, 3.12, 3.13\r\n\r\n### Operating systems tested on:\r\n\r\nLinux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-124689\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/paravoid","@type":"Person","name":"paravoid"},"datePublished":"2024-09-05T00:31:50.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/123720/cpython/issues/123720"}