René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"asyncio: start_tls() loses buffered data from StreamReader","articleBody":"# Bug report\n\n### Bug description:\n\n## Bug description\nWhen using `StreamWriter.start_tls()` (or `loop.start_tls()`) to upgrade a connection to TLS mid-stream, any data already buffered in the `StreamReader` is lost.\n\nThis commonly occurs when implementing servers that support the PROXY protocol (e.g., behind HAProxy with `send-proxy`). In this scenario:\n\n1. The proxy sends the PROXY protocol header followed immediately by the TLS ClientHello in the same TCP segment\n2. The server reads the PROXY header using `reader.readline()`\n3. The TLS ClientHello data is now buffered in `StreamReader._buffer`\n4. When `writer.start_tls()` is called, a new `SSLProtocol` is created, but the buffered data is not transferred to it\n5. The TLS handshake hangs waiting for ClientHello data that was already received but lost\n\n## Reproducible example\n\n### Server (server.py)\n\n```python\nimport asyncio\nimport ssl\n\nasync def handle_client(reader, writer):\n    # Read PROXY protocol header\n    proxy_header = await reader.readline()\n    print(f\"PROXY header: {proxy_header}\")\n\n    # Upgrade to TLS - this is where data loss occurs\n    context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)\n    context.load_cert_chain('server.crt', 'server.key')\n    \n    await writer.start_tls(context)\n    print(\"TLS handshake completed\")  # Never reached - hangs here\n\n    data = await reader.read(1024)\n    print(f\"Received: {data}\")\n    \n    writer.close()\n    await writer.wait_closed()\n\nasync def main():\n    server = await asyncio.start_server(handle_client, '127.0.0.1', 8000)\n    async with server:\n        await server.serve_forever()\n\nasyncio.run(main())\n```\n\n### HAProxy config (haproxy.cfg)\n```\nfrontend proxy_frontend\n    mode tcp\n    bind *:5002\n    default_backend proxy_backend\n\nbackend proxy_backend\n    mode tcp\n    server app 127.0.0.1:8000 send-proxy\n```\n\n### Client command\n```bash\necho \"Test message\" | openssl s_client -connect 127.0.0.1:5002\n```\n\n### Expected behavior\nTLS handshake completes and server receives \"Test message\".\n\n### Actual behavior\nTLS handshake hangs indefinitely because the TLS ClientHello data buffered in `StreamReader` is lost when `start_tls()` is called.\n\n## Additional context\nI have a fix ready with tests and will submit a PR shortly.\n\nThe fix transfers buffered data from `StreamReader._buffer` to `SSLProtocol._incoming` before the TLS handshake begins.\n\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nLinux\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-142354\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/kasimov-maxim","@type":"Person","name":"kasimov-maxim"},"datePublished":"2025-12-06T16:08:38.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/142352/cpython/issues/142352"}