René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Leverage eager tasks to optimize asyncio gather \u0026 TaskGroups further","articleBody":"# Feature or enhancement\r\n\r\ngh-97696 introduced eager tasks factory, which speeds up some async-heavy workloads by up to 50% when opted in.\r\n\r\ninstalling the eager tasks factory applies out-of-the-box when gathering tasks (`asyncio.gather(...)`), and when creating tasks as part of a `TaskGroup`, e.g.:\r\n\r\n```\r\nasyncio.get_event_loop().set_task_factory(asyncio.eager_task_factory)\r\n\r\nawait asyncio.gather(coro1, coro2, coro3)\r\n\r\nasync with asyncio.TaskGroup() as tg:\r\n    tg.create_task(coro1)\r\n    tg.create_task(coro2)\r\n    tg.create_task(coro3)\r\n```\r\n\r\nin both examples, `coro{1,2,3}` will eagerly execute the first step, and potentially complete without scheduling to the event loop if the coros don't block.\r\n\r\nthe implementation of both `gather` and `TaskGroup` uses callbacks internally that end up getting scheduled to the event loop even if all the tasks were able to finish synchronously, and blocking the coroutine in which `gather` or `TaskGroup()` was awaited, preventing from the task to complete eagerly even if otherwise it could.\r\n\r\napplications that use multiple levels of nested gathers / TaskGroups can benefit significantly from eagerly completing multiple levels without blocking.\r\n\r\nthis can be achieved by modifying the implementation of `gather` and `TaskGroup` to detect when it can complete without blocking, and skip scheduling wakeup \u0026 done callbacks.\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-104140\n* gh-104138\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/itamaro","@type":"Person","name":"itamaro"},"datePublished":"2023-05-03T22:43:17.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/104144/cpython/issues/104144"}