René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Double linked list implementation for asyncio tasks","articleBody":"Currently `asyncio` tasks are stored in a `Weakset`, this is inefficient and in some cases causes bugs because of thread safety (https://github.com/python/cpython/issues/80788). In terms of memory usage it requires maintaining a full set and their corresponding weakref callback to cleanup objects when deallocated and finalized by the gc. In applications where tasks are created at fast pace this becomes a bottle neck, to mitigate this now `asyncio` tasks will now be stored in a global double linked of tasks for cases where Task is a subclass of `_asyncio.Task` in other cases we still rely on the weakset. This reduces the work done by the gc speedups the execution and reduces memory usage. In some of my own benchmarks I have seen 15- 20% improvement and pyperformance benchmarks reflect roughly the same.\r\n\r\nhttps://github.com/faster-cpython/benchmarking-public/blob/main/results/bm-20230805-3.13.0a0-1d32835/bm-20230805-linux-x86_64-kumaraditya303-linked_list-3.13.0a0-1d32835-vs-base.md\r\n\r\nUpdated: https://github.com/faster-cpython/benchmarking-public/tree/main/results/bm-20240622-3.14.0a0-4717aaa#vs-base\r\n\r\n\u003c!-- gh-linked-prs --\u003e\r\n### Linked PRs\r\n* gh-107804\r\n* gh-120995\r\n* gh-121007\r\n* gh-126577\n\u003c!-- /gh-linked-prs --\u003e\r\n","author":{"url":"https://github.com/kumaraditya303","@type":"Person","name":"kumaraditya303"},"datePublished":"2023-08-09T10:05:59.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/107803/cpython/issues/107803"}