René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"`PyThreadState` and `PyInterpreterState` are not freed","articleBody":"The following test program gains about 10M per second under top, on Python 3.11 only (confirmed for all development stages: 3.11.0a4, 3.11.0b1, 3.11.0rc1, 3.11.0)  .  3.10 shows no memory growth.\r\n\r\n```py\r\nfrom threading import Thread\r\n\r\nwhile True:\r\n    t1 = Thread()\r\n    t1.start()\r\n```\r\n\r\nFor direct results, here's the same program using the `resource` module that I just saw used over at #98467:\r\n\r\n```py\r\nfrom threading import Thread\r\nimport resource\r\n\r\nprint(\"Before\", resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024, \"MB\")\r\n\r\ni = 0\r\n\r\nwhile True:\r\n    i += 1\r\n    t1 = Thread()\r\n    t1.start()\r\n    if i % 10000 == 0:\r\n        print(\r\n            f\"memory after {i} iterations:\",\r\n            resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / 1024,\r\n            \"MB\",\r\n        )\r\n```\r\n\r\nThe above program under Py 3.11.0 prints:\r\n\r\n```\r\n$ .venv-3.11/bin/python test5.py \r\nBefore 8.14453125 MB\r\nmemory after 10000 iterations: 11.6875 MB\r\nmemory after 20000 iterations: 15.30078125 MB\r\nmemory after 30000 iterations: 18.65234375 MB\r\nmemory after 40000 iterations: 22.26171875 MB\r\nmemory after 50000 iterations: 25.87109375 MB\r\nmemory after 60000 iterations: 29.22265625 MB\r\nmemory after 70000 iterations: 32.83203125 MB\r\nmemory after 80000 iterations: 36.44140625 MB\r\nmemory after 90000 iterations: 39.78125 MB\r\nmemory after 100000 iterations: 43.390625 MB\r\n...\r\n```\r\n\r\nunder Python 3.10 it prints:\r\n\r\n```\r\n$ .venv-3.10/bin/python test5.py \r\nBefore 9.25 MB\r\nmemory after 10000 iterations: 9.46484375 MB\r\nmemory after 20000 iterations: 9.47265625 MB\r\nmemory after 30000 iterations: 9.47265625 MB\r\nmemory after 40000 iterations: 9.47265625 MB\r\nmemory after 50000 iterations: 9.47265625 MB\r\nmemory after 60000 iterations: 9.47265625 MB\r\nmemory after 70000 iterations: 9.47265625 MB\r\nmemory after 80000 iterations: 9.47265625 MB\r\n```\r\n\r\n\r\nthe issue looks extremely similar to another one we just fixed in greenlet, over at https://github.com/python-greenlet/greenlet/issues/328, although this one is much more surprising.    Issue #98467 might also be related.\r\n\r\n\r\nEnvironment:\r\n```\r\n$ uname -a\r\nLinux photon3 5.17.14-200.fc35.x86_64 #1 SMP PREEMPT Thu Jun 9 14:02:42 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux\r\n\r\n$ python3\r\nPython 3.11.0 (main, Nov  5 2022, 23:27:22) [GCC 11.3.1 20220421 (Red Hat 11.3.1-3)] on linux\r\n```\r\n\n\n\u003c!-- gh-pr-number: gh-99268 --\u003e\n* PR: gh-99268\n\u003c!-- /gh-pr-number --\u003e\n\n\n\u003c!-- gh-pr-number: gh-99301 --\u003e\n* PR: gh-99301\n\u003c!-- /gh-pr-number --\u003e\n\n\n\u003c!-- gh-pr-number: gh-99385 --\u003e\n* PR: gh-99385\n\u003c!-- /gh-pr-number --\u003e\n","author":{"url":"https://github.com/zzzeek","@type":"Person","name":"zzzeek"},"datePublished":"2022-11-07T15:11:48.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":13},"url":"https://github.com/99205/cpython/issues/99205"}