René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Memory leak on executables embedded with 3.13","articleBody":"# Bug report\r\n\r\n### Bug description:\r\n\r\nSince 67807cfc87135fdce4992d38d2ffe3e44747e73b, `_testembed.exe` (x64 Release) increases the memory usage in the cycle of `Py_Initialize` and `Py_Finalize`.\r\n\r\nI monitored the Task Manager (`taskmgr.exe`) on Windows10/11, with a change against `Programs/_testembed.c`:\r\n\r\n```c\r\n- #define INIT_LOOPS 4\r\n+ #define INIT_LOOPS 200\r\n```\r\nCommands:\r\n* `_testembed.exe test_repeated_simple_init` or\r\n* `python -m test test_embed -m test_simple_initialization_api` (no log)\r\n\r\nWith RADIX TREE (as-is)\r\n\r\n|Loop |67807cf|main|3.11.7|\r\n| :---: | :---: | :---: | :---: |\r\n| 100 | 67MB | 210MB | 4MB |\r\n| 200 | 135MB | 406MB | 4MB |\r\n\r\nNo RADIX TREE (as-is)\r\n\r\n|Loop |67807cf|main|3.11.7|\r\n| :---: | :---: | :---: | :---: |\r\n| 100 | 10MB | 168MB | 4MB |\r\n| 200 | 16MB | 336MB | 4MB |\r\n\r\nRecent `obmalloc.c` looks ready for finalization. Just adding my rough (invalid?) experiment made the leaks even. So, I hope that they share the same issue. Otherwise, ea2c0016502472aa8baa3149050ada776d17a009 or 15d4c9fabce67b8a1b5bd9dec9612014ec18291a has another leak. https://github.com/python/cpython/issues/98359#issuecomment-1847609983\r\n\u003cdetails\u003e\r\n\u003csummary\u003epatch\u003c/summary\u003e\r\n\r\n```c\r\nvoid\r\n_PyInterpreterState_FinalizeAllocatedBlocks(PyInterpreterState *interp)\r\n{\r\n    if (has_own_state(interp)) {\r\n        Py_ssize_t leaked = _PyInterpreterState_GetAllocatedBlocks(interp);\r\n        assert(has_own_state(interp) || leaked == 0);\r\n        interp-\u003eruntime-\u003eobmalloc.interpreter_leaks += leaked;\r\n\r\n+       OMState *state = \u0026interp-\u003eobmalloc;\r\n+       for (uint i = 0; i \u003c maxarenas; ++i) {\r\n+           if (allarenas[i].address == 0) {\r\n+               continue;\r\n+           }\r\n+           _PyObject_Arena.free(_PyObject_Arena.ctx,\r\n+                                (void *)allarenas[i].address, ARENA_SIZE);\r\n+           allarenas[i].address = 0;\r\n+           --narenas_currently_allocated;\r\n+       }\r\n+       PyMem_RawFree(allarenas);\r\n    }\r\n}\r\n```\r\n\r\n\u003c/details\u003e\r\n\r\nWith RADIX TREE (patched)\r\n\r\n|Loop |67807cf|main|\r\n| :---: | :---: | :---: |\r\n| 100 | 42MB | 40MB |\r\n| 200 | 80MB | 80MB |\r\n\r\nNo RADIX TREE (patched)\r\n\r\n|Loop |67807cf|main|\r\n| :---: | :---: | :---: |\r\n| 100 | 3.5M | 3.3M |\r\n| 200 | 3.7M | 3.4M |\r\n\r\n\r\ncc @ericsnowcurrently @nascheme\r\n\r\n### CPython versions tested on:\r\n\r\n3.12, 3.13, CPython main branch: 3aea6c4823e90172c9bc36cd20dc51b295d8a3c4\r\n\r\n### Operating systems tested on:\r\n\r\nWindows\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-113218\n* gh-113412\n* gh-118531\n* gh-118618\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/neonene","@type":"Person","name":"neonene"},"datePublished":"2023-12-13T14:08:40.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":20},"url":"https://github.com/113055/cpython/issues/113055"}