René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[subinterpreters] Refleaks on Windows Under Specific Conditions","articleBody":"# Bug report\n\n### Bug description:\n\nOne of my recent PRs, gh-117662, led to one of the Windows refleak buildbots failing[^1].  I'm fairly confident the PR revealed an existing source of refleaks (albeit an unlikely one), rather that introducing new leaks.  I have resolved the failures with gh-117913, but the underlying potential source of refleaks remains.\n\n[^1]: https://buildbot.python.org/all/#/builders/920/builds/733\n\nI've been able to reproduce the leak conditions, but only on Windows (for now?).  The following conditions are necessary:\n\n* interpreter created using a config with `check_multi_interp_extensions=False` (e.g. \"legacy\")\n* interpreter imports test.support.os_helper\n\n```python\ntry:\n    import _interpreters\nexcept ModuleNotFoundError:\n    import _xxsubinterpreters as _interpreters\nconfig = _interpreters.new_config('legacy')  # critically, check_multi_interp_extensions=False\ninterpid = _interpreters.create(config)\n_interpreters.exec(interpid, 'import test.support.os_helper')\n_interpreters.destroy(interpid)\n# leaks a bunch of objects\n```\n\nWhen the bug is triggered, around 190 objects are leaked.  At first I thought this was interpreter finalization failing silently, but now I think it is something else.\n\nI suspect the underlying problem relates to legacy extension modules.  In the specific example above, I'm pretty sure the `_ctypes` module is leaking and ~190 is how many objects it holds, directly or indirectly.  The main clue there is that, when I run `./python -v ...`, the `_ctypes` module is never noted as destroyed, whereas all other modules are.  As to test.support.os_helper, `_ctypes` is one of the modules that gets indirectly imported.  All that said, I haven't been able to reproduce the leak if the subinterpreter above imports `_ctypes` instead.\n\nWhy did we only see this on Windows?  The only clue I can think of is that on Windows stdlib extension modules are all turned into builtin modules, IIRC.  The logic of the `check_multi_interp_extensions` check (via `_PyImport_CheckSubinterpIncompatibleExtensionAllowed()` in Python/import.c) is specific to extension modules, not builtin modules.\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nWindows","author":{"url":"https://github.com/ericsnowcurrently","@type":"Person","name":"ericsnowcurrently"},"datePublished":"2024-04-16T15:51:25.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/117936/cpython/issues/117936"}