René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Legacy-mode subinterpreters in Python 3.12: `import _tkinter` leads to shutdown crash ","articleBody":"# Bug report\r\n\r\n### Bug description:\r\n\r\nHello everyone,\r\n\r\nI am working on an application embedding multiple Python subinterpreters - for which the Python version should be upgraded from Python 3.10 to Python 3.12.1. For the time being, the \"legacy version\" of subinterpreters (i.e., using a global, shared GIL) should be used, since all Python extensions (including `_tkinter` and all extensions with single-phase initialization) should be supported.\r\n\r\nIf I understand the docs correctly, using the legacy `Py_NewInterpreter()` method should preserve the existing behavior. Still, the following application crashes at shutdown:\r\n\r\n```c++\r\n#include \u003cPython.h\u003e\r\n\r\nclass PythonInterpreter {\r\n public:\r\n    PythonInterpreter(PyThreadState* parent, int id)\r\n    : mParent(parent)\r\n    , mId(id) {\r\n        PyEval_RestoreThread(mParent);\r\n        mThread = Py_NewInterpreter();\r\n        PyObject* globals = PyModule_GetDict(PyImport_AddModule(\"__main__\"));\r\n        PyRun_String(\"import _tkinter\", Py_single_input, globals, globals);\r\n        PyEval_SaveThread();\r\n        std::cout \u003c\u003c \"Subinterpreter \" \u003c\u003c id \u003c\u003c \" done\" \u003c\u003c std::endl;\r\n    }\r\n\r\n    virtual ~PythonInterpreter() {\r\n        std::cout \u003c\u003c \"destructor \" \u003c\u003c mId \u003c\u003c std::endl;\r\n        PyThreadState_Swap(mThread);\r\n        Py_EndInterpreter(mThread);\r\n    }\r\n\r\n private:\r\n    PyThreadState* mParent;\r\n    PyThreadState* mThread;\r\n    int mId;\r\n};\r\n\r\nint main(int /* argc */, char** /* argv[] */) {\r\n    PyConfig config;\r\n    PyConfig_InitPythonConfig(\u0026config);\r\n    PyStatus status = Py_InitializeFromConfig(\u0026config);\r\n    if (PyStatus_Exception(status)) {\r\n        PyConfig_Clear(\u0026config);\r\n        Py_ExitStatusException(status);\r\n    } else {\r\n        PyConfig_Clear(\u0026config);\r\n    }\r\n    PyThreadState* s0 = PyThreadState_Get();\r\n    PyEval_SaveThread();\r\n    PythonInterpreter* i0 = new PythonInterpreter(s0, 0);\r\n    PythonInterpreter* i1 = new PythonInterpreter(s0, 1);\r\n    delete i0;\r\n    delete i1;\r\n    PyEval_RestoreThread(s0);\r\n    Py_Finalize();\r\n}\r\n```\r\n\r\nWhen compiling and running this program with a debug build of Python 3.12.1 (or later) on Linux, I get this output:\r\n\r\n```\r\nSubinterpreter 0 done\r\nSubinterpreter 1 done\r\ndestructor 0\r\ndestructor 1\r\nmain: Objects/dictobject.c:283: unicode_get_hash: Assertion `Py_IS_TYPE(((PyObject*)(((o)))), (\u0026PyUnicode_Type))' failed.\r\n```\r\n\r\nWith a non-debug build, the program exits with a segmentation fault.\r\n\r\nThe `gdb` backtrace looks as follows:\r\n\r\n```\r\n#0  0x00007ffff6311387 in raise () from /lib64/libc.so.6\r\n#1  0x00007ffff6312a78 in abort () from /lib64/libc.so.6\r\n#2  0x00007ffff630a1a6 in __assert_fail_base () from /lib64/libc.so.6\r\n#3  0x00007ffff630a252 in __assert_fail () from /lib64/libc.so.6\r\n#4  0x00007ffff6a9baf5 in unicode_get_hash (o=\u003coptimized out\u003e) at Objects/dictobject.c:2143\r\n#5  _PyDict_Next (op=op@entry=0x7fffecfd2270, ppos=ppos@entry=0x7fffffffd2a8, pkey=pkey@entry=0x7fffffffd2a0, pvalue=pvalue@entry=0x7fffffffd298, \r\n    phash=phash@entry=0x0) at Objects/dictobject.c:2142\r\n#6  0x00007ffff6a9c0d8 in PyDict_Next (op=op@entry=0x7fffecfd2270, ppos=ppos@entry=0x7fffffffd2a8, pkey=pkey@entry=0x7fffffffd2a0, \r\n    pvalue=pvalue@entry=0x7fffffffd298) at Objects/dictobject.c:2189\r\n#7  0x00007ffff6ab3751 in _PyModule_ClearDict (d=0x7fffecfd2270) at Objects/moduleobject.c:624\r\n#8  0x00007ffff6ab40dd in _PyModule_Clear (m=m@entry=0x7fffed02ca70) at Objects/moduleobject.c:604\r\n#9  0x00007ffff6c11bd4 in finalize_modules_clear_weaklist (interp=interp@entry=0x7fffed03c020, weaklist=weaklist@entry=0x7fffef912da0, verbose=verbose@entry=0)\r\n    at Python/pylifecycle.c:1526\r\n#10 0x00007ffff6c125ef in finalize_modules (tstate=tstate@entry=0x7fffed099950) at Python/pylifecycle.c:1609\r\n#11 0x00007ffff6c202da in Py_EndInterpreter (tstate=0x7fffed099950) at Python/pylifecycle.c:2220\r\n#12 0x00000000004014b5 in PythonInterpreter::~PythonInterpreter (this=0x5092f0, __in_chrg=\u003coptimized out\u003e) at main.cc:25\r\n#13 PythonInterpreter::~PythonInterpreter (this=0x5092f0, __in_chrg=\u003coptimized out\u003e) at main.cc:26\r\n#14 0x00000000004012e2 in main () at main.cc:60\r\n```\r\n\r\nDigging further into the backtrace, it looks like the Python garbage collector is trying to decrease the reference counter to the `_tkinter` module twice, despite it having been increased only once. Oddly enough, the program runs just fine when destroying the interpreters in reverse order:\r\n\r\n```c++\r\n    PythonInterpreter* i0 = new PythonInterpreter(s0, 0);\r\n    PythonInterpreter* i1 = new PythonInterpreter(s0, 1);\r\n    delete i1;\r\n    delete i0;\r\n```\r\n\r\nCan anyone help me shed some light into this issue? Is there anything I am overlooking?\r\n\r\n### CPython versions tested on:\r\n\r\n3.12.1, 3.12.2, 3.13.0a4\r\n\r\n### Operating systems tested on:\r\n\r\nLinux, Windows\n```[tasklist]\n### Tasks\n```\n\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-120290\n* gh-120315\n* gh-120652\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/car-bianco","@type":"Person","name":"car-bianco"},"datePublished":"2024-02-19T11:22:04.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/115649/cpython/issues/115649"}