René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"C API: Design an API to disallow creating more than one extension instance per Python process","articleBody":"Somewhere in `PyModuleDef`, we should add a way to disallow globally the creation of more than one extension instance.\r\n\r\nOr maybe the restriction should be that an extension cannot be loaded in two different interpreters. Honestly, I'm not sure.\r\n\r\n@ericsnowcurrently no longer wants to treat the **main** interpreter differently: see issue gh-109857.\r\n\r\nPreviously, I proposed to directly expose PyInterpreterState_IsMain() to the limited C API, but the idea got rejected.\r\n\r\n---\r\n\r\nThe syslog module behaves differently if it's used in the **main interpreter**:\r\n\r\n```c\r\nstatic inline int\r\nis_main_interpreter(void)\r\n{\r\n    return (PyInterpreterState_Get() == PyInterpreterState_Main());\r\n}\r\n\r\n...\r\n    if (!is_main_interpreter()) {\r\n        PyErr_SetString(PyExc_RuntimeError, \"subinterpreter can't use syslog.openlog()\");\r\n        return NULL;\r\n    }\r\n```\r\n\r\nThe syslog module stores a Python object in `static PyObject *S_ident_o = NULL;`. It's a way to keep a string passed to C openlog() function alive until closelog() is called. The problem is to decide which extension instance and/or which Python interpreter is responsible for that memory: the memory must be kept alive until closelog() is called. But what if each Python interpreter call openlog()? Calling openlog() more than once overrides the previous call. But if closelog() is called, if multiple interpreters called openlog(), you might think that syslog is still usable, whereas closelog() has been called.\r\n\r\nSee issue gh-99127 and PR gh-99128 which added `is_main_interpreter()` function to syslog.\r\n\r\n---\r\n\r\nCython already implements a similar, but different check: it raises **\"this module can only be loaded into one interpreter per process\"** error message.\r\n\r\nCython uses a different logic to handle PEP 489 \"Multi-phase extension module initialization\". It uses a ``static PY_INT64_T main_interpreter_id = -1;`` variable which is initialized at the first call of ``__Pyx_check_single_interpreter()``.\r\n\r\n```c\r\n//#if CYTHON_PEP489_MULTI_PHASE_INIT\r\nstatic CYTHON_SMALL_CODE int __Pyx_check_single_interpreter(void) {\r\n    #if PY_VERSION_HEX \u003e= 0x030700A1\r\n    static PY_INT64_T main_interpreter_id = -1;\r\n    PY_INT64_T current_id = PyInterpreterState_GetID(PyThreadState_Get()-\u003einterp);\r\n    if (main_interpreter_id == -1) {\r\n        main_interpreter_id = current_id;\r\n        return (unlikely(current_id == -1)) ? -1 : 0;\r\n    } else if (unlikely(main_interpreter_id != current_id))\r\n\r\n    #else\r\n    static PyInterpreterState *main_interpreter = NULL;\r\n    PyInterpreterState *current_interpreter = PyThreadState_Get()-\u003einterp;\r\n    if (!main_interpreter) {\r\n        main_interpreter = current_interpreter;\r\n    } else if (unlikely(main_interpreter != current_interpreter))\r\n    #endif\r\n\r\n    {\r\n        PyErr_SetString(\r\n            PyExc_ImportError,\r\n            \"Interpreter change detected - this module can only be loaded into one interpreter per process.\");\r\n        return -1;\r\n    }\r\n    return 0;\r\n}\r\n```","author":{"url":"https://github.com/vstinner","@type":"Person","name":"vstinner"},"datePublished":"2023-10-19T23:10:09.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":3},"url":"https://github.com/111088/cpython/issues/111088"}