René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Requiring A Current Thread State for PyThreadState_Clear(), etc. is Problematic","articleBody":"In `PyThreadState_Clear()` and `PyInterpreterState_Clear()` the code expects a \"current\" thread state (i.e. set by `PyThreadState_Swap()`) to be set.  Presumably this is for the following:\r\n\r\n* to ensure the GIL is held\r\n* to use for logging audit events (actually only needs the current interpreter?)\r\n* to use in object finalizers for objects held by the target thread/interpreter state\r\n\r\nFor `PyThreadState_Clear()`, this is a problem when the current thread state is also the target thread state, since the function is progressively rendering the thread state unusable.  For `PyInterpreterState_Clear()` the \"current\" thread state must belong to the target interpreter, but will be rendered unusable before it's potentially needed.\r\n\r\n(FWIW, it isn't clear to me yet why we don't see frequent crashes from this already (at least when we're clearing the main interpreter during finalization).  I do suspect that this contributes to some of the occasional CI failures we see relative to finalization.)\r\n\r\nIdeally auditing and the various object finalizers could work without having a current thread state, especially in the case where the current thread state could only possibly be the one getting cleared.  Could we split up `PyThreadState` so at least the problematic part is guaranteed not be used in `PyThreadState_Clear()` or `PyInterpreterState_Clear()`?  Are there other complexities we have to deal with?\r\n\r\n(Note that we'd still require that the GIL be held by the current OS thread when `PyThreadState_Clear()` or `PyInterpreterState_Clear()` are called.)\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eFTR, in gh-101308 I noted various cases where the lifecycle of thread states has an unexpected status.  Pretty much all of them relate `PyThreadState_Clear()` requiring a current thread state.\u003c/summary\u003e\r\n\r\n\u003cBR/\u003e\r\nDealing with the main issue above would make it trivial to address the following:\r\n\r\n* an \"active\" thread state (i.e. \"current\", holding the GIL) must be \"bound\" (to an OS thread) already\r\n* a \"bound\" thread state must be \"alive\" (initialized but not finalizing)\r\n* binding a thread state should be done separately from `PyThreadState_New()`\r\n* the thread state passed to `PyThreadState_Clear()` should no longer be \"active\" or \"bound\"\r\n* the current thread state must belong to the target interpreter of `PyThreadState_Clear()`\r\n* the current thread state must belong to the target interpreter of `PyInterpreterState_Clear()` (for the main interpreter it must be the main thread)\r\n\r\n\u003c/details\u003e","author":{"url":"https://github.com/ericsnowcurrently","@type":"Person","name":"ericsnowcurrently"},"datePublished":"2023-01-30T20:27:15.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/101436/cpython/issues/101436"}