René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Audit all built-in modules for thread safety","articleBody":"# Feature or enhancement\n\n### Proposal:\n\nWe should audit every built-in module for thread safety and make any necessary fixes. This can be done separately from tagging the modules as safe using `Py_mod_gil`; any data races in built-in modules are considered bugs in the free-threaded build and not incompatibilities.\n\nBelow is a list of all source files that contain at least one module definition, as defined by files that contain either a call to `PyModule_Create()` or an array of `PyModuleDef_Slot`s. If you'd like to help out with this task, take a look at one of the incomplete files. If the conversion is trivial, check it off and attach the PR to this issue. Otherwise, convert the line into an issue and assign it to yourself.\n\nEvery module is different, but here are a few high-level points to guide the work:\n1. Use [`PyMutex`](https://github.com/python/cpython/blob/main/Include/internal/pycore_lock.h) as the basic unit of locking.\n2. If you need to hold a lock for a longer period of time, especially across calls that may reenter Python code or acquire other locks, use [critical sections](https://github.com/python/cpython/blob/main/Include/internal/pycore_critical_section.h).\n3. If the module you're looking at is a thin wrapper around related code elsewhere in CPython (e.g., `Modules/_codecsmodule.c` and `Python/codecs.c`), you can also audit/convert the related non-module code. Otherwise, try to contain your work to just the module code, and create separate issues for any dependencies that aren't thread-safe.\n4. Remember that C API functions are generally expected to handle thread-safety internally. C code that operates on Python objects using only C API calls is usually thread-safe by default (but look out for code that reads or modifies `PyObject*`s in C structs, since that needs synchronization).\n5. Watch out for functions/macros that return borrowed references, like `PyList_GetItem()` or `PyDict_GetItem()`. If other threads could have references to the object, prefer functions like `PyList_GetItemRef()` or `PyDict_GetItemRef()` that return owned references (and will safely raise an error if the index/key doesn't exist).\n\n\n### Files to audit\n- [x] Modules/_abc.c\n- [ ] Modules/arraymodule.c - https://github.com/python/cpython/pull/130771\n- [x] Modules/_asynciomodule.c\n- [x] Modules/atexitmodule.c\n- [ ] Modules/binascii.c\n- [x] Modules/_bisectmodule.c\n- [x] Modules/_blake2/blake2module.c\n- [x] Modules/_bz2module.c\n- [ ] Modules/cjkcodecs/cjkcodecs.h\n- [ ] Modules/cjkcodecs/multibytecodec.c\n- [x] Modules/cmathmodule.c\n- [x] Modules/_codecsmodule.c\n- [x] Modules/_collectionsmodule.c\n- [ ] Modules/_contextvarsmodule.c\n- [x] Modules/_csv.c\n- [x] Modules/_ctypes/_ctypes.c\n- [x] Modules/_ctypes/_ctypes_test.c\n- [ ] Modules/_cursesmodule.c\n- [ ] Modules/_curses_panel.c\n- [ ] Modules/_datetimemodule.c\n- [x] Modules/_dbmmodule.c\n- [ ] Modules/_decimal/_decimal.c\n- [ ] Modules/_elementtree.c\n- [x] Modules/errnomodule.c\n- [ ] Modules/faulthandler.c\n- [ ] Modules/fcntlmodule.c\n- [x] Modules/_functoolsmodule.c\n- [x] Modules/gcmodule.c\n- [x] Modules/_gdbmmodule.c\n- [x] Modules/grpmodule.c\n- [x] Modules/_hashopenssl.c\n- [x] Modules/_heapqmodule.c\n- [x] Modules/_io/_iomodule.c\n- [ ] Modules/itertoolsmodule.c\n- [x] Modules/_json.c - https://github.com/python/cpython/pull/119438\n- [ ] Modules/_localemodule.c\n- [x] Modules/_lsprof.c\n- [x] Modules/_lzmamodule.c\n- [x] Modules/mathmodule.c\n- [x] Modules/md5module.c\n- [x] Modules/mmapmodule.c\n- [ ] Modules/_multiprocessing/multiprocessing.c\n- [ ] Modules/_multiprocessing/posixshmem.c\n- [ ] Modules/_opcode.c\n- [ ] Modules/_operator.c\n- [ ] Modules/overlapped.c\n- [ ] Modules/_pickle.c\n- [ ] Modules/posixmodule.c\n- [ ] Modules/_posixsubprocess.c\n- [x] Modules/pwdmodule.c\n- [ ] Modules/pyexpat.c\n- [x] Modules/_queuemodule.c\n- [x] Modules/_randommodule.c\n- [ ] Modules/readline.c\n- [x] Modules/resource.c\n- [ ] Modules/_scproxy.c\n- [x] Modules/selectmodule.c\n- [x] Modules/sha1module.c\n- [x] Modules/sha2module.c\n- [x] Modules/sha3module.c\n- [ ] Modules/signalmodule.c\n- [x] Modules/socketmodule.c\n- [ ] Modules/_sqlite/module.c\n- [x] Modules/_sre/sre.c\n- [x] Modules/_ssl.c\n- [ ] Modules/_stat.c\n- [x] Modules/_statisticsmodule.c\n- [x] Modules/_struct.c\n- [x] Modules/_suggestions.c\n- [ ] Modules/symtablemodule.c\n- [ ] Modules/_sysconfig.c\n- [x] Modules/syslogmodule.c\n- [x] Modules/termios.c\n- [ ] Modules/_testbuffer.c\n- [ ] Modules/_testcapimodule.c\n- [ ] Modules/_testclinic.c\n- [ ] Modules/_testclinic_limited.c\n- [ ] Modules/_testexternalinspection.c\n- [ ] Modules/_testimportmultiple.c\n- [ ] Modules/_testinternalcapi.c\n- [ ] Modules/_testlimitedcapi.c\n- [ ] Modules/_testmultiphase.c\n- [ ] Modules/_testsinglephase.c\n- [x] Modules/_threadmodule.c\n- [ ] Modules/timemodule.c\n- [ ] Modules/_tkinter.c\n- [ ] Modules/_tracemalloc.c\n- [ ] Modules/_typingmodule.c\n- [ ] Modules/unicodedata.c\n- [x] Modules/_uuidmodule.c\n- [x] Modules/_weakref.c\n- [ ] Modules/_winapi.c\n- [ ] Modules/_xxinterpchannelsmodule.c\n- [ ] Modules/_xxinterpqueuesmodule.c\n- [ ] Modules/xxlimited_35.c\n- [ ] Modules/xxlimited.c\n- [ ] Modules/xxmodule.c\n- [ ] Modules/_xxsubinterpretersmodule.c\n- [ ] Modules/xxsubtype.c\n- [ ] Modules/_xxtestfuzz/_xxtestfuzz.c\n- [x] Modules/zlibmodule.c\n- [x] Modules/_zoneinfo.c\n- [ ] Objects/genobject.c (including ag_running_async)\n- [ ] PC/msvcrtmodule.c\n- [ ] PC/python3dll.c\n- [ ] PC/winreg.c\n- [ ] PC/winsound.c\n- [ ] Python/bltinmodule.c\n- [ ] Python/import.c\n- [x] Python/instrumentation.c\n- [ ] Python/marshal.c\n- [ ] Python/Python-ast.c\n- [ ] Python/Python-tokenize.c\n- [ ] Python/sysmodule.c\n- [x] Python/_warnings.c\n\n\n\u003cdetails\u003e\n\u003csummary\u003eCompleted Issues\u003c/summary\u003e\n\n### Completed Issues\n- [ ] https://github.com/python/cpython/issues/116664\n- [ ] https://github.com/python/cpython/issues/114271\n- [ ] https://github.com/python/cpython/issues/112062\n- [ ] https://github.com/python/cpython/issues/111916\n- [ ] https://github.com/python/cpython/issues/116616\n\n\u003c/details\u003e\n\n### Has this already been discussed elsewhere?\n\nI have already discussed this feature proposal on Discourse\n\n### Links to previous discussion of this feature:\n\nhttps://peps.python.org/pep-0703/, especially [this section](https://peps.python.org/pep-0703/#py-mod-gil-slot)\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* gh-117488\n* gh-117530\n* gh-118344\n* gh-119438\n* gh-120103\n* gh-125328\n* gh-131319\n* gh-135036\n* gh-135196\n* gh-135309\n* gh-135434\n* gh-136555\n* gh-136658\n* gh-136695\n* gh-136760\n* gh-137021\n* gh-137221\n* gh-137222\n* gh-138229\n* gh-138339\n* gh-138395\n* gh-138467\n* gh-138504\n* gh-138575\n* gh-138793\n* gh-139237\n* gh-139825\n* gh-139987\n* gh-140068\n* gh-140229\n* gh-140321\n* gh-140391\n* gh-140459\n* gh-140555\n* gh-140711\n* gh-141365\n* gh-141825\n* gh-141923\n* gh-141990\n* gh-142161\n* gh-142261\n* gh-142432\n* gh-142756\n* gh-142947\n* gh-143074\n* gh-143234\n\u003c!-- /gh-linked-prs --\u003e\n","author":{"url":"https://github.com/swtaarrs","@type":"Person","name":"swtaarrs"},"datePublished":"2024-03-13T16:52:32.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/116738/cpython/issues/116738"}