René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Disable GIL by default in free-threaded build","articleBody":"# Feature or enhancement\r\n\r\nThis issue is to track the remaining bugs that prevent the Python test suite from passing with the GIL disabled, as well as the PRs under review that fix known issues.\r\n\r\nNote that this is not intended to cover all free-threading related issues: some thread-safety issues don't result in tests crashing or failing and are not tracked here. This is intended to cover the *minimum* set of changes needed to ensure that the test suite passes reliably with the GIL disabled.\r\n\r\n### PRs\r\n\r\n* https://github.com/python/cpython/pull/117528\r\n* https://github.com/python/cpython/pull/117824\r\n* https://github.com/python/cpython/pull/118112\r\n\r\n### Integration branch\r\n\r\n* https://github.com/colesbury/cpython/tree/nogil-integration\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eBranches to merge\u003c/summary\u003e\r\n\r\n```\r\ngit merge disable-gil-default \u0026\u0026\r\ngit merge gh-112075-lock-split-keys \u0026\u0026\r\ngit merge gh-117783-immortalize\r\n```\r\n\u003c/details\u003e\r\n\r\nThis branch contains the merge of the above PRs and is used to find bugs that are not covered by outstanding PRs. Note that setting `PYTHON_GIL=0` isn't sufficient because some tests do not propagate the environment variable to tests in subprocesses. Instead, the branch changes the default mode to disable the GIL for testing purposes.\r\n\r\n### Issues without PRs that fix them\r\n\r\n* N/A\r\n\r\n### Bugs without issues\r\n\r\n**Windows**\r\n\r\n* test_no_stale_references times out - https://github.com/python/cpython/actions/runs/8514458340/job/23320231100?pr=117437\r\n\r\n**Linux**\r\n\r\n* `test.test_concurrent_futures.test_deadlock` deadlocks ? - https://buildbot.python.org/all/#/builders/1281/builds/148/steps/5/logs/stdio\r\n\r\n**macOS**\r\n\r\n* `test_interrupt_main_subthread` - `RuntimeError: release unlocked lock` https://buildbot.python.org/all/#/builders/1367/builds/117/steps/5/logs/stdio\r\n* `test.test_multiprocessing_spawn.test_processes.WithProcessesTestPool.test_release_task_refs` - `10 != 0`\r\n\r\nhttps://buildbot.python.org/all/#/builders/1367/builds/119/steps/5/logs/stdio\r\n* `test_daemon_threads_shutdown_stderr_deadlock` - `Assertion failed: (tstate == NULL || tstate-\u003einterp == interp), function _PyEval_ReleaseLock, file ceval_gil.c,`\r\n* `test_queue` `test_shutdown_all_methods_in_many_threads` - AttributeError: 'Queue' object has no attribute '_lock'\r\n\r\n```\r\ntest_shutdown_all_methods_in_many_threads (test.test_queue.CQueueTest.test_shutdown_all_methods_in_many_threads) ... Warning -- Uncaught thread exception: AttributeError\r\nException in thread Thread-2081 (_read_msg_thread):\r\nTraceback (most recent call last):\r\n  File \"/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py\", line 1039, in _bootstrap_inner\r\n    self.run()\r\n    ~~~~~~~~^^\r\n  File \"/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py\", line 990, in run\r\n    self._target(*self._args, **self._kwargs)\r\n    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\r\n  File \"/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/test/test_queue.py\", line 352, in _read_msg_thread\r\n    q.get(False)\r\n    ~~~~~^^^^^^^\r\n  File \"/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/queue.py\", line 216, in get\r\n    item = self._get()\r\n           ~~~~~~~~~^^\r\n  File \"/Users/ec2-user/buildbot/buildarea/pull_request.itamaro-macos-arm64-aws.macos-with-brew.refleak.nogil/build/Lib/threading.py\", line 304, in __enter__\r\n    return self._lock.__enter__()\r\n           ^^^^^^^^^^\r\nAttributeError: 'Queue' object has no attribute '_lock'\r\n```\r\n\r\n**Old bugs?**\r\n\r\n* `test.test_smtpnet.SmtpSSLTest.test_connect_default_port` connection closed (*possibly* related)\r\n* `test.test_multiprocessing_forkserver.test_processes` hangs ?\r\n\r\n\r\n\u003c!-- gh-linked-prs --\u003e\r\n### Linked PRs\r\n* gh-117437\r\n* gh-118295\n* gh-141804\n* gh-141911\n\u003c!-- /gh-linked-prs --\u003e\r\n","author":{"url":"https://github.com/colesbury","@type":"Person","name":"colesbury"},"datePublished":"2024-03-13T18:19:06.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":2},"url":"https://github.com/116749/cpython/issues/116749"}