René's URL Explorer Experiment

Title: Python 3.14 free-threaded support by greateggsgreg · Pull Request #2721 · pythonnet/pythonnet · GitHub

Open Graph Title: Python 3.14 free-threaded support by greateggsgreg · Pull Request #2721 · pythonnet/pythonnet

X Title: Python 3.14 free-threaded support by greateggsgreg · Pull Request #2721 · pythonnet/pythonnet

Description: First step toward Python 3.14 free-threaded (Py_GIL_DISABLED) support, tracked under #2720 (and the umbrella #2610). After this PR, pythonnet runs the full pytest suite on python3.14t against both .NET 8 and .NET 10 on Linux. Existing GIL builds (3.11 - 3.14) are unaffected. What changes Free-threaded PyObject layout + refcount FT PyObject is 16 bytes longer (ob_tid + ob_flags + ob_mutex + ob_gc_bits + ob_ref_local + ob_ref_shared replaces ob_refcnt) and the refcount is split, so there is no single offset to read. ABI.Initialize detects the build via sys._is_gil_enabled() and sets ObjectHeadOffset = 16 on FT, shifting the generated TypeOffset* values to absolute PyHeapTypeObject offsets. Runtime.Refcount prefers a P/Invoke into Py_REFCNT (a real symbol on CPython 3.14+, required on FT). Older Pythons fall back to the existing direct read. Atomic type creation in ReflectedClrType.GetOrCreate / TypeManager.GetType The classic check-then-act on ClassManager.cache and TypeManager.cache raced two ways under FT: duplicate cache.Add (ArgumentException), and partial-type visibility from outside-the-lock readers observing a half-initialised type during recursive class init. Two-cache design: cache — only fully-initialised types; outside-the-lock fast path is safe. _inProgressCache — partial types visible only to the lock-holding builder, so self-referential definitions resolve via the reentrant lock. Serialisation snapshots remain Dictionary<,> on the wire for binary compatibility. Atomic GCHandle ownership in tp_clear / tp_dealloc subtype_clear (main thread) and the .NET finalizer thread can both reach a CLR-bound Python object's GCHandle slot. Previous read-then-zero in ManagedType.TryFreeGCHandle and ClassDerivedObject.tp_dealloc (strong→weak swap) let two threads observe the same handle and double-free it. Both paths now use Interlocked.Exchange to atomically claim the slot. Reflection.Emit is not thread-safe Concurrent TypeBuilder / ModuleBuilder operations corrupt the IL stream or throw "Duplicate type name within an assembly". Two emit-and-bake sequences are now serialised: ClassDerivedObject.CreateDerivedType (Python subclasses of CLR types) on _buildersLock. DelegateManager.GetDispatcher (Python callables → CLR delegates) on a new _emitLock. Concurrent collections / atomics on shared state Plain Dictionary<,>, HashSet<>, and counters that the GIL implicitly serialised tear under FT. ConcurrentDictionary<,>: ExtensionType.loadedExtensions, CLRObject.reflectedObjects, ClassManager.cache, TypeManager.cache, ModuleObject.cache / allNames, Interop.delegateTypes, ClassBase.ClearVisited, InternString._string2interns / _intern2strings. Interlocked + Volatile: Finalizer._throttled (counter), Runtime.run (epoch), PyBuffer.disposedValue (dispose flag — finalizer + explicit Dispose no longer double-free _view). Lock-protected (where nested mutation or list-with-equality semantics rule out lock-free containers): GenericUtil.mapping (nested Dictionary/List), PythonEngine.ShutdownHandlers, Runtime._pyRefs. volatile bool: Runtime._isInitialized / _typesInitialized, PythonEngine.initialized, Finalizer.started. Guard the .NET-finalizer / Python-shutdown interaction The .NET finalizer thread can dispatch Py_DecRef concurrently with Py_Finalize; a stale ob_ref_local read after teardown crashes the process. Runtime._Py_IsFinalizing() guards on Finalizer.ThrottledCollect, PyObject's finalizer, and the Refcount > 0 Debug.Assert in XDecref / AddFinalizedObject (kept on GIL, skipped on FT). Tests + CI New tests in tests/test_thread.py cover the refcount/ABI invariants, the ConcurrentDictionary caches, atomic GCHandle ownership, the two Reflection.Emit locks, and the ShutdownHandlers lock. Some tests are marked @freethreaded_only because the GIL-build code path triggers a separate pre-existing pythonnet crash under high-contention CLR allocation that's reproducible on master and out of scope for this PR. .github/workflows/main.yml adds 3.14t to the Linux/macOS matrix and skips Mono on 3.14t (clr-loader's mono backend isn't yet validated against free-threaded Python). Verification Linux aarch64, full pytest suite (tests/, --ignore=tests/domain_tests): .NET 8 .NET 10 3.11 - 3.14 478 passed, 11 skipped 478 passed, 11 skipped 3.14t 484 passed, 5 skipped 484 passed, 5 skipped (11 skipped on GIL = 5 pre-existing + 6 FT-only tests; 5 skipped on FT = the 5 pre-existing skips.) 3.13t is not in the matrix because cffi <2.0 does not support free-threaded 3.13 and clr-loader requires cffi.

Open Graph Description: First step toward Python 3.14 free-threaded (Py_GIL_DISABLED) support, tracked under #2720 (and the umbrella #2610). After this PR, pythonnet runs the full pytest suite on python3.14t against both ...

X Description: First step toward Python 3.14 free-threaded (Py_GIL_DISABLED) support, tracked under #2720 (and the umbrella #2610). After this PR, pythonnet runs the full pytest suite on python3.14t against both ...

Opengraph URL: https://github.com/pythonnet/pythonnet/pull/2721

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width