René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[BUG]: Legacy driver callback bookkeeping races under free-threaded Python, causing native abort","articleBody":"### Is this a duplicate?\n\n- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cuda-python/issues) for this bug and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)\n\n### Type of Bug\n\nRuntime Error\n\n### Component\n\ncuda.bindings\n\n### Describe the bug\n\nA nightly CI job aborted with exit code 134 while running tests/legacy_api/test_legacy_cuda.py::test_culogs_functions with four parallel threads.\n[Failing job](https://github.com/NVIDIA/cuda-python/actions/runs/40791244909/job/123179307143)\n\nEnvironment:\n- Commit: d649075a50b97245b63381052481f2eaadd57f1c\n- Linux aarch64, Python 3.15.2 free-threaded\n- CUDA Toolkit 13.4.2, NVIDIA driver 595.99.02, L4 GPU\n- pytest-run-parallel==0.10.0, --parallel-threads=4, --count=5\nThe Python stack points to cuLogsUnregisterCallback(callback_handle). Disassembly of the exact wheel places the native abort in map-node deletion during std::map::erase().\nThe legacy bindings store callback allocations in the shared m_global._allocated C++ map. \n\ncuLogsRegisterCallback inserts entries, while cuLogsUnregisterCallback frees callback data and erases entries without synchronization. Concurrent access under free-threaded Python is the likely cause of memory corruption. Async-notification callback APIs also access this map.\n\nExpected behavior: concurrent registration and unregistration of independent callbacks completes safely.\n\nSuggested fix: synchronize callback bookkeeping at the binding-generation source, covering all APIs that share the map, and add concurrent regression coverage. Serializing the affected test could provide an interim CI mitigation.\n\nThe full CUDA test has not been reproduced locally. The matching ARM Python 3.14t job passed on the same driver version, suggesting an intermittent race.\n\nSome history: \n\nRecent test changes likely exposed an older bug. I found no recent change introducing the unsafe map operations.\n- May 2025: The cuLogs* wrappers already used the shared map without synchronization. ([#597](https://github.com/NVIDIA/cuda-python/pull/597))\n- September 2, 2026: CI enabled four parallel test threads for free-threaded Python. ([#2194](https://github.com/NVIDIA/cuda-python/pull/2194))\n- October 5: test_culogs_functions was added, exercising registration and cleanup. ([#2978](https://github.com/NVIDIA/cuda-python/pull/2978))\n- October 15: Context pooling increased actual overlap between worker threads. The commit explicitly notes that context creation previously kept them apart. This is the strongest candidate for making the existing race easier to trigger. ([#3122](https://github.com/NVIDIA/cuda-python/pull/3122))\n\n\n### How to Reproduce\n\nNot reproducible. \n\n### Expected behavior\n\nExpected behavior: concurrent registration and unregistration of independent callbacks completes safely.\n\n\n### Operating System\n\n_No response_\n\n### nvidia-smi output\n\n_No response_","author":{"url":"https://github.com/jpascucci-nv","@type":"Person","name":"jpascucci-nv"},"datePublished":"2026-10-29T15:15:21Z","url":"https://github.com/NVIDIA/cuda-python/issues/3282"}