René's URL Explorer Experiment

Title: gh-149816: Fix SNI callback callable race by kiri11 · Pull Request #150018 · python/cpython · GitHub

Open Graph Title: gh-149816: Fix SNI callback callable race by kiri11 · Pull Request #150018 · python/cpython

X Title: gh-149816: Fix SNI callback callable race by kiri11 · Pull Request #150018 · python/cpython

Description: Problem Fix a use-after-free race in the SSL SNI callback on free-threaded builds. When sni_callback is replaced or cleared on one thread while another thread is mid-handshake, the old callback object could be freed before the handshake thread finishes calling it. Reader side (_servername_callback): acquire sslctx's critical section, snapshot the callback with Py_XNewRef into a local, then release. The local strong reference keeps the callable alive for the duration of the call. Setter side (sni_callback_set): already holds the same critical section via @critical_section clinic annotation. Uses Py_XSETREF to swap the pointer and decref the old callback atomically. The OpenSSL callback registration is set after the pointer is stored, so a concurrent handshake always finds a valid callable. Setter clearing (sni_callback = None): unregisters the OpenSSL callback first, then clears the Python reference. Serialized against the reader by the same critical section. Testing Adds a free-threading stress test that spawns handshake workers and a callback-toggling thread concurrently. Plus manually tested on MacOS. Before % ./python.exe -m test test_ssl -m test_sni_callback_race Raised RLIMIT_NOFILE: 256 -> 1024 Using random seed: 2655205590 0:00:00 load avg: 4.31 Run 1 test sequentially in a single process 0:00:00 load avg: 4.31 [1/1] test_ssl ================== WARNING: ThreadSanitizer: data race (pid=68672) Write of size 8 at 0x000126f10588 by thread T29: #0 _ssl__SSLContext_sni_callback_set_impl _ssl.c:5304 (_ssl.cpython-316t-darwin.so:arm64+0xe5f0) #1 _ssl__SSLContext_sni_callback_set _ssl.c.h:2266 (_ssl.cpython-316t-darwin.so:arm64+0xe5f0) #2 _PyObject_GenericSetAttrWithDict object.c:2049 (python.exe:arm64+0x1000dd900) #3 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10026e378) Previous read of size 8 at 0x000126f10588 by thread T13: #0 _servername_callback _ssl.c:5159 (_ssl.cpython-316t-darwin.so:arm64+0xffa0) #1 final_server_name (libssl.3.dylib:arm64+0x5c31c) #2 _PyEval_EvalFrameDefault generated_cases.c.h:4142 (python.exe:arm64+0x1001c5b50) #3 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10026e378) Thread T29 (tid=707684, running) created by main thread at: #0 pthread_create (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84) #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x10026d9f8) #2 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10032782c) #3 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x100326894) #4 cfunction_call methodobject.c:564 (python.exe:arm64+0x1000d27b4) Thread T13 (tid=707668, running) created by main thread at: #0 pthread_create (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31d84) #1 do_start_joinable_thread thread_pthread.h:281 (python.exe:arm64+0x10026d9f8) #2 do_start_new_thread _threadmodule.c:1919 (python.exe:arm64+0x10032782c) #3 thread_PyThread_start_joinable_thread _threadmodule.c:2042 (python.exe:arm64+0x100326894) #4 cfunction_call methodobject.c:564 (python.exe:arm64+0x1000d27b4) SUMMARY: ThreadSanitizer: data race _ssl.c:5304 in _ssl__SSLContext_sni_callback_set_impl ================== Warning -- Unraisable exception Exception ignored in ssl servername callback while calling set SNI callback : Traceback (most recent call last): File "/Users/kignatev/PycharmProjects/cpython/Lib/ssl.py", line 963, in do_handshake self._sslobj.do_handshake() ~~~~~~~~~~~~~~~~~~~~~~~~~^^ SystemError: null argument to internal routine Fatal Python error: Segmentation fault Stack (most recent call first): Current thread's C stack trace (most recent call first): Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _Py_DumpStack+0x44 [0x104db912c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at faulthandler_fatal_error+0x330 [0x104dddf98] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _ZN6__tsanL21CallUserSignalHandlerEPNS_11ThreadStateEbbiPN11__sanitizer23__sanitizer_siginfo_padEPv+0x104 [0x10570f910] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _Z10sighandleriPN11__sanitizer23__sanitizer_siginfo_padEPv+0x1a8 [0x10570fb68] Binary file "/usr/lib/system/libsystem_platform.dylib", at _sigtramp+0x38 [0x182e257a4] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_EvalFrameDefault+0xc71c [0x104d152e4] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_Vector+0x2d8 [0x104d076ec] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at object_vacall+0x138 [0x104babb78] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at PyObject_CallFunctionObjArgs+0x38 [0x104babce0] Binary file "/Users/kignatev/PycharmProjects/cpython/Modules/_ssl.cpython-316t-darwin.so", at _servername_callback+0x274 [0x10f9c01c8] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at final_server_name+0x6c [0x113db4320] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at tls_parse_all_extensions+0xa8 [0x113db3a10] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at tls_post_process_client_hello+0x560 [0x113dcf678] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at state_machine+0x59c [0x113dbe824] Binary file "/Users/kignatev/PycharmProjects/cpython/Modules/_ssl.cpython-316t-darwin.so", at _ssl__SSLSocket_do_handshake+0x184 [0x10f9c093c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_EvalFrameDefault+0x4f8c [0x104d0db54] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_Vector+0x2d8 [0x104d076ec] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyObject_VectorcallPrepend+0x144 [0x104bab0c8] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at context_run+0x88 [0x104d3b77c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at PyObject_Vectorcall+0x58 [0x104ba9a40] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _Py_VectorCallInstrumentation_StackRefSteal+0x12c [0x104d07d10] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_EvalFrameDefault+0x21b4 [0x104d0ad7c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_Vector+0x2d8 [0x104d076ec] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyObject_VectorcallPrepend+0x144 [0x104bab0c8] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at thread_run+0xe0 [0x104e6ff58] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at pythread_wrapper+0x1c [0x104db637c] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at __tsan_thread_start_func+0x8c [0x105709cf8] Binary file "/usr/lib/system/libsystem_pthread.dylib", at _pthread_start+0x88 [0x182e1bc58] Binary file "/usr/lib/system/libsystem_pthread.dylib", at thread_start+0x8 [0x182e16c1c] ThreadSanitizer:DEADLYSIGNAL ==68672==ERROR: ThreadSanitizer: SEGV on unknown address 0x000000000000 (pc 0x000104d0ef10 bp 0x00017e422240 sp 0x00017e421fe0 T707674) ==68672==The signal is caused by a READ memory access. ==68672==Hint: address points to the zero page. Warning -- Unraisable exception Exception ignored in ssl servername callback while calling set SNI callback : Traceback (most recent call last): File "/Users/kignatev/PycharmProjects/cpython/Lib/ssl.py", line 963, in do_handshake self._sslobj.do_handshake() ~~~~~~~~~~~~~~~~~~~~~~~~~^^ SystemError: null argument to internal routine #0 _PyEval_EvalFrameDefault generated_cases.c.h:5550 (python.exe:arm64+0x1001c6f10) #1 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x1001bf6e8) #2 _PyEval_Vector ceval.c:2156 (python.exe:arm64+0x1001bf6e8) #3 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100063b74) #4 object_vacall call.c:823 (python.exe:arm64+0x100063b74) #5 PyObject_CallFunctionObjArgs call.c:977 (python.exe:arm64+0x100063cdc) #6 _servername_callback _ssl.c:5214 (_ssl.cpython-316t-darwin.so:arm64+0x101c4) #7 final_server_name (libssl.3.dylib:arm64+0x5c31c) #8 tls_parse_all_extensions (libssl.3.dylib:arm64+0x5ba0c) #9 tls_post_process_client_hello (libssl.3.dylib:arm64+0x77674) #10 state_machine (libssl.3.dylib:arm64+0x66820) #11 _ssl__SSLSocket_do_handshake_impl _ssl.c:1098 (_ssl.cpython-316t-darwin.so:arm64+0x10938) #12 _ssl__SSLSocket_do_handshake _ssl.c.h:30 (_ssl.cpython-316t-darwin.so:arm64+0x10938) #13 _PyEval_EvalFrameDefault generated_cases.c.h:4142 (python.exe:arm64+0x1001c5b50) #14 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x1001bf6e8) #15 _PyEval_Vector ceval.c:2156 (python.exe:arm64+0x1001bf6e8) #16 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1000630c4) #17 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x1000630c4) #18 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1001f3778) #19 context_run context.c:727 (python.exe:arm64+0x1001f3778) #20 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x100061a3c) #21 PyObject_Vectorcall call.c:327 (python.exe:arm64+0x100061a3c) #22 _Py_VectorCallInstrumentation_StackRefSteal ceval.c:775 (python.exe:arm64+0x1001bfd0c) #23 _PyEval_EvalFrameDefault generated_cases.c.h:1846 (python.exe:arm64+0x1001c2d78) #24 _PyEval_EvalFrame pycore_ceval.h:122 (python.exe:arm64+0x1001bf6e8) #25 _PyEval_Vector ceval.c:2156 (python.exe:arm64+0x1001bf6e8) #26 _PyObject_VectorcallTstate pycore_call.h:144 (python.exe:arm64+0x1000630c4) #27 _PyObject_VectorcallPrepend call.c:855 (python.exe:arm64+0x1000630c4) #28 thread_run _threadmodule.c:388 (python.exe:arm64+0x100327f54) #29 pythread_wrapper thread_pthread.h:234 (python.exe:arm64+0x10026e378) #30 __tsan_thread_start_func (libclang_rt.tsan_osx_dynamic.dylib:arm64+0x31cf4) #31 _pthread_start (libsystem_pthread.dylib:arm64e+0x6c54) #32 thread_start (libsystem_pthread.dylib:arm64e+0x1c18) ==68672==Register values: x[0] = 0x000000010e21dc88 x[1] = 0x00000001171c02e8 x[2] = 0x0000000000000000 x[3] = 0x000000017e422250 x[4] = 0x0000000000000003 x[5] = 0x0000000000000000 x[6] = 0x0000000000000000 x[7] = 0x0000000000366b00 x[8] = 0x0000000000000030 x[9] = 0x00000001171c0338 x[10] = 0x0000000000000001 x[11] = 0x0000000000000003 x[12] = 0x000000010505bda1 x[13] = 0x0000000000000003 x[14] = 0x0000000000000075 x[15] = 0x00000001eec259a0 x[16] = 0x0000000000000018 x[17] = 0x000000010e21dc70 x[18] = 0x0000000000000000 x[19] = 0x00000001171c0328 x[20] = 0x00000001171c0358 x[21] = 0x0000000104fd7da8 x[22] = 0x000000030e1001c0 x[23] = 0x000000030e1001c0 x[24] = 0x0000000000000000 x[25] = 0x0000000000000001 x[26] = 0x000000000000001d x[27] = 0x00000001171c02e8 x[28] = 0x00000001240b8000 fp = 0x000000017e422240 lr = 0x0000000104d152e4 sp = 0x000000017e421fe0 ThreadSanitizer can not provide additional info. SUMMARY: ThreadSanitizer: SEGV generated_cases.c.h:5550 in _PyEval_EvalFrameDefault ==68672==ABORTING Fatal Python error: Aborted Stack (most recent call first): Current thread's C stack trace (most recent call first): Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _Py_DumpStack+0x44 [0x104db912c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at faulthandler_fatal_error+0x330 [0x104dddf98] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _ZN6__tsanL21CallUserSignalHandlerEPNS_11ThreadStateEbbiPN11__sanitizer23__sanitizer_siginfo_padEPv+0x104 [0x10570f910] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _Z10sighandleriPN11__sanitizer23__sanitizer_siginfo_padEPv+0x1a8 [0x10570fb68] Binary file "/usr/lib/system/libsystem_platform.dylib", at _sigtramp+0x38 [0x182e257a4] Binary file "/usr/lib/system/libsystem_pthread.dylib", at pthread_kill+0x128 [0x182e1b8d8] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at wrap_pthread_kill+0x134 [0x105710278] Binary file "/usr/lib/system/libsystem_c.dylib", at abort+0x94 [0x182d22790] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _ZN11__sanitizer6AtexitEPFvvE+0x0 [0x1056f11b4] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _ZN11__sanitizer3DieEv+0x68 [0x1056f06e4] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _ZN11__sanitizer13SignalContextC1EPvS1_+0x0 [0x1056fd974] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _ZN6__tsanL21CallUserSignalHandlerEPNS_11ThreadStateEbbiPN11__sanitizer23__sanitizer_siginfo_padEPv+0x104 [0x10570f910] Binary file "/opt/homebrew/Cellar/llvm/22.1.5/lib/clang/22/lib/darwin/libclang_rt.tsan_osx_dynamic.dylib", at _Z10sighandleriPN11__sanitizer23__sanitizer_siginfo_padEPv+0x1a8 [0x10570fb68] Binary file "/usr/lib/system/libsystem_platform.dylib", at _sigtramp+0x38 [0x182e257a4] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_EvalFrameDefault+0xc71c [0x104d152e4] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_Vector+0x2d8 [0x104d076ec] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at object_vacall+0x138 [0x104babb78] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at PyObject_CallFunctionObjArgs+0x38 [0x104babce0] Binary file "/Users/kignatev/PycharmProjects/cpython/Modules/_ssl.cpython-316t-darwin.so", at _servername_callback+0x274 [0x10f9c01c8] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at final_server_name+0x6c [0x113db4320] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at tls_parse_all_extensions+0xa8 [0x113db3a10] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at tls_post_process_client_hello+0x560 [0x113dcf678] Binary file "/opt/homebrew/Cellar/openssl@3/3.6.2/lib/libssl.3.dylib", at state_machine+0x59c [0x113dbe824] Binary file "/Users/kignatev/PycharmProjects/cpython/Modules/_ssl.cpython-316t-darwin.so", at _ssl__SSLSocket_do_handshake+0x184 [0x10f9c093c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_EvalFrameDefault+0x4f8c [0x104d0db54] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_Vector+0x2d8 [0x104d076ec] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyObject_VectorcallPrepend+0x144 [0x104bab0c8] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at context_run+0x88 [0x104d3b77c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at PyObject_Vectorcall+0x58 [0x104ba9a40] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _Py_VectorCallInstrumentation_StackRefSteal+0x12c [0x104d07d10] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_EvalFrameDefault+0x21b4 [0x104d0ad7c] Binary file "/Users/kignatev/PycharmProjects/cpython/python.exe", at _PyEval_Vector+0x2d8 [0x104d076ec] zsh: abort ./python.exe -m test test_ssl -m test_sni_callback_race After 0:00:00 load avg: 4.18 [1/1] test_ssl passed Using Repro: % ./python.exe cpython-ft/repros/49.py finding: 49 SSLContext.sni_callback replacement race expected signal: native crash/fatal error/sanitizer use-after-free during concurrent SNI handshakes parameters: seconds=10; Python=3.16.0a0; GIL_enabled=False; workers=28 ------------------------------------------------------------------------ Completed without interpreter crash: 53271 handshakes, 0 Python-level failures, 245825 toggles. More info Issue: gh-149816

Open Graph Description: Problem Fix a use-after-free race in the SSL SNI callback on free-threaded builds. When sni_callback is replaced or cleared on one thread while another thread is mid-handshake, the old callback obj...

X Description: Problem Fix a use-after-free race in the SSL SNI callback on free-threaded builds. When sni_callback is replaced or cleared on one thread while another thread is mid-handshake, the old callback obj...

Opengraph URL: https://github.com/python/cpython/pull/150018

X: @github

direct link

Domain: github.com

Links:

Viewport: width=device-width