René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"UBSan: Calling a function through pointer to incorrect function type is undefined behavior","articleBody":"# Bug report\n\n### Bug description:\n\nUBSan (UndefinedBehaviorSanitizer) in LLVM.org Clang 17 makes [`-fsanitize=function`](https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks) available for C; previously, it was only for C++. (So it may also be made available in future Apple Xcode clang and GCC.) By default, it is implied by \u003ctt\u003e-fsanitize=undefined\u003c/tt\u003e (which is what `./configure --with-undefined-behavior-sanitizer` uses), but it can be disabled using \u003ctt\u003e-fno-sanitize=function\u003c/tt\u003e.\n\nFor a project such as CPython, which has long relied on function pointers for callbacks, yet seems to have only required that callbacks behave as expected under typical ABI calling conventions, rather than more strictly be declared/defined as a type compatible with the function pointer they will be called as, this leads to *numerous* errors from UBSan.\n\u003cdetails\u003e\n\u003csummary\u003eExamples when starting Python REPL:\u003c/summary\u003e\n\u003cpre\u003e\n% ./python.exe           \nObjects/object.c:2731:5: runtime error: call to function list_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:347: note: list_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:2731:5 in \nObjects/object.c:878:16: runtime error: call to function long_hash through pointer to incorrect function type 'long (*)(struct _object *)'\nlongobject.c:3295: note: long_hash defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:878:16 in \nInclude/internal/pycore_object.h:365:43: runtime error: call to function type_is_gc through pointer to incorrect function type 'int (*)(struct _object *)'\ntypeobject.c:5347: note: type_is_gc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Include/internal/pycore_object.h:365:43 in \nObjects/abstract.c:157:26: runtime error: call to function dict_subscript through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\ndictobject.c:2511: note: dict_subscript defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:157:26 in \nObjects/abstract.c:2954:14: runtime error: call to function tupleiter_next through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ntupleobject.c:999: note: tupleiter_next defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:2954:14 in \nObjects/abstract.c:236:19: runtime error: call to function dict_ass_sub through pointer to incorrect function type 'int (*)(struct _object *, struct _object *, struct _object *)'\ndictobject.c:2546: note: dict_ass_sub defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:236:19 in \nObjects/call.c:242:18: runtime error: call to function type_call through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ntypeobject.c:1647: note: type_call defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/call.c:242:18 in \nObjects/typeobject.c:10309:24: runtime error: call to function classmethod_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:94: note: classmethod_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:10309:24 in \nModules/gcmodule.c:493:16: runtime error: call to function list_traverse through pointer to incorrect function type 'int (*)(struct _object *, int (*)(struct _object *, void *), void *)'\nlistobject.c:2704: note: list_traverse defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Modules/gcmodule.c:493:16 in \nModules/gcmodule.c:605:20: runtime error: call to function list_traverse through pointer to incorrect function type 'int (*)(struct _object *, int (*)(struct _object *, void *), void *)'\nlistobject.c:2704: note: list_traverse defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Modules/gcmodule.c:605:20 in \nObjects/dictobject.c:3569:17: runtime error: call to function visit_reachable through pointer to incorrect function type 'int (*)(struct _object *, void *)'\ngcmodule.c:502: note: visit_reachable defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/dictobject.c:3569:17 in \nObjects/descrobject.c:694:5: runtime error: call to function visit_reachable through pointer to incorrect function type 'int (*)(struct _object *, void *)'\ngcmodule.c:502: note: visit_reachable defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:694:5 in \n[…]\nObjects/typeobject.c:4892:19: runtime error: call to function classmethod_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:94: note: classmethod_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:4892:19 in \nPython/generated_cases.c.h:3859:17: runtime error: call to function func_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nfuncobject.c:913: note: func_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3859:17 in \nObjects/object.c:1442:19: runtime error: call to function member_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:160: note: member_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1442:19 in \nObjects/object.c:1503:15: runtime error: call to function method_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:136: note: method_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1503:15 in \nPython/generated_cases.c.h:3857:13: runtime error: call to function meth_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nmethodobject.c:160: note: meth_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3857:13 in \nObjects/typeobject.c:2212:19: runtime error: call to function method_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:136: note: method_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:2212:19 in \nObjects/descrobject.c:188:16: runtime error: call to function func_get_name through pointer to incorrect function type 'struct _object *(*)(struct _object *, void *)'\nfuncobject.c:582: note: func_get_name defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:188:16 in \nPython/generated_cases.c.h:3248:20: runtime error: call to function tupleiter_next through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ntupleobject.c:999: note: tupleiter_next defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3248:20 in \nObjects/object.c:1564:19: runtime error: call to function member_set through pointer to incorrect function type 'int (*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:227: note: member_set defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1564:19 in \nObjects/descrobject.c:241:16: runtime error: call to function func_set_name through pointer to incorrect function type 'int (*)(struct _object *, struct _object *, void *)'\nfuncobject.c:588: note: func_set_name defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:241:16 in \nPython/generated_cases.c.h:3364:21: runtime error: call to function tupleiter_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:984: note: tupleiter_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3364:21 in \nPython/ceval.c:591:5: runtime error: call to function func_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nfuncobject.c:913: note: func_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/ceval.c:591:5 in \nObjects/object.c:1031:18: runtime error: call to function _Py_module_getattro through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\nmoduleobject.c:877: note: _Py_module_getattro defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1031:18 in \nObjects/abstract.c:2895:25: runtime error: call to function dictitems_iter through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ndictobject.c:5180: note: dictitems_iter defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:2895:25 in \nPython/generated_cases.c.h:3193:13: runtime error: call to function dictview_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ndictobject.c:4587: note: dictview_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3193:13 in \nPython/ceval.c:1929:13: runtime error: call to function tupleiter_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:984: note: tupleiter_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/ceval.c:1929:13 in \nObjects/abstract.c:2334:19: runtime error: call to function tuplecontains through pointer to incorrect function type 'int (*)(struct _object *, struct _object *)'\ntupleobject.c:354: note: tuplecontains defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:2334:19 in \nObjects/typeobject.c:1698:19: runtime error: call to function AttributeError_init through pointer to incorrect function type 'int (*)(struct _object *, struct _object *, struct _object *)'\nexceptions.c:2279: note: AttributeError_init defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:1698:19 in \nPython/generated_cases.c.h:1351:13: runtime error: call to function AttributeError_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nexceptions.c:2315: note: AttributeError_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:1351:13 in \nPython/generated_cases.c.h:623:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:623:13 in \nPython/generated_cases.c.h:749:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:749:13 in \nPython/generated_cases.c.h:3809:17: runtime error: call to function method_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nclassobject.c:236: note: method_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3809:17 in \nObjects/descrobject.c:393:24: runtime error: call to function dict_pop through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *const *, long)'\ndictobject.c.h:138: note: dict_pop defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:393:24 in \nPython/generated_cases.c.h:3260:17: runtime error: call to function dictiter_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ndictobject.c:4047: note: dictiter_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3260:17 in \nObjects/object.c:1699:15: runtime error: call to function long_bool through pointer to incorrect function type 'int (*)(struct _object *)'\nlongobject.c:4890: note: long_bool defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1699:15 in \nPython/generated_cases.c.h:4816:13: runtime error: call to function dict_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ndictobject.c:2373: note: dict_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4816:13 in \nPython/generated_cases.c.h:3896:17: runtime error: call to function method_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nclassobject.c:236: note: method_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3896:17 in \nPython/generated_cases.c.h:4625:19: runtime error: call to function dict_pop through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *const *, long)'\ndictobject.c.h:138: note: dict_pop defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4625:19 in \nObjects/descrobject.c:467:24: runtime error: call to function list_append through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\nlistobject.c:842: note: list_append defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:467:24 in \nInclude/internal/pycore_object.h:142:9: runtime error: call to function PyObject_Free through pointer to incorrect function type 'void (*)(struct _object *)'\nobmalloc.c:829: note: PyObject_Free defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Include/internal/pycore_object.h:142:9 in \nObjects/call.c:361:18: runtime error: call to function type_call through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ntypeobject.c:1647: note: type_call defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/call.c:361:18 in \nPython/generated_cases.c.h:2931:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2931:13 in \nPython/generated_cases.c.h:3362:25: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3362:25 in \nPython/generated_cases.c.h:928:13: runtime error: call to function list_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:347: note: list_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:928:13 in \nPython/ceval.c:1604:5: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/ceval.c:1604:5 in \nPython/bltinmodule.c:378:16: runtime error: call to function gen_iternext through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ngenobject.c:609: note: gen_iternext defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/bltinmodule.c:378:16 in \nObjects/object.c:1702:15: runtime error: call to function list_length through pointer to incorrect function type 'long (*)(struct _object *)'\nlistobject.c:438: note: list_length defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1702:15 in \nPython/generated_cases.c.h:3519:13: runtime error: call to function method_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nclassobject.c:236: note: method_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3519:13 in \nObjects/methodobject.c:540:18: runtime error: call to function rlock_acquire through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\n_threadmodule.c:314: note: rlock_acquire defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/methodobject.c:540:18 in \nPython/generated_cases.c.h:2767:13: runtime error: call to function list_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:347: note: list_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2767:13 in \nObjects/methodobject.c:551:18: runtime error: call to function rlock_release through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\n_threadmodule.c:364: note: rlock_release defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/methodobject.c:551:18 in \nObjects/typeobject.c:2096:5: runtime error: call to function list_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:347: note: list_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:2096:5 in \nPython/generated_cases.c.h:130:13: runtime error: call to function method_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nclassobject.c:236: note: method_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:130:13 in \nObjects/object.c:1181:15: runtime error: call to function type_setattro through pointer to incorrect function type 'int (*)(struct _object *, struct _object *, struct _object *)'\ntypeobject.c:4938: note: type_setattro defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:1181:15 in \nPython/generated_cases.c.h:1510:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:1510:13 in \nPython/generated_cases.c.h:164:13: runtime error: call to function listiter_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:3222: note: listiter_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:164:13 in \n[…]\n\nModules/gcmodule.c:1033:24: runtime error: call to function _list_clear through pointer to incorrect function type 'int (*)(struct _object *)'\nlistobject.c:597: note: _list_clear defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Modules/gcmodule.c:1033:24 in \nObjects/abstract.c:62:26: runtime error: call to function list_length through pointer to incorrect function type 'long (*)(struct _object *)'\nlistobject.c:438: note: list_length defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:62:26 in \nObjects/abstract.c:270:19: runtime error: call to function dict_ass_sub through pointer to incorrect function type 'int (*)(struct _object *, struct _object *, struct _object *)'\ndictobject.c:2546: note: dict_ass_sub defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:270:19 in \nPython/bltinmodule.c:329:16: runtime error: call to function gen_iternext through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ngenobject.c:609: note: gen_iternext defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/bltinmodule.c:329:16 in \nObjects/abstract.c:1141:18: runtime error: call to function tupleconcat through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\ntupleobject.c:443: note: tupleconcat defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:1141:18 in \nObjects/abstract.c:440:15: runtime error: call to function bytes_buffer_getbuffer through pointer to incorrect function type 'int (*)(struct _object *, Py_buffer *, int)'\nbytesobject.c:1663: note: bytes_buffer_getbuffer defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:440:15 in \nObjects/methodobject.c:441:24: runtime error: call to function int_from_bytes through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *const *, long, struct _object *)'\nlongobject.c.h:396: note: int_from_bytes defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/methodobject.c:441:24 in \n[…]\nPython/generated_cases.c.h:4814:13: runtime error: call to function method_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nclassobject.c:236: note: method_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4814:13 in \nPython/generated_cases.c.h:4815:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4815:13 in \nPython/generated_cases.c.h:1529:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:1529:13 in \nPython/generated_cases.c.h:648:17: runtime error: call to function slice_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nsliceobject.c:359: note: slice_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:648:17 in \nObjects/abstract.c:953:13: runtime error: call to function long_add through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\nlongobject.c:3473: note: long_add defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:953:13 in \nPython/generated_cases.c.h:3322:21: runtime error: call to function listiter_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:3222: note: listiter_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3322:21 in \nPython/generated_cases.c.h:2821:13: runtime error: call to function PyObject_Free through pointer to incorrect function type 'void (*)(struct _object *)'\nobmalloc.c:829: note: PyObject_Free defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2821:13 in \nObjects/listobject.c:946:26: runtime error: call to function gen_iternext through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ngenobject.c:609: note: gen_iternext defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/listobject.c:946:26 in \nPython/generated_cases.c.h:4503:19: runtime error: call to function list_extend through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\nlistobject.c:861: note: list_extend defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4503:19 in \nPython/generated_cases.c.h:4507:13: runtime error: call to function gen_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ngenobject.c:128: note: gen_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4507:13 in \nPython/generated_cases.c.h:2378:17: runtime error: call to function structseq_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nstructseq.c:119: note: structseq_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2378:17 in \nPython/ceval.c:1605:5: runtime error: call to function dict_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ndictobject.c:2373: note: dict_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/ceval.c:1605:5 in \nPython/generated_cases.c.h:1548:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:1548:13 in \nObjects/descrobject.c:439:24: runtime error: call to function _io_FileIO_isatty through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\nfileio.c.h:532: note: _io_FileIO_isatty defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:439:24 in \nObjects/methodobject.c:484:24: runtime error: call to function _io_FileIO_readall through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *)'\nfileio.c.h:284: note: _io_FileIO_readall defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/methodobject.c:484:24 in \nObjects/descrobject.c:372:24: runtime error: call to function _io_FileIO_close through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _typeobject *, struct _object *const *, unsigned long, struct _object *)'\nfileio.c.h:29: note: _io_FileIO_close defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/descrobject.c:372:24 in \nObjects/object.c:783:15: runtime error: call to function bytes_richcompare through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, int)'\nbytesobject.c:1524: note: bytes_richcompare defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/object.c:783:15 in \nPython/generated_cases.c.h:650:13: runtime error: call to function memory_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nmemoryobject.c:1141: note: memory_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:650:13 in \nObjects/abstract.c:804:9: runtime error: call to function memory_releasebuf through pointer to incorrect function type 'void (*)(struct _object *, Py_buffer *)'\nmemoryobject.c:1593: note: memory_releasebuf defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:804:9 in \nObjects/abstract.c:1950:25: runtime error: call to function list_item through pointer to incorrect function type 'struct _object *(*)(struct _object *, long)'\nlistobject.c:460: note: list_item defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/abstract.c:1950:25 in \nModules/timemodule.c:2087:5: runtime error: call to function visit_reachable through pointer to incorrect function type 'int (*)(struct _object *, void *)'\ngcmodule.c:502: note: visit_reachable defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Modules/timemodule.c:2087:5 in \nPython/generated_cases.c.h:2660:21: runtime error: call to function set_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nsetobject.c:489: note: set_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2660:21 in \nPython/generated_cases.c.h:4440:13: runtime error: call to function tupledealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ntupleobject.c:187: note: tupledealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4440:13 in \nPython/generated_cases.c.h:2822:13: runtime error: call to function PyObject_Free through pointer to incorrect function type 'void (*)(struct _object *)'\nobmalloc.c:829: note: PyObject_Free defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2822:13 in \nPython/generated_cases.c.h:2157:13: runtime error: call to function dict_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ndictobject.c:2373: note: dict_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2157:13 in \nObjects/typeobject.c:4872:19: runtime error: call to function getset_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:180: note: getset_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:4872:19 in \nPython/generated_cases.c.h:4730:17: runtime error: call to function func_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nfuncobject.c:913: note: func_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:4730:17 in \nObjects/typeobject.c:4905:15: runtime error: call to function method_get through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *, struct _object *)'\ndescrobject.c:136: note: method_get defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:4905:15 in \nObjects/typeobject.c:1863:16: runtime error: call to function tupletraverse through pointer to incorrect function type 'int (*)(struct _object *, int (*)(struct _object *, void *), void *)'\ntupleobject.c:606: note: tupletraverse defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Objects/typeobject.c:1863:16 in \n[…]\nInclude/internal/pycore_call.h:187:11: runtime error: call to function range_vectorcall through pointer to incorrect function type 'struct _object *(*)(struct _object *, struct _object *const *, unsigned long, struct _object *)'\nrangeobject.c:148: note: range_vectorcall defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Include/internal/pycore_call.h:187:11 in \nPython/generated_cases.c.h:2878:13: runtime error: call to function mappingproxy_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\ndescrobject.c:1160: note: mappingproxy_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2878:13 in \nPython/generated_cases.c.h:3320:25: runtime error: call to function list_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nlistobject.c:347: note: list_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:3320:25 in \nPython/generated_cases.c.h:2766:13: runtime error: call to function set_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nsetobject.c:489: note: set_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:2766:13 in \nModules/_abc.c:52:5: runtime error: call to function visit_reachable through pointer to incorrect function type 'int (*)(struct _object *, void *)'\ngcmodule.c:502: note: visit_reachable defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Modules/_abc.c:52:5 in \nPython/bltinmodule.c:1408:25: runtime error: call to function tupleiter_next through pointer to incorrect function type 'struct _object *(*)(struct _object *)'\ntupleobject.c:999: note: tupleiter_next defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/bltinmodule.c:1408:25 in \nPython/generated_cases.c.h:673:17: runtime error: call to function slice_dealloc through pointer to incorrect function type 'void (*)(struct _object *)'\nsliceobject.c:359: note: slice_dealloc defined here\nSUMMARY: UndefinedBehaviorSanitizer: undefined-behavior Python/generated_cases.c.h:673:17 in \n\u003cbr /\u003e\n(Omitting remaining errors due to GitHub comment length limit.)\n\u003cbr /\u003e\nPython 3.13.0a1+ (heads/patch-103194-dirty:0a6e69f9a2, Oct 21 2023, 14:45:09) [Clang 17.0.3 ] on darwin\nType \"help\", \"copyright\", \"credits\" or \"license\" for more information.\n\u0026gt;\u0026gt;\u0026gt;\n\u003c/pre\u003e\n\u003c/details\u003e\n\nExample workaround for the first error and likely many others, where instead of casting functions to incompatible pointers, the functions use compatible signatures and cast their parameter(s):\n```diff\n--- a/Objects/listobject.c\n+++ b/Objects/listobject.c\n@@ -343,8 +343,9 @@ PyList_Append(PyObject *op, PyObject *newitem)\n /* Methods */\n \n static void\n-list_dealloc(PyListObject *op)\n+list_dealloc(PyObject *self)\n {\n+    PyListObject *op = (PyListObject *)self;\n     Py_ssize_t i;\n     PyObject_GC_UnTrack(op);\n     Py_TRASHCAN_BEGIN(op, list_dealloc)\n@@ -3104,7 +3105,7 @@ PyTypeObject PyList_Type = {\n     \"list\",\n     sizeof(PyListObject),\n     0,\n-    (destructor)list_dealloc,                   /* tp_dealloc */\n+    list_dealloc,                               /* tp_dealloc */\n     0,                                          /* tp_vectorcall_offset */\n     0,                                          /* tp_getattr */\n     0,                                          /* tp_setattr */\n```\n\nIn other cases, it may be less disruptive to introduce a wrapper function with the correct signature:\n```diff\n@@ -615,6 +617,13 @@ _list_clear(PyListObject *a)\n     return 0;\n }\n \n+static int\n+_list_clear_wrap(PyObject *self)\n+{\n+    PyListObject *a = (PyListObject *)self;\n+    return _list_clear(a);\n+}\n+\n /* a[ilow:ihigh] = v if v != NULL.\n  * del a[ilow:ihigh] if v == NULL.\n  *\n@@ -3123,7 +3133,7 @@ PyTypeObject PyList_Type = {\n         _Py_TPFLAGS_MATCH_SELF | Py_TPFLAGS_SEQUENCE,  /* tp_flags */\n     list___init____doc__,                       /* tp_doc */\n     (traverseproc)list_traverse,                /* tp_traverse */\n-    (inquiry)_list_clear,                       /* tp_clear */\n+    _list_clear_wrap,                           /* tp_clear */\n     list_richcompare,                           /* tp_richcompare */\n     0,                                          /* tp_weaklistoffset */\n     list_iter,                                  /* tp_iter */\n```\n\nLikely instances of this can be found at compile time using e.g. `-Wcast-function-type` (although this emits false positives for when the function pointer is cast back to the correct type before called, and this warning is suppressed by intermediate casts through `(void *)`):\n```\nObjects/listobject.c:3162:5: warning: cast from 'void (*)(PyListObject *)' to 'destructor' (aka 'void (*)(struct _object *)') converts to incompatible function type [-Wcast-function-type-strict]\n 3162 |     (destructor)list_dealloc,                   /* tp_dealloc */\n      |     ^~~~~~~~~~~~~~~~~~~~~~~~\n```\n\nI would be interested in combing through and replacing similar instances. But I would not be surprised if sooner or later I encounter an instance that is won’t-fix because it involves a stable API, or if I am told that this problem should be ignored because fixing it is too disruptive or requires disproportionate review effort. I am not aware how immediate any danger is from optimizing compilers exploiting this type of undefined behavior.\n\n### CPython versions tested on:\n\nCPython main branch\n\n### Operating systems tested on:\n\nmacOS\n\n--- \n\nWritten by @picnixz:\n\nSee https://github.com/python/cpython/issues/111178#issuecomment-2642725837 for the remaining files to fix.\n\nFor detecting the UBSan failures, contributors may configure Python with:\n \n```sh\n./configure                                                 \\\n    --with-pydebug                                          \\\n    --prefix=\"$(pwd)/build\"                                 \\\n    CC=clang LD=clang                                       \\\n    CFLAGS=\"-fsanitize=function     -fsanitize-recover\"     \\\n    LDFLAGS=\"-fsanitize=function    -fsanitize-recover\"\n```\n\nThe complete list of failures can be retrieved as follows:\n \n```sh\nPAT='runtime error: call to function (\\w+) through pointer' \u0026\u0026 \\\nmake -j12 2\u003e\u00261 \u003e/dev/null | \\\n    grep -E \"$PAT\" | \\\n    sed -r \"s#^(.+): $PAT.+#\\1@\\2#g\" | \\\n    sort -k1,2 -t@ -u | \\\n    awk 'BEGIN {\n            PROCINFO[\"sorted_in\"]=\"@ind_num_asc\";\n            FS=SUBSEP=\"@\"\n        } {\n            A[$1][length(A[$1])+1]=$2\n        } END {\n            for (m in A) {\n                for (i in A[m]) {\n                    if (i == 1) printf \"%s\\n\", m;\n                    print \"#\", A[m][i]\n                }\n                print \"\"\n            }\n        }' | \\\n    sed -r \"s#\"$(pwd)\"/?(\\./)?(.+)#\\2#g\"\n```\n\nNote that different builds should be configured in order to hunt all UBSan failures (e.g., `--with-trace-refs` or `--disable-gil` to expose conditional compiled code guarded by macros).\n\n---\n\nNote by @encukou:\n\nWe use the macro `_Py_NO_SANITIZE_UNDEFINED` to disable the UB sanitizer in some hard-to-fix cases, so that we can get a stable, regression-monitoring checker sooner. This issue should only be closed after the macro is removed.\n\n---\n\n\u003c!-- gh-linked-prs --\u003e\n### Linked PRs\n* python/cpython#112687\n* python/cpython#112752\n* python/cpython#112782\n* python/cpython#112742\n* python/cpython#112792\n* python/cpython#112793\n* python/cpython#112820\n* python/cpython#112861\n* python/cpython#112863\n* python/cpython#112892\n* python/cpython#112893\n* python/cpython#122972\n* python/cpython#123004\n* gh-124733\n* gh-124763\n* gh-124804\n* gh-124806\n* gh-124888\n* gh-124895\n* gh-124896\n* gh-124900\n* gh-124902\n* gh-124903\n* gh-124908\n* gh-124940\n* gh-124942\n* gh-124943\n* gh-124964\n* gh-124970\n* gh-125043\n* gh-125180\n* gh-125182\n* gh-127982\n* gh-128154\n* gh-128178\n* gh-128235\n* gh-128236\n* gh-128237\n* gh-128238\n* gh-128239\n* gh-128240\n* gh-128241\n* gh-128242\n* gh-128243\n* gh-128244\n* gh-128245\n* gh-128246\n* gh-128247\n* gh-128248\n* gh-128249\n* gh-128250\n* gh-128251\n* gh-128252\n* gh-128253\n* gh-128259\n* gh-128447\n* gh-129060\n* gh-129071\n* gh-129074\n* gh-129083\n* gh-129084\n* gh-129087\n* gh-129088\n* gh-129090\n* gh-129100\n* gh-129101\n* gh-129772\n* gh-129773\n* gh-129774\n* gh-129775\n* gh-129776\n* gh-129777\n* gh-129778\n* gh-129779\n* gh-129780\n* gh-129781\n* gh-129782\n* gh-129783\n* gh-129784\n* gh-129785\n* gh-129786\n* gh-129787\n* gh-129788\n* gh-129789\n* gh-129790\n* gh-129791\n* gh-129792\n* gh-129793\n* gh-129794\n* gh-129795\n* gh-129796\n* gh-129797\n* gh-129798\n* gh-129799\n* gh-129800\n* gh-129801\n* gh-129802\n* gh-130446\n* gh-130575\n* gh-130589\n* gh-130590\n* gh-130591\n* gh-130682\n* gh-130683\n* gh-130684\n* gh-130719\n* gh-131101\n* gh-131102\n* gh-131135\n* gh-131157\n* gh-131159\n* gh-131160\n* gh-131161\n* gh-131162\n* gh-131163\n* gh-131180\n* gh-131191\n* gh-131192\n* gh-131193\n* gh-131227\n* gh-131228\n* gh-131455\n* gh-131456\n* gh-131463\n* gh-131464\n* gh-131496\n* gh-131602\n* gh-131603\n* gh-131605\n* gh-131606\n* gh-131607\n* gh-131608\n* gh-131609\n* gh-131610\n* gh-131611\n* gh-131612\n* gh-131613\n* gh-131614\n* gh-131615\n* gh-131616\n* gh-131659\n* gh-131660\n* gh-131663\n* gh-131664\n* gh-131665\n* gh-131667\n* gh-131668\n* gh-131669\n* gh-131673\n* gh-131674\n* gh-131714\n* gh-131977\n* gh-131979\n* gh-132020\n* gh-132395\n* gh-133072\n* gh-135539\n* gh-135547\n\u003c!-- /gh-linked-prs --\u003e","author":{"url":"https://github.com/chrstphrchvz","@type":"Person","name":"chrstphrchvz"},"datePublished":"2023-10-22T13:30:50.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":43},"url":"https://github.com/111178/cpython/issues/111178"}