René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"Cannot use CUDA Python to interact with external C/C++ code","articleBody":"# The problem\r\n\r\nThe Cython example below demonstrates an attempt to use CUDA Python to interact with some external C++ code. Note that the \"external code\" is included inline in the Cython.\r\n\r\n```cython\r\n# distutils: language=c++\r\n# distutils: extra_compile_args=-I/usr/local/cuda/include/\r\n\r\nfrom cuda.ccudart cimport cudaMemAllocationHandleType\r\n\r\ncdef extern from *:\r\n    \"\"\"\r\n    #include \u003ccuda_runtime_api.h>\r\n    \r\n    void foo(cudaMemAllocationHandleType x) {\r\n        return;\r\n    }\r\n    \"\"\"\r\n    void foo(cudaMemAllocationHandleType x)\r\n\r\nfoo(cudaMemAllocationHandleType.cudaMemHandleTypeNone)\r\n```\r\n\r\nThe external code is a function `foo` that accepts a `cudaMemAllocationHandleType`. We attempt to invoke that function from Cython by passing  in a  `cuda.ccudart.cudaMemAlloccationHandleType`, but this fails with an error like:\r\n\r\n```cython\r\nerror: cannot convert '__pyx_t_4cuda_7ccudart_cudaMemAllocationHandleType' to 'cudaMemAllocationHandleType'\r\n 4857 |   foo(__pyx_e_4cuda_7ccudart_cudaMemHandleTypeNone);\r\n```\r\n\r\nTo reproduce the problem, save the example above to a flle `foo.pyx`, then run  `cythonize -i foo.pyx`.\r\n\r\n# Why this happens\r\n\r\nThis is because the function `foo` expects a `cudaMemAllocationHandleType` that is defined in the CUDA runtime library. But CUDA Python \"rewrites\" the runtime library at the Cython layer, and has its _own_ `cudaMemAllocationHandleType` (which ends up with a mangled name when transpiled from Cython to C++). The two are not interchangeable.\r\n\r\n# A potential solution\r\n\r\nA potential solution, proposed by @leofang in an offline discussion, is to use extern declarations for types in `ccudart.pxd`, rather than to redefine them. For example:\r\n\r\n```diff\r\ndiff --git a/cuda/ccudart.pxd b/cuda/ccudart.pxd\r\nindex 57e1e96..6c0b5d4 100644\r\n--- a/cuda/ccudart.pxd\r\n+++ b/cuda/ccudart.pxd\r\n@@ -678,11 +678,12 @@ cdef enum cudaMemAllocationType:\r\n     cudaMemAllocationTypePinned = 1\r\n     cudaMemAllocationTypeMax = 2147483647\r\n \r\n-cdef enum cudaMemAllocationHandleType:\r\n-    cudaMemHandleTypeNone = 0\r\n-    cudaMemHandleTypePosixFileDescriptor = 1\r\n-    cudaMemHandleTypeWin32 = 2\r\n-    cudaMemHandleTypeWin32Kmt = 4\r\n+cdef extern from 'driver_types.h':\r\n+    ctypedef enum cudaMemAllocationHandleType 'cudaMemAllocationHandleType':\r\n+        cudaMemHandleTypeNone = 0\r\n+        cudaMemHandleTypePosixFileDescriptor = 1\r\n+        cudaMemHandleTypeWin32 = 2\r\n+        cudaMemHandleTypeWin32Kmt = 4\r\n \r\n cdef struct cudaMemPoolProps:\r\n     cudaMemAllocationType allocType\r\ndiff --git a/setup.py b/setup.py\r\nindex 394166e..16fad9f 100644\r\n--- a/setup.py\r\n+++ b/setup.py\r\n@@ -30,6 +30,7 @@ except Exception:\r\n \r\n include_dirs = [\r\n     os.path.dirname(sysconfig.get_path(\"include\")),\r\n+    '/usr/local/cuda-11.4/include',\r\n ]\r\n \r\n library_dirs = [get_python_lib(), os.path.join(os.sys.prefix, \"lib\")]\r\n```\r\n\r\n## Gotcha\r\n\r\nCurrently, we ship a single version of CUDA Python that is built with the latest CUDA toolkit, and we expect it to work for older minor versions of  the CUDA toolkit by leveraging CUDA enhanced compatibility.\r\n\r\nHistorically, there have been cases when the runtime API has changed across minor versions of the CUDA toolkit. In particular, the names/ordering of enum members have changed between minor versions. For example, in CUDA 10.1, there was a typo in the enum member `cudaErrorDeviceUninitilialized` that was fixed in 10.2.\r\n\r\nIt's not clear how we would handle the situation if something like that were to happen again. In the example above, we would have to have separate extern declarations for 10.1 and 10.2 somehow.","author":{"url":"https://github.com/shwina","@type":"Person","name":"shwina"},"datePublished":"2022-05-11T19:10:53Z","url":"https://github.com/NVIDIA/cuda-python/issues/22"}