René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[BUG]: CUDA API calls through `cuda-bindings` 3x slower than direct CUDA C++ API calls","articleBody":"### Is this a duplicate?\n\n- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cuda-python/issues) for this bug and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)\n\n### Type of Bug\n\nPerformance\n\n### Component\n\ncuda.bindings\n\n### Describe the bug\n\nI have a same CUDA kernel, and I am calling it using two different methods.\n\n1. Compile the kernel to `.cubin`, load it in Python and call it using `cuda-bindings`. The TMA descriptors are also created via `cuda-bindings`.\n2. Compile the kernel and relevant C++ host functions (including TMA desc, kernel attribute and launch config) to `.so`, load the `.so` in Python and call the C++ host function.\n\nAs revealed in the two timelines below, the first path is 3x slower than the second path. \n\u003cimg width=\"582\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/b6962032-34ce-4180-aed7-09baeabffac2\" />\n\u003cimg width=\"803\" alt=\"Image\" src=\"https://github.com/user-attachments/assets/597bc426-2ccd-47a2-9a04-b75c66133dc6\" />\n\n### How to Reproduce\n\nFollow the instructions in the reproduction repo https://github.com/lucifer1004/cuda-python-repro\n\n### Expected behavior\n\nCUDA Python's perf should be on par with CUDA C++.\n\n### Operating System\n\nUbuntu Linux 24.04\n\n### nvidia-smi output\n\n+-----------------------------------------------------------------------------------------+\n| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.9     |\n|-----------------------------------------+------------------------+----------------------+\n| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |\n| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |\n|                                         |                        |               MIG M. |\n|=========================================+========================+======================|\n|   0  NVIDIA H100 80GB HBM3          On  |   00000000:53:00.0 Off |                    0 |\n| N/A   27C    P0             69W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   1  NVIDIA H100 80GB HBM3          On  |   00000000:64:00.0 Off |                    0 |\n| N/A   28C    P0             68W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   2  NVIDIA H100 80GB HBM3          On  |   00000000:75:00.0 Off |                    0 |\n| N/A   27C    P0             69W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   3  NVIDIA H100 80GB HBM3          On  |   00000000:86:00.0 Off |                    0 |\n| N/A   29C    P0             68W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   4  NVIDIA H100 80GB HBM3          On  |   00000000:97:00.0 Off |                    0 |\n| N/A   30C    P0             69W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   5  NVIDIA H100 80GB HBM3          On  |   00000000:A8:00.0 Off |                    0 |\n| N/A   28C    P0             68W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   6  NVIDIA H100 80GB HBM3          On  |   00000000:B9:00.0 Off |                    0 |\n| N/A   28C    P0             69W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n|   7  NVIDIA H100 80GB HBM3          On  |   00000000:CA:00.0 Off |                    0 |\n| N/A   26C    P0             66W /  700W |       0MiB /  81559MiB |      0%      Default |\n|                                         |                        |             Disabled |\n+-----------------------------------------+------------------------+----------------------+\n                                                                                         \n+-----------------------------------------------------------------------------------------+\n| Processes:                                                                              |\n|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |\n|        ID   ID                                                               Usage      |\n|=========================================================================================|\n|  No running processes found                                                             |\n+-----------------------------------------------------------------------------------------+","author":{"url":"https://github.com/lucifer1004","@type":"Person","name":"lucifer1004"},"datePublished":"2025-05-27T05:44:26Z","url":"https://github.com/NVIDIA/cuda-python/issues/659"}