René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[FEA]: Faster initialization time for `cuda.core` abstractions","articleBody":"### Is this a duplicate?\n\n- [x] I confirmed there appear to be no [duplicate issues](https://github.com/NVIDIA/cuda-python/issues) for this request and that I agree to the [Code of Conduct](CODE_OF_CONDUCT.md)\n\n### Area\n\ncuda.core\n\n### Is your feature request related to a problem? Please describe.\n\nAs mentioned [in a previous issue](https://github.com/NVIDIA/cuda-python/issues/439), equivalent operations using CuPy can be significantly faster. In this issue, I am requesting that the initialization of `cuda.core` abstractions have less overhead. Specifically, when compared to their CuPy counterparts, the initialization of Device, Stream, and Event abstractions are slower.\n\n```\n>>> timeit.timeit('cp.cuda.Device()', setup='import cupy as cp')\n0.06881106700166129\n>>> timeit.timeit('device = ccx.Device()', setup='import cuda.core.experimental as ccx')\n0.5686513699911302\n```\n\n```\n>>> timeit.timeit('cp.cuda.Stream()', setup='import cupy as cp')\n1.0035127629962517\n>>> timeit.timeit('device.create_stream()', setup='import cuda.core.experimental as ccx; device = ccx.Device(); device.set_current()')\n5.299269804003416\n```\n\n```\n>>> timeit.timeit('cp.cuda.Event()', setup='import cupy as cp')\n0.393913417996373\n>>> timeit.timeit('device.create_event()', setup='import cuda.core.experimental as ccx; device = ccx.Device(); device.set_current()')\n3.1525100879953243\n```\n\nThis has caused noticeable performance regressions in `nvmath-python` when transitioning from `cupy.cuda` to `cuda.core` for our benchmarks for smaller/medium size arrays or on faster devices where python overhead is more significant.\n\nSpecifically, we currently use event recording frequently in order to autoselect algorithms/plans and to wait for computation to complete before returning to the user (for host APIs), and we frequently use the Device constructor to check the current device.\n\n### Describe the solution you'd like\n\nThese init functions should just as fast as or faster than CuPy's abstractions.\n\n### Describe alternatives you've considered\n\n- Refactoring our internal implementation to call `Device()` less often by passing around one `Device` and being careful about context switching.\n- Using less event recording? Trying to reuse the same two `Event`s. I don't think this is feasible.\n\n### Additional context\n\nI doesn't seem like the best long term solution for `nvmath-python` to try to work around these issues. ","author":{"url":"https://github.com/carterbox","@type":"Person","name":"carterbox"},"datePublished":"2025-05-23T23:34:17Z","url":"https://github.com/NVIDIA/cuda-python/issues/658"}