René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"[BUG] sparse matrix multiplication causes access violation","articleBody":"Creating a sparse matrix and then multiplying with same or other sparse matrix or with another matrix causes crash and access violation. \r\n\r\nDescription\r\n===========\r\n\r\n* Did you build ArrayFire yourself or did you use the official installers:\r\n       use official installer\r\n* Which backend is experiencing this issue? (CPU, CUDA, OpenCL)\r\n       CUDA\r\n* Do you have a workaround?\r\n       NO\r\n* Can the bug be reproduced reliably on your system?\r\n       YES\r\n* A clear and concise description of what you expected to happen.\r\n* Run your executable with AF_TRACE=all and AF_PRINT_ERRORS=1 environment\r\n  variables set.\r\n* Screenshot or terminal output of the results\r\nsee below\r\n\r\n\r\n\r\nReproducible Code and/or Steps\r\n------------------------------\r\nI am running the following code n GOOGLE tests, However, I am having trouble with sparse matrices in other places as well. Similar nature. \r\n\r\n\r\n\r\n  TEST(TestSuiteName, TestArrayFire)\r\n  {\r\n      float v[]   = {5, 8, 3, 6};\r\n      int r[]     = {0, 0, 2, 3, 4};\r\n      int c[]     = {0, 1, 2, 1};\r\n      const int M = 4, N = 4, nnz = 4;\r\n      array vals    = array(dim4(nnz), v);\r\n      array row_ptr = array(dim4(M + 1), r);\r\n      array col_idx = array(dim4(nnz), c);\r\n   \r\n      af_print(vals);\r\n      af_print(row_ptr);\r\n      af_print(col_idx);\r\n  \r\n      // Create sparse array (CSR) from af::arrays containing values,\r\n      // row pointers, and column indices.\r\n      array sparse = af::sparse(M, N, vals, row_ptr, col_idx, AF_STORAGE_CSR);\r\n      af_print(sparse);\r\n  \r\n      auto res = sparse * sparse;\r\n      af_print(res);\r\n   }\r\n\r\nStackTrace when debugging:\r\n-------------------------------\r\n\r\naf.dll!af_get_last_error(char * * str, __int64 * len) Line 46 (f:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\unified\\error.cpp:46)\r\naf.dll!af::operator*(const af::array \u0026 lhs, const af::array \u0026 rhs) Line 939 (f:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\cpp\\array.cpp:939)\r\n\r\nOutput:\r\n-------------------\r\nLoaded 'C:\\Windows\\System32\\dcomp.dll'. \r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(361) ] Default device: 0\r\n[unified][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\unified\\symbol_manager.cpp(153) ] Device Count: 4.\r\n[unified][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\unified\\symbol_manager.cpp(143) ] Attempting: Default System Paths\r\nLoaded 'C:\\Programs\\ArratFire\\v3\\lib\\afcuda.dll'. Symbols loaded.\r\nLoaded 'C:\\Windows\\System32\\nvcuda.dll'. \r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\cublas64_12.dll'. Module was built without symbols.\r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\cufft64_11.dll'. Module was built without symbols.\r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\cusolver64_11.dll'. Module was built without symbols.\r\n2\r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\cusparse64_12.dll'. Module was built without symbols.\r\nUnloaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\cusparse64_12.dll'.\r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\nvrtc64_120_0.dll'. Module was built without symbols.\r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\cublasLt64_12.dll'. Module was built without symbols.\r\nLoaded 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.2\\bin\\nvJitLink_120_0.dll'. Module was built without symbols.\r\n[unified][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\unified\\symbol_manager.cpp(146) ] Found: afcuda.dll\r\nLoaded 'C:\\Windows\\System32\\DriverStore\\FileRepository\\nvdmi.inf_amd64_893ed8ff453738db\\nvcuda64.dll'. \r\n[unified][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\unified\\symbol_manager.cpp(153) ] Device Count: 1.\r\n[unified][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\api\\unified\\symbol_manager.cpp(208) ] AF_DEFAULT_BACKEND: cuda\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\common\\DependencyModule.cpp(101) ] Attempting to load: forge.dll\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\common\\DependencyModule.cpp(104) ] Found: forge.dll\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\device_manager.cpp(494) ] CUDA Driver supports up to CUDA 12.2.0 ArrayFire CUDA Runtime 12.0.0\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\device_manager.cpp(479) ] CUDA driver version(12.2.0) not part of the CudaToDriverVersion array. Please create an issue or a pull request on the ArrayFire repository to update the CudaToDriverVersion variable with this version of the CUDA runtime.\r\n\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\device_manager.cpp(562) ] Found 1 CUDA devices\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\device_manager.cpp(590) ] Found device: NVIDIA GeForce GTX 1050 Ti with Max-Q Design (sm_61) (4 GB | ~2076.416015625 GFLOPs | 6 SMs)\r\n[platform][1688279120][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\device_manager.cpp(625) ] AF_CUDA_DEFAULT_DEVICE: \r\n[platform][1688279121][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\device_manager.cpp(644) ] Default device: 0(NVIDIA GeForce GTX 1050 Ti with Max-Q Design)\r\n[mem][1688279121][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\common\\DefaultMemoryManager.cpp(128) ] memory[0].max_bytes: 3 GB\r\n[mem][1688279121][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\memory.cpp(157) ] nativeAlloc:    1 KB 0x1001000000\r\n[mem][1688279121][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\memory.cpp(157) ] nativeAlloc:    1 KB 0x1001000400\r\n[mem][1688279121][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\memory.cpp(157) ] nativeAlloc:    1 KB 0x1001000800\r\nals\r\n[4 1 1 1]\r\n[mem][1688279137][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\memory.cpp(157) ] nativeAlloc:    1 KB 0x1001000c00\r\n[jit][1688279137][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\compile_module.cpp(466) ] {291210446400920389   : loaded from C:\\Users\\rohrb\\AppData\\Local\\Temp\\\\ArrayFire\\KER291210446400920389_CU_61_AF_38.bin for NVIDIA GeForce GTX 1050 Ti with Max-Q Design }\r\n[kernel][1688279137][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\Kernel.hpp(42) ] Launching arrayfire::cuda::transpose\u003cfloat,false,false\u003e: Blocks: [1, 1, 1] Threads: [32, 8, 1] Shared Memory: 0\r\n    5.0000 \r\n    8.0000 \r\n    3.0000 \r\n    6.0000 \r\n\r\nrow_ptr\r\n[5 1 1 1]\r\n[jit][1688279139][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\compile_module.cpp(466) ] {9963207849209086996  : loaded from C:\\Users\\rohrb\\AppData\\Local\\Temp\\\\ArrayFire\\KER9963207849209086996_CU_61_AF_38.bin for NVIDIA GeForce GTX 1050 Ti with Max-Q Design }\r\n[kernel][1688279139][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\Kernel.hpp(42) ] Launching arrayfire::cuda::transpose\u003cint,false,false\u003e: Blocks: [1, 1, 1] Threads: [32, 8, 1] Shared Memory: 0\r\n2\r\n         0 \r\n         2 \r\n         3 \r\n         4 \r\n\r\ncol_idx\r\n[4 1 1 1]\r\n[kernel][1688279147][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\Kernel.hpp(42) ] Launching arrayfire::cuda::transpose\u003cint,false,false\u003e: Blocks: [1, 1, 1] Threads: [32, 8, 1] Shared Memory: 0\r\n         0 \r\n         1 \r\n         2 \r\n         1 \r\n\r\nsparse\r\nStorage Format : AF_STORAGE_CSR\r\n[4 4 1 1]\r\nsparse: Values\r\n[4 1 1 1]\r\n[kernel][1688279152][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\Kernel.hpp(42) ] Launching arrayfire::cuda::transpose\u003cfloat,false,false\u003e: Blocks: [1, 1, 1] Threads: [32, 8, 1] Shared Memory: 0\r\n    5.0000 \r\n    8.0000 \r\n    3.0000 \r\n    6.0000 \r\n\r\n\r\nsparse: RowIdx\r\n[5 1 1 1]\r\n[kernel][1688279152][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\Kernel.hpp(42) ] Launching arrayfire::cuda::transpose\u003cint,false,false\u003e: Blocks: [1, 1, 1] Threads: [32, 8, 1] Shared Memory: 0\r\n2\r\n         0 \r\n         2 \r\n         3 \r\n         4 \r\n\r\nsparse: ColIdx\r\n[4 1 1 1]\r\n[kernel][1688279152][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\cuda\\Kernel.hpp(42) ] Launching arrayfire::cuda::transpose\u003cint,false,false\u003e: Blocks: [1, 1, 1] Threads: [32, 8, 1] Shared Memory: 0\r\n         0 \r\n         1 \r\n         2 \r\n         1 \r\n\r\nException thrown at 0x00007FFF04ADADCC (af.dll) in runTests.exe: 0xC0000005: Access violation reading location 0xFFFFFFFFFFFFFFFF.\r\nunknown file: error: SEH exception with code 0xc0000005 thrown in the test body.\r\n\r\nunknown file: error: SEH exception with code 0xc0000005 thrown in the test body.\r\n\r\nSystem Information\r\n------------------\r\n\r\nPlease provide the following information:\r\n1. ArrayFire version\r\nArrayFire v3.8.3 \r\n\r\n2. Devices installed on the system\r\n2 Graphics Cards 1 CPU\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\common\\DependencyModule.cpp(104) ] Found: forge.dll\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(217) ] Found 5 OpenCL platforms\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(229) ] Found 1 devices on platform NVIDIA CUDA\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(234) ] Found device NVIDIA GeForce GTX 1050 Ti with Max-Q Design on platform NVIDIA CUDA\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(229) ] Found 1 devices on platform Intel(R) OpenCL HD Graphics\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(234) ] Found device Intel(R) UHD Graphics 630 on platform Intel(R) OpenCL HD Graphics\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(229) ] Found 1 devices on platform Intel(R) OpenCL\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(234) ] Found device Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz on platform Intel(R) OpenCL\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(229) ] Found 1 devices on platform Intel(R) FPGA Emulation Platform for OpenCL(TM)\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(234) ] Found device Intel(R) FPGA Emulation Device on platform Intel(R) FPGA Emulation Platform for OpenCL(TM)\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(229) ] Found 0 devices on platform Intel(R) FPGA SDK for OpenCL(TM)\r\n[platform][1688279118][13192] [ F:\\buildbot\\worker\\win10-cuda-installer\\build\\src\\backend\\opencl\\device_manager.cpp(239) ] Found 4 OpenCL devices\r\n\r\n4. (optional) Output from the af::info() function if applicable.\r\n2023-07-01 22:53:14.745 info : Setting CUDA Backend\r\n2023-07-01 22:53:14.745 info : Testing Backend: \r\n2023-07-01 22:53:14.745 info : ArrayFire v3.8.3 (CUDA, 64-bit Windows, build 987d5675a)\r\nPlatform: CUDA Runtime 12.0, Driver: 12020\r\n[0] NVIDIA GeForce GTX 1050 Ti with Max-Q Design, 4096 MB, CUDA Compute 6.1\r\n\r\n5. Output from the following scripts:\r\n\r\nRun one of the following commands based on your OS\r\n\r\n\r\nWindows:\r\nDownload clinfo from https://github.com/Oblomov/clinfo\r\n\r\nIf you have NVIDIA GPUs. Run nvidia-smi usually located in\r\nC:\\Program Files\\NVIDIA Corporation\\NVSMI\r\n\r\nProvide driver version for your GPU. (This is vendor specific)\r\n536.40\r\n\r\n\r\nChecklist\r\n---------\r\n\r\n- [x] Using the latest available ArrayFire release\r\n- [x] GPU drivers are up to date\r\n","author":{"url":"https://github.com/rohrbachd","@type":"Person","name":"rohrbachd"},"datePublished":"2023-07-02T06:41:12.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":5},"url":"https://github.com/3460/arrayfire/issues/3460"}