René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"About Efficiency Sparse Dense Multiplication","articleBody":"Hi, thank you so much for a great project. May I ask a small question related to sparse and dense matrix operation? The case is that I want to conduct a multiplication between sparse and dense matrix. In detail, the sparsity of the 2 project is given as below:\r\n```\r\nX shape: (200000, 50000) \r\nC shape: (50000, 100)\r\n#nnz X: 99500704\r\n#nnz C: 3160328\r\nSparsity of X:  0.0099500704\r\nSparsity of C:  0.6320656\r\n```\r\nWhere X is a sparse matrix and C is a dense one. In theory, C should be stored in dense format in order to make the operation to be efficient instead of sparse one. However, using bitmap/dense matrix provided by GraphBLAS is not as efficient as compared to MKL code for sparse * dense operation\r\n```\r\n##################### python-graphblas (Default Setting - 32 Threads - CSR Format * BitmapC Format) #####################\r\nConvert Scipy Sparse Format -\u003e python-graphblas Format (s) 0.6137485504150391\r\nSparse * Sparse computing time (s) 2.929335117340088\r\nSparse * Sparse computing time (s) 2.9536073207855225\r\nSparse * Sparse computing time (s) 2.9327011108398438\r\nSparse * Sparse computing time (s) 2.946722984313965\r\nSparse * Sparse computing time (s) 2.942711114883423\r\nSparse * Sparse computing time (s) 2.959540843963623\r\nSparse * Sparse computing time (s) 2.9557604789733887\r\nSparse * Sparse computing time (s) 2.946617364883423\r\nSparse * Sparse computing time (s) 2.9554543495178223\r\nSparse * Sparse computing time (s) 2.9580225944519043\r\nSparse * Sparse computing time (s) 2.9371848106384277\r\nSparse * Sparse computing time (s) 2.9504570960998535\r\nSparse * Sparse computing time (s) 2.956547498703003\r\nSparse * Sparse computing time (s) 2.9532806873321533\r\nSparse * Sparse computing time (s) 2.947199821472168\r\nSparse * Sparse computing time (s) 2.952831745147705\r\nSparse * Sparse computing time (s) 2.950216293334961\r\nSparse * Sparse computing time (s) 2.950721025466919\r\nSparse * Sparse computing time (s) 2.958610773086548\r\nSparse * Sparse computing time (s) 2.952730894088745\r\nMean Runtime:  2.9495126962661744\r\nStd Runtime:  0.008198427827772714\r\n\r\n##################### python-graphblas (Default Setting - 32 Threads - CSR Format * FullC Format) #####################\r\nConvert Scipy Sparse Format -\u003e python-graphblas Format (s) 0.5988132953643799\r\nSparse * Sparse computing time (s) 1.73736572265625\r\nSparse * Sparse computing time (s) 1.744694709777832\r\nSparse * Sparse computing time (s) 1.7468664646148682\r\nSparse * Sparse computing time (s) 1.7360756397247314\r\nSparse * Sparse computing time (s) 1.7450006008148193\r\nSparse * Sparse computing time (s) 1.7598528861999512\r\nSparse * Sparse computing time (s) 1.7438948154449463\r\nSparse * Sparse computing time (s) 1.7407243251800537\r\nSparse * Sparse computing time (s) 1.7503554821014404\r\nSparse * Sparse computing time (s) 1.740696907043457\r\nSparse * Sparse computing time (s) 1.757706642150879\r\nSparse * Sparse computing time (s) 1.7401001453399658\r\nSparse * Sparse computing time (s) 1.7460541725158691\r\nSparse * Sparse computing time (s) 1.7468533515930176\r\nSparse * Sparse computing time (s) 1.751969337463379\r\nSparse * Sparse computing time (s) 1.7436254024505615\r\nSparse * Sparse computing time (s) 1.7455754280090332\r\nSparse * Sparse computing time (s) 1.7475886344909668\r\nSparse * Sparse computing time (s) 1.7609961032867432\r\nSparse * Sparse computing time (s) 1.7456836700439453\r\nMean Runtime:  1.7465840220451354\r\nStd Runtime:  0.006642586857441714\r\n```\r\n\r\n\r\n```\r\n##################### MKL (Default Settings - 16 Threads - Sparse CSR Format * Dense Numpy Format) #####################\r\nSparse * Sparse computing time (s) 0.6445832252502441\r\nSparse * Sparse computing time (s) 0.6223111152648926\r\nSparse * Sparse computing time (s) 0.6190366744995117\r\nSparse * Sparse computing time (s) 0.6191632747650146\r\nSparse * Sparse computing time (s) 0.617084264755249\r\nSparse * Sparse computing time (s) 0.6194536685943604\r\nSparse * Sparse computing time (s) 0.6174299716949463\r\nSparse * Sparse computing time (s) 0.6181790828704834\r\nSparse * Sparse computing time (s) 0.6196620464324951\r\nSparse * Sparse computing time (s) 0.6216928958892822\r\nSparse * Sparse computing time (s) 0.6174044609069824\r\nSparse * Sparse computing time (s) 0.6212775707244873\r\nSparse * Sparse computing time (s) 0.6180646419525146\r\nSparse * Sparse computing time (s) 0.6177294254302979\r\nSparse * Sparse computing time (s) 0.6187028884887695\r\nSparse * Sparse computing time (s) 0.6197123527526855\r\nSparse * Sparse computing time (s) 0.6219933032989502\r\nSparse * Sparse computing time (s) 0.6392252445220947\r\nSparse * Sparse computing time (s) 0.6407394409179688\r\nSparse * Sparse computing time (s) 0.6248526573181152\r\n\r\nMean Runtime:  0.6229149103164673\r\nStd Runtime:  0.008091237575214866\r\n```\r\n\r\nCould you please kindly suggest any format from GraphBLAS to solve this problem? Thank you so much in advance!","author":{"url":"https://github.com/khoinpd0411","@type":"Person","name":"khoinpd0411"},"datePublished":"2024-11-24T21:11:25.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":4},"url":"https://github.com/550/python-graphblas/issues/550"}