Title: [Dev] Potentially improve performance through block reduction by LeiWang1999 · Pull Request #63 · microsoft/BitBLAS · GitHub
Open Graph Title: [Dev] Potentially improve performance through block reduction by LeiWang1999 · Pull Request #63 · microsoft/BitBLAS
X Title: [Dev] Potentially improve performance through block reduction by LeiWang1999 · Pull Request #63 · microsoft/BitBLAS
Description: BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment. - [Dev] Potentially improve performance through block reduction by LeiWang1999 · Pull Request #63 · microsoft/BitBLAS
Open Graph Description: In our recent evaluations, we observed that batch inference on matrix multiplication shapes for 7b/13b models didn't achieve the expected theoretical speedup. This performance bottleneck appear...
X Description: In our recent evaluations, we observed that batch inference on matrix multiplication shapes for 7b/13b models didn't achieve the expected theoretical speedup. This performance bottleneck ap...
Opengraph URL: https://github.com/microsoft/BitBLAS/pull/63
X: @github
Domain: github.com
| route-pattern | /_view_fragments/voltron/pull_requests/show/:user_id/:repository/:id/pull_request_layout(.:format) |
| route-controller | voltron_pull_requests_fragments |
| route-action | pull_request_layout |
| fetch-nonce | v2:b1f5a6fa-f75d-8a06-ef80-bd60910ade22 |
| current-catalog-service-hash | ae870bc5e265a340912cde392f23dad3671a0a881730ffdadd82f2f57d81641b |
| request-id | D0C8:225311:D8AA61:12695E1:6A4C6040 |
| html-safe-nonce | f3291cc64382f940ae3d89038817501af5e2bba5d5d7f626f3f89b26a9fdac9c |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJEMEM4OjIyNTMxMTpEOEFBNjE6MTI2OTVFMTo2QTRDNjA0MCIsInZpc2l0b3JfaWQiOiI4MTE5MjI5NDIyNTA0Nzk2MjI0IiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 0768779571cdb4ae5be1a4b14d01b3e24fcf4bf757d0cdd1c54cc5db7d16681d |
| hovercard-subject-tag | pull_request:1945905558 |
| github-keyboard-shortcuts | repository,pull-request-list,pull-request-conversation,pull-request-files-changed,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/_view_fragments/voltron/pull_requests/show/microsoft/BitBLAS/63/pull_request_layout |
| twitter:image | https://opengraph.githubassets.com/388886890c931eb94df38d6aefd87194de6e98cdb5c36a420830d15b2b352549/microsoft/BitBLAS/pull/63 |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/388886890c931eb94df38d6aefd87194de6e98cdb5c36a420830d15b2b352549/microsoft/BitBLAS/pull/63 |
| og:image:alt | In our recent evaluations, we observed that batch inference on matrix multiplication shapes for 7b/13b models didn't achieve the expected theoretical speedup. This performance bottleneck appear... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| og:author:username | LeiWang1999 |
| hostname | github.com |
| expected-hostname | github.com |
| None | 3d11bb817438277de2a940854450e83a7d32b6aeb5014e9e6b00a6423900251c |
| turbo-cache-control | no-cache |
| go-import | github.com/microsoft/BitBLAS git https://github.com/microsoft/BitBLAS.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 762181983 |
| octolytics-dimension-repository_nwo | microsoft/BitBLAS |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 762181983 |
| octolytics-dimension-repository_network_root_nwo | microsoft/BitBLAS |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 14099438da5379150f15a2892474c7c7e6c0e55e |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width