Title: GitHub - microsoft/MInference: [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
Open Graph Title: GitHub - microsoft/MInference: [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
X Title: GitHub - microsoft/MInference: [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy.
Description: [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 10x for pre-filling on an A100 while maintaining accuracy. - microsoft/MInference
Open Graph Description: [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 1...
X Description: [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference la...
Mail addresses
opencode@microsoft.com
Opengraph URL: https://github.com/microsoft/MInference
X: @github
Domain: patch-diff.githubusercontent.com
| route-pattern | /:user_id/:repository |
| route-controller | files |
| route-action | disambiguate |
| fetch-nonce | v2:a4091922-73b5-9c9d-bd8c-cf9b6a666ee2 |
| current-catalog-service-hash | f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb |
| request-id | ACBC:2FF234:5BCA94:7E962D:698318EB |
| html-safe-nonce | 358fb13ef43c991ce6d74fa6ad26726c7b6c3e66e458f42a02b6640842ad6286 |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBQ0JDOjJGRjIzNDo1QkNBOTQ6N0U5NjJEOjY5ODMxOEVCIiwidmlzaXRvcl9pZCI6IjMwOTc1ODAxNTEzNjY2ODY5NTUiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ== |
| visitor-hmac | a738f50012fbe45e600b8fd4d9c372da6cac9e7be685cbf5d2269c2d025e8208 |
| hovercard-subject-tag | repository:804362023 |
| github-keyboard-shortcuts | repository,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | / |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/microsoft/MInference |
| twitter:image | https://opengraph.githubassets.com/979348612ba07278df5fc3f9586ae9180258e6ae2556e16ceda569680450d2df/microsoft/MInference |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/979348612ba07278df5fc3f9586ae9180258e6ae2556e16ceda569680450d2df/microsoft/MInference |
| og:image:alt | [NeurIPS'24 Spotlight, ICLR'25, ICML'25] To speed up Long-context LLMs' inference, approximate and dynamic sparse calculate the attention, which reduces inference latency by up to 1... |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | c6741528aa4d4ba81c57bbff8f8cc0de0e9115cb2993431dc9dac8d489f7b4ee |
| turbo-cache-control | no-preview |
| go-import | github.com/microsoft/MInference git https://github.com/microsoft/MInference.git |
| octolytics-dimension-user_id | 6154722 |
| octolytics-dimension-user_login | microsoft |
| octolytics-dimension-repository_id | 804362023 |
| octolytics-dimension-repository_nwo | microsoft/MInference |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | false |
| octolytics-dimension-repository_network_root_id | 804362023 |
| octolytics-dimension-repository_network_root_nwo | microsoft/MInference |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | d47248b3090603bdc219cc44ef088eef8eed0928 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
Links:
Viewport: width=device-width
