| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository |
| route-controller | files |
| route-action | disambiguate |
| fetch-nonce | v2:20dce624-4b65-cb6a-b06b-fe0b72e6a88b |
| current-catalog-service-hash | f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb |
| request-id | BDB6:1A1BB6:D58FD6:1219F54:6AFE77BC |
| html-safe-nonce | c23c9c2f0dc8ae5018f735d859521f8ac41215c0d345da9315566c4eabc8a89a |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJCREI2OjFBMUJCNjpENThGRDY6MTIxOUY1NDo2QUZFNzdCQyIsInZpc2l0b3JfaWQiOiI3MDg3NzYzOTUzNzQ4NDQxMDIwIiwicmVnaW9uX2VkZ2UiOiJpYWQiLCJyZWdpb25fcmVuZGVyIjoiaWFkIn0= |
| visitor-hmac | 7fd8e19640c66a97fc4f6fa78dc30925b874c1138ab250f70aca75ce34f318d7 |
| hovercard-subject-tag | repository:874566161 |
| github-keyboard-shortcuts | repository,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | // |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/vonchenplus/llama.cpp |
| twitter:image | https://opengraph.githubassets.com/5a63d7964f27617d140d9a1f27520435e62aeec365aae235ba9c52f2d25c8272/vonchenplus/llama.cpp |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/5a63d7964f27617d140d9a1f27520435e62aeec365aae235ba9c52f2d25c8272/vonchenplus/llama.cpp |
| og:image:alt | LLM inference in C/C++. Contribute to vonchenplus/llama.cpp development by creating an account on GitHub. |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | b3c0f155eafc27e60db6e254d1ffff0e072e9ab3b4018a5ee518797c19d89f86 |
| turbo-cache-control | no-cache |
| go-import | github.com/vonchenplus/llama.cpp git https://github.com/vonchenplus/llama.cpp.git |
| octolytics-dimension-user_id | 3349963 |
| octolytics-dimension-user_login | vonchenplus |
| octolytics-dimension-repository_id | 874566161 |
| octolytics-dimension-repository_nwo | vonchenplus/llama.cpp |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | true |
| octolytics-dimension-repository_parent_id | 612354784 |
| octolytics-dimension-repository_parent_nwo | ggml-org/llama.cpp |
| octolytics-dimension-repository_network_root_id | 612354784 |
| octolytics-dimension-repository_network_root_nwo | ggml-org/llama.cpp |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 4a10310af51a1bfffbedc4fd81cb582acf0cb0ff |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| Skip to content | https://redirect.github.com/vonchenplus/llama.cpp#start-of-content |
| https://redirect.github.com/ |
| Sign in | https://redirect.github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fvonchenplus%2Fllama.cpp |
| GitHubRepository hosting, code review, and collaboration | https://github.com/platform |
| CopilotAI coding agent for tasks, projects, and automated work | https://github.com/features/copilot |
| ActionsCI/CD and automation | https://github.com/features/actions |
| Code QualityQuality analysis, fixes, and enforcement | https://github.com/features/code-quality |
| CodespacesCloud development environments | https://github.com/features/codespaces |
| Code SecurityVulnerability detection and remediation | https://github.com/security/advanced-security/code-security |
| Secret ProtectionSecret scanning and leak prevention | https://github.com/security/advanced-security/secret-protection |
| All Features | https://github.com/features |
| Documentation | https://docs.github.com/en |
| Training | https://learn.github.com/skills |
| Certifications | https://learn.github.com/credentials |
| Build with Copilot | https://github.com/features/ai |
| Customize Copilot | https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview |
| Integrations | https://github.com/marketplace |
| Trending repos | https://github.com/trending |
| GitHub Sponsors | https://github.com/open-source/sponsors |
| Developer articles | https://github.com/resources/articles |
| ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place. | https://github.blog/changelog/ |
| Enterprises | https://github.com/enterprise |
| Team | https://github.com/team |
| Startups | https://github.com/enterprise/startups |
| Nonprofits | https://github.com/solutions/industry/nonprofits |
| Healthcare | https://github.com/solutions/industry/healthcare |
| Financial services | https://github.com/solutions/industry/financial-services |
| Manufacturing | https://github.com/solutions/industry/manufacturing |
| Government | https://github.com/solutions/industry/government |
| See all industries | https://github.com/solutions/industry |
| Why GitHub | https://github.com/why-github |
| Customer Stories | https://github.com/customer-stories |
| Business insights | https://github.com/solutions/executive-insights |
| Trust Center | https://github.com/trust-center |
| GitHub Advanced SecurityEnterprise-grade code security | https://github.com/security/advanced-security |
| Copilot for businessEnterprise-grade AI features | https://github.com/features/copilot/copilot-business |
| Premium SupportEnterprise-grade 24/7 support | https://github.com/enterprise/premium-support |
| Documentation | https://docs.github.com/en |
| Resource center | https://github.com/resources |
| Blog | https://github.blog/ |
| Events & webinars | https://github.com/resources/events |
| Community Forum | https://github.com/orgs/community/discussions |
| Customer Stories | https://github.com/customer-stories |
| Integrations | https://github.com/marketplace |
| Customer support | https://help.github.com/ |
| Training | https://learn.github.com/skills |
| GitHub Status | https://www.githubstatus.com/ |
| About | https://github.com/about |
| Partners | https://github.com/partners |
| Changelog | https://github.blog/changelog/ |
| GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub. | https://githubuniverse.com/ |
| Pricing | https://github.com/pricing |
| Sign in | https://redirect.github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fvonchenplus%2Fllama.cpp |
| Sign up | https://redirect.github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=vonchenplus%2Fllama.cpp |
| Reload | https://redirect.github.com/vonchenplus/llama.cpp |
| Reload | https://redirect.github.com/vonchenplus/llama.cpp |
| Reload | https://redirect.github.com/vonchenplus/llama.cpp |
| vonchenplus | https://redirect.github.com/vonchenplus |
| llama.cpp | https://redirect.github.com/vonchenplus/llama.cpp |
| Code | https://redirect.github.com/vonchenplus/llama.cpp |
| Pull requests | https://redirect.github.com/vonchenplus/llama.cpp/pulls |
| Actions | https://redirect.github.com/vonchenplus/llama.cpp/actions |
| Projects | https://redirect.github.com/vonchenplus/llama.cpp/projects |
| Security and quality | https://redirect.github.com/vonchenplus/llama.cpp/security |
| Insights | https://redirect.github.com/vonchenplus/llama.cpp/pulse |
| https://redirect.github.com/vonchenplus/llama.cpp |
| llama.cpp | https://redirect.github.com/vonchenplus/llama.cpp |
| ggml-org/llama.cpp | https://redirect.github.com/ggml-org/llama.cpp |
| Notifications | https://redirect.github.com/login?return_to=%2Fvonchenplus%2Fllama.cpp |
| Fork0 (0) | https://redirect.github.com/login?return_to=%2Fvonchenplus%2Fllama.cpp |
| Star0 (0) | https://redirect.github.com/login?return_to=%2Fvonchenplus%2Fllama.cpp |
| Readme | https://redirect.github.com/vonchenplus/llama.cpp#readme-ov-file |
| License | https://redirect.github.com/vonchenplus/llama.cpp#License-1-ov-file |
| Contributing | https://redirect.github.com/vonchenplus/llama.cpp#contributing-ov-file |
| Security policy | https://redirect.github.com/vonchenplus/llama.cpp#security-ov-file |
| Activity | https://redirect.github.com/vonchenplus/llama.cpp/activity |
| 0 forks | https://redirect.github.com/vonchenplus/llama.cpp/forks |
| Report repository | https://redirect.github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fvonchenplus%2Fllama.cpp&report=vonchenplus+%28user%29 |
| Branches | https://redirect.github.com/vonchenplus/llama.cpp/branches |
| Tags | https://redirect.github.com/vonchenplus/llama.cpp/tags |
| https://redirect.github.com/vonchenplus/llama.cpp/branches |
| https://redirect.github.com/vonchenplus/llama.cpp/tags |
| 4,400 Commits | https://redirect.github.com/vonchenplus/llama.cpp/commits/master/ |
| https://redirect.github.com/vonchenplus/llama.cpp/commits/master/ |
| .devops | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/.devops |
| .devops | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/.devops |
| .github | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/.github |
| .github | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/.github |
| Sources/llama | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/Sources/llama |
| Sources/llama | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/Sources/llama |
| ci | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/ci |
| ci | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/ci |
| cmake | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/cmake |
| cmake | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/cmake |
| common | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/common |
| common | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/common |
| docs | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/docs |
| docs | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/docs |
| examples | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/examples |
| examples | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/examples |
| ggml | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/ggml |
| ggml | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/ggml |
| gguf-py | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/gguf-py |
| gguf-py | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/gguf-py |
| grammars | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/grammars |
| grammars | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/grammars |
| include | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/include |
| include | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/include |
| media | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/media |
| media | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/media |
| models | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/models |
| models | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/models |
| pocs | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/pocs |
| pocs | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/pocs |
| prompts | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/prompts |
| prompts | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/prompts |
| requirements | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/requirements |
| requirements | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/requirements |
| scripts | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/scripts |
| scripts | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/scripts |
| spm-headers | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/spm-headers |
| spm-headers | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/spm-headers |
| src | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/src |
| src | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/src |
| tests | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/tests |
| tests | https://redirect.github.com/vonchenplus/llama.cpp/tree/master/tests |
| .clang-format | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.clang-format |
| .clang-format | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.clang-format |
| .clang-tidy | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.clang-tidy |
| .clang-tidy | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.clang-tidy |
| .dockerignore | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.dockerignore |
| .dockerignore | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.dockerignore |
| .ecrc | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.ecrc |
| .ecrc | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.ecrc |
| .editorconfig | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.editorconfig |
| .editorconfig | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.editorconfig |
| .flake8 | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.flake8 |
| .flake8 | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.flake8 |
| .gitignore | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.gitignore |
| .gitignore | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.gitignore |
| .gitmodules | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.gitmodules |
| .gitmodules | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.gitmodules |
| .pre-commit-config.yaml | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.pre-commit-config.yaml |
| .pre-commit-config.yaml | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/.pre-commit-config.yaml |
| AUTHORS | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/AUTHORS |
| AUTHORS | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/AUTHORS |
| CMakeLists.txt | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CMakeLists.txt |
| CMakeLists.txt | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CMakeLists.txt |
| CMakePresets.json | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CMakePresets.json |
| CMakePresets.json | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CMakePresets.json |
| CODEOWNERS | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CODEOWNERS |
| CODEOWNERS | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CODEOWNERS |
| CONTRIBUTING.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CONTRIBUTING.md |
| CONTRIBUTING.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CONTRIBUTING.md |
| LICENSE | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/LICENSE |
| LICENSE | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/LICENSE |
| Makefile | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/Makefile |
| Makefile | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/Makefile |
| Package.swift | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/Package.swift |
| Package.swift | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/Package.swift |
| README.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/README.md |
| README.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/README.md |
| SECURITY.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/SECURITY.md |
| SECURITY.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/SECURITY.md |
| convert_hf_to_gguf.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_hf_to_gguf.py |
| convert_hf_to_gguf.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_hf_to_gguf.py |
| convert_hf_to_gguf_update.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_hf_to_gguf_update.py |
| convert_hf_to_gguf_update.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_hf_to_gguf_update.py |
| convert_llama_ggml_to_gguf.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_llama_ggml_to_gguf.py |
| convert_llama_ggml_to_gguf.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_llama_ggml_to_gguf.py |
| convert_lora_to_gguf.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_lora_to_gguf.py |
| convert_lora_to_gguf.py | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/convert_lora_to_gguf.py |
| flake.lock | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/flake.lock |
| flake.lock | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/flake.lock |
| flake.nix | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/flake.nix |
| flake.nix | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/flake.nix |
| mypy.ini | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/mypy.ini |
| mypy.ini | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/mypy.ini |
| poetry.lock | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/poetry.lock |
| poetry.lock | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/poetry.lock |
| pyproject.toml | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/pyproject.toml |
| pyproject.toml | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/pyproject.toml |
| pyrightconfig.json | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/pyrightconfig.json |
| pyrightconfig.json | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/pyrightconfig.json |
| requirements.txt | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/requirements.txt |
| requirements.txt | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/requirements.txt |
| README | https://redirect.github.com/vonchenplus/llama.cpp |
| Contributing | https://redirect.github.com/vonchenplus/llama.cpp |
| License | https://redirect.github.com/vonchenplus/llama.cpp |
| Security | https://redirect.github.com/vonchenplus/llama.cpp |
| https://redirect.github.com/vonchenplus/llama.cpp#llamacpp |
| https://user-images.githubusercontent.com/1991296/230134379-7181e485-c521-4d23-a0d6-f7b3b61ba524.png |
| https://opensource.org/licenses/MIT |
| https://github.com/ggerganov/llama.cpp/actions/workflows/server.yml |
| Roadmap | https://github.com/users/ggerganov/projects/7 |
| Project status | https://github.com/ggerganov/llama.cpp/discussions/3471 |
| Manifesto | https://github.com/ggerganov/llama.cpp/discussions/205 |
| ggml | https://github.com/ggerganov/ggml |
| LLaMA | https://arxiv.org/abs/2302.13971 |
| https://redirect.github.com/vonchenplus/llama.cpp#recent-api-changes |
| Changelog for libllama API | https://github.com/ggerganov/llama.cpp/issues/9289 |
| Changelog for llama-server REST API | https://github.com/ggerganov/llama.cpp/issues/9291 |
| https://redirect.github.com/vonchenplus/llama.cpp#hot-topics |
| ggml-org#10123 | https://github.com/ggml-org/llama.cpp/discussions/10123 |
| ggml-org#9669 | https://github.com/ggml-org/llama.cpp/discussions/9669 |
| discussion | https://github.com/ggerganov/llama.cpp/discussions/9268 |
| tool | https://huggingface.co/spaces/CISCai/gguf-editor |
| https://redirect.github.com/vonchenplus/llama.cpp#description |
| ggml | https://github.com/ggerganov/ggml |
| HOWTO-add-model.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/development/HOWTO-add-model.md |
| https://redirect.github.com/vonchenplus/llama.cpp#text-only |
| Mistral 7B | https://huggingface.co/mistralai/Mistral-7B-v0.1 |
| Mixtral MoE | https://huggingface.co/models?search=mistral-ai/Mixtral |
| DBRX | https://huggingface.co/databricks/dbrx-instruct |
| Falcon | https://huggingface.co/models?search=tiiuae/falcon |
| Chinese LLaMA / Alpaca | https://github.com/ymcui/Chinese-LLaMA-Alpaca |
| Chinese LLaMA-2 / Alpaca-2 | https://github.com/ymcui/Chinese-LLaMA-Alpaca-2 |
| Vigogne (French) | https://github.com/bofenghuang/vigogne |
| BERT | https://github.com/ggerganov/llama.cpp/pull/5423 |
| Koala | https://bair.berkeley.edu/blog/2023/04/03/koala/ |
| Baichuan 1 & 2 | https://huggingface.co/models?search=baichuan-inc/Baichuan |
| derivations | https://huggingface.co/hiyouga/baichuan-7b-sft |
| Aquila 1 & 2 | https://huggingface.co/models?search=BAAI/Aquila |
| Starcoder models | https://github.com/ggerganov/llama.cpp/pull/3187 |
| Refact | https://huggingface.co/smallcloudai/Refact-1_6B-fim |
| MPT | https://github.com/ggerganov/llama.cpp/pull/3417 |
| Bloom | https://github.com/ggerganov/llama.cpp/pull/3553 |
| Yi models | https://huggingface.co/models?search=01-ai/Yi |
| StableLM models | https://huggingface.co/stabilityai |
| Deepseek models | https://huggingface.co/models?search=deepseek-ai/deepseek |
| Qwen models | https://huggingface.co/models?search=Qwen/Qwen |
| PLaMo-13B | https://github.com/ggerganov/llama.cpp/pull/3557 |
| Phi models | https://huggingface.co/models?search=microsoft/phi |
| GPT-2 | https://huggingface.co/gpt2 |
| Orion 14B | https://github.com/ggerganov/llama.cpp/pull/5118 |
| InternLM2 | https://huggingface.co/models?search=internlm2 |
| CodeShell | https://github.com/WisdomShell/codeshell |
| Gemma | https://ai.google.dev/gemma |
| Mamba | https://github.com/state-spaces/mamba |
| Grok-1 | https://huggingface.co/keyfan/grok-1-hf |
| Xverse | https://huggingface.co/models?search=xverse |
| Command-R models | https://huggingface.co/models?search=CohereForAI/c4ai-command-r |
| SEA-LION | https://huggingface.co/models?search=sea-lion |
| GritLM-7B | https://huggingface.co/GritLM/GritLM-7B |
| GritLM-8x7B | https://huggingface.co/GritLM/GritLM-8x7B |
| OLMo | https://allenai.org/olmo |
| OLMo 2 | https://allenai.org/olmo |
| OLMoE | https://huggingface.co/allenai/OLMoE-1B-7B-0924 |
| Granite models | https://huggingface.co/collections/ibm-granite/granite-code-models-6624c5cec322e4c148c8b330 |
| GPT-NeoX | https://github.com/EleutherAI/gpt-neox |
| Pythia | https://github.com/EleutherAI/pythia |
| Snowflake-Arctic MoE | https://huggingface.co/collections/Snowflake/arctic-66290090abe542894a5ac520 |
| Smaug | https://huggingface.co/models?search=Smaug |
| Poro 34B | https://huggingface.co/LumiOpen/Poro-34B |
| Bitnet b1.58 models | https://huggingface.co/1bitLLM |
| Flan T5 | https://huggingface.co/models?search=flan-t5 |
| Open Elm models | https://huggingface.co/collections/apple/openelm-instruct-models-6619ad295d7ae9f868b759ca |
| ChatGLM3-6b | https://huggingface.co/THUDM/chatglm3-6b |
| ChatGLM4-9b | https://huggingface.co/THUDM/glm-4-9b |
| SmolLM | https://huggingface.co/collections/HuggingFaceTB/smollm-6695016cad7167254ce15966 |
| EXAONE-3.0-7.8B-Instruct | https://huggingface.co/LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct |
| FalconMamba Models | https://huggingface.co/collections/tiiuae/falconmamba-7b-66b9a580324dd1598b0f6d4a |
| Jais | https://huggingface.co/inceptionai/jais-13b-chat |
| Bielik-11B-v2.3 | https://huggingface.co/collections/speakleash/bielik-11b-v23-66ee813238d9b526a072408a |
| RWKV-6 | https://github.com/BlinkDL/RWKV-LM |
| GigaChat-20B-A3B | https://huggingface.co/ai-sage/GigaChat-20B-A3B-instruct |
| https://redirect.github.com/vonchenplus/llama.cpp#multimodal |
| LLaVA 1.5 models | https://huggingface.co/collections/liuhaotian/llava-15-653aac15d994e992e2677a7e |
| LLaVA 1.6 models | https://huggingface.co/collections/liuhaotian/llava-16-65b9e40155f60fd046a5ccf2 |
| BakLLaVA | https://huggingface.co/models?search=SkunkworksAI/Bakllava |
| Obsidian | https://huggingface.co/NousResearch/Obsidian-3B-V0.5 |
| ShareGPT4V | https://huggingface.co/models?search=Lin-Chen/ShareGPT4V |
| MobileVLM 1.7B/3B models | https://huggingface.co/models?search=mobileVLM |
| Yi-VL | https://huggingface.co/models?search=Yi-VL |
| Mini CPM | https://huggingface.co/models?search=MiniCPM |
| Moondream | https://huggingface.co/vikhyatk/moondream2 |
| Bunny | https://github.com/BAAI-DCAI/Bunny |
| Qwen2-VL | https://huggingface.co/collections/Qwen/qwen2-vl-66cee7455501d7126940800d |
| abetlen/llama-cpp-python | https://github.com/abetlen/llama-cpp-python |
| go-skynet/go-llama.cpp | https://github.com/go-skynet/go-llama.cpp |
| withcatai/node-llama-cpp | https://github.com/withcatai/node-llama-cpp |
| lgrammel/modelfusion | https://modelfusion.dev/integration/model-provider/llamacpp |
| offline-ai/cli | https://github.com/offline-ai/cli |
| tangledgroup/llama-cpp-wasm | https://github.com/tangledgroup/llama-cpp-wasm |
| ngxson/wllama | https://github.com/ngxson/wllama |
| yoshoku/llama_cpp.rb | https://github.com/yoshoku/llama_cpp.rb |
| edgenai/llama_cpp-rs | https://github.com/edgenai/llama_cpp-rs |
| mdrokz/rust-llama.cpp | https://github.com/mdrokz/rust-llama.cpp |
| utilityai/llama-cpp-rs | https://github.com/utilityai/llama-cpp-rs |
| SciSharp/LLamaSharp | https://github.com/SciSharp/LLamaSharp |
| LM-Kit.NET | https://docs.lm-kit.com/lm-kit-net/index.html |
| donderom/llm4s | https://github.com/donderom/llm4s |
| phronmophobic/llama.clj | https://github.com/phronmophobic/llama.clj |
| mybigday/llama.rn | https://github.com/mybigday/llama.rn |
| kherud/java-llama.cpp | https://github.com/kherud/java-llama.cpp |
| deins/llama.cpp.zig | https://github.com/Deins/llama.cpp.zig |
| netdur/llama_cpp_dart | https://github.com/netdur/llama_cpp_dart |
| xuegao-tzx/Fllama | https://github.com/xuegao-tzx/Fllama |
| distantmagic/resonance | https://github.com/distantmagic/resonance |
| (more info) | https://github.com/ggerganov/llama.cpp/pull/6326 |
| guile_llama_cpp | https://savannah.nongnu.org/projects/guile-llama-cpp |
| srgtuszy/llama-cpp-swift | https://github.com/srgtuszy/llama-cpp-swift |
| ShenghaiWang/SwiftLlama | https://github.com/ShenghaiWang/SwiftLlama |
| AI Sublime Text plugin | https://github.com/yaroslavyaroslav/OpenAI-sublime-text |
| cztomsik/ava | https://github.com/cztomsik/ava |
| Dot | https://github.com/alexpinel/Dot |
| eva | https://github.com/ylsdamxssjxxdd/eva |
| iohub/collama | https://github.com/iohub/coLLaMA |
| janhq/jan | https://github.com/janhq/jan |
| KanTV | https://github.com/zhouwg/kantv?tab=readme-ov-file |
| KodiBot | https://github.com/firatkiral/kodibot |
| llama.vim | https://github.com/ggml-org/llama.vim |
| LARS | https://github.com/abgulati/LARS |
| Llama Assistant | https://github.com/vietanhdev/llama-assistant |
| LLMFarm | https://github.com/guinmoon/LLMFarm?tab=readme-ov-file |
| LLMUnity | https://github.com/undreamai/LLMUnity |
| LMStudio | https://lmstudio.ai/ |
| LocalAI | https://github.com/mudler/LocalAI |
| LostRuins/koboldcpp | https://github.com/LostRuins/koboldcpp |
| MindMac | https://mindmac.app |
| MindWorkAI/AI-Studio | https://github.com/MindWorkAI/AI-Studio |
| Mobile-Artificial-Intelligence/maid | https://github.com/Mobile-Artificial-Intelligence/maid |
| Mozilla-Ocho/llamafile | https://github.com/Mozilla-Ocho/llamafile |
| nat/openplayground | https://github.com/nat/openplayground |
| nomic-ai/gpt4all | https://github.com/nomic-ai/gpt4all |
| ollama/ollama | https://github.com/ollama/ollama |
| oobabooga/text-generation-webui | https://github.com/oobabooga/text-generation-webui |
| PocketPal AI | https://github.com/a-ghorbani/pocketpal-ai |
| psugihara/FreeChat | https://github.com/psugihara/FreeChat |
| ptsochantaris/emeltal | https://github.com/ptsochantaris/emeltal |
| pythops/tenere | https://github.com/pythops/tenere |
| ramalama | https://github.com/containers/ramalama |
| semperai/amica | https://github.com/semperai/amica |
| withcatai/catai | https://github.com/withcatai/catai |
| akx/ggify | https://github.com/akx/ggify |
| akx/ollama-dl | https://github.com/akx/ollama-dl |
| crashr/gppm | https://github.com/crashr/gppm |
| gpustack/gguf-parser | https://github.com/gpustack/gguf-parser-go/tree/main/cmd/gguf-parser |
| Styled Lines | https://marketplace.unity.com/packages/tools/generative-ai/styled-lines-llama-cpp-model-292902 |
| Paddler | https://github.com/distantmagic/paddler |
| GPUStack | https://github.com/gpustack/gpustack |
| llama_cpp_canister | https://github.com/onicai/llama_cpp_canister |
| Lucy's Labyrinth | https://github.com/MorganRO8/Lucys_Labyrinth |
| https://redirect.github.com/vonchenplus/llama.cpp#supported-backends |
| Metal | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#metal-build |
| BLAS | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#blas-build |
| BLIS | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/backend/BLIS.md |
| SYCL | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/backend/SYCL.md |
| MUSA | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#musa |
| CUDA | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#cuda |
| HIP | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#hip |
| Vulkan | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#vulkan |
| CANN | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md#cann |
| https://redirect.github.com/vonchenplus/llama.cpp#building-the-project |
| include/llama.h | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/include/llama.h |
| how to build | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md |
| brew, flox or nix | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/install.md |
| documentation for Docker | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/docker.md |
| releases | https://github.com/ggerganov/llama.cpp/releases |
| https://redirect.github.com/vonchenplus/llama.cpp#obtaining-and-quantizing-models |
| Hugging Face | https://huggingface.co |
| number of LLMs | https://huggingface.co/models?library=gguf&sort=trending |
| Trending | https://huggingface.co/models?library=gguf&sort=trending |
| LLaMA | https://huggingface.co/models?sort=trending&search=llama+gguf |
| GGUF | https://github.com/ggerganov/ggml/blob/master/docs/gguf.md |
| GGUF-my-repo space | https://huggingface.co/spaces/ggml-org/gguf-my-repo |
| GGUF-my-LoRA space | https://huggingface.co/spaces/ggml-org/gguf-my-lora |
| ggml-org#10123 | https://github.com/ggml-org/llama.cpp/discussions/10123 |
| GGUF-editor space | https://huggingface.co/spaces/CISCai/gguf-editor |
| ggml-org#9268 | https://github.com/ggml-org/llama.cpp/discussions/9268 |
| Inference Endpoints | https://ui.endpoints.huggingface.co/ |
| ggml-org#9669 | https://github.com/ggml-org/llama.cpp/discussions/9669 |
| read this documentation | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/quantize/README.md |
| llama-cli | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/main |
| https://redirect.github.com/vonchenplus/llama.cpp#llama-cli |
| https://redirect.github.com/vonchenplus/llama.cpp#a-cli-tool-for-accessing-and-experimenting-with-most-of-llamacpps-functionality |
| Supported templates | https://github.com/ggerganov/llama.cpp/wiki/Templates-supported-by-llama_chat_apply_template |
| grammars/ | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/grammars |
| GBNF Guide | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/grammars/README.md |
| https://grammar.intrinsiclabs.ai/ | https://grammar.intrinsiclabs.ai/ |
| llama-server | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/server |
| https://redirect.github.com/vonchenplus/llama.cpp#llama-server |
| OpenAI API | https://github.com/openai/openai-openapi |
| https://redirect.github.com/vonchenplus/llama.cpp#a-lightweight-openai-api-compatible-http-server-for-serving-llms |
| llama-perplexity | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/perplexity |
| https://redirect.github.com/vonchenplus/llama.cpp#llama-perplexity |
| 1 | https://redirect.github.com/vonchenplus/llama.cpp#user-content-fn-1-d29bca9ac93f9e17559b36676e3741c6 |
| 2 | https://redirect.github.com/vonchenplus/llama.cpp#user-content-fn-2-d29bca9ac93f9e17559b36676e3741c6 |
| https://redirect.github.com/vonchenplus/llama.cpp#a-tool-for-measuring-the-perplexity-12-and-other-quality-metrics-of-a-model-over-a-given-text |
| llama-bench | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/llama-bench |
| https://redirect.github.com/vonchenplus/llama.cpp#llama-bench |
| https://redirect.github.com/vonchenplus/llama.cpp#benchmark-the-performance-of-the-inference-for-various-parameters |
| llama-run | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/run |
| https://redirect.github.com/vonchenplus/llama.cpp#llama-run |
| 3 | https://redirect.github.com/vonchenplus/llama.cpp#user-content-fn-3-d29bca9ac93f9e17559b36676e3741c6 |
| https://redirect.github.com/vonchenplus/llama.cpp#a-comprehensive-example-for-running-llamacpp-models-useful-for-inferencing-used-with-ramalama-3 |
| llama-simple | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/simple |
| https://redirect.github.com/vonchenplus/llama.cpp#llama-simple |
| https://redirect.github.com/vonchenplus/llama.cpp#a-minimal-example-for-implementing-apps-with-llamacpp-useful-for-developers |
| https://redirect.github.com/vonchenplus/llama.cpp#contributing |
| good first issues | https://github.com/ggerganov/llama.cpp/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22 |
| CONTRIBUTING.md | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/CONTRIBUTING.md |
| Inference at the edge | https://github.com/ggerganov/llama.cpp/discussions/205 |
| Changelog podcast | https://changelog.com/podcast/532 |
| https://redirect.github.com/vonchenplus/llama.cpp#other-documentation |
| main (cli) | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/main/README.md |
| server | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/examples/server/README.md |
| GBNF grammars | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/grammars/README.md |
| https://redirect.github.com/vonchenplus/llama.cpp#development-documentation |
| How to build | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/build.md |
| Running on Docker | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/docker.md |
| Build on Android | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/android.md |
| Performance troubleshooting | https://redirect.github.com/vonchenplus/llama.cpp/blob/master/docs/development/token_generation_performance_tips.md |
| GGML tips & tricks | https://github.com/ggerganov/llama.cpp/wiki/GGML-Tips-&-Tricks |
| https://redirect.github.com/vonchenplus/llama.cpp#seminal-papers-and-background-on-the-models |
| Introducing LLaMA: A foundational, 65-billion-parameter large language model | https://ai.facebook.com/blog/large-language-model-llama-meta-ai/ |
| LLaMA: Open and Efficient Foundation Language Models | https://arxiv.org/abs/2302.13971 |
| Language Models are Few-Shot Learners | https://arxiv.org/abs/2005.14165 |
| Aligning language models to follow instructions | https://openai.com/research/instruction-following |
| Training language models to follow instructions with human feedback | https://arxiv.org/abs/2203.02155 |
| https://redirect.github.com/vonchenplus/llama.cpp#references |
| examples/perplexity/README.md | https://redirect.github.com/vonchenplus/examples/perplexity/README.md |
| ↩ | https://redirect.github.com/vonchenplus/llama.cpp#user-content-fnref-1-d29bca9ac93f9e17559b36676e3741c6 |
| https://huggingface.co/docs/transformers/perplexity | https://huggingface.co/docs/transformers/perplexity |
| ↩ | https://redirect.github.com/vonchenplus/llama.cpp#user-content-fnref-2-d29bca9ac93f9e17559b36676e3741c6 |
| RamaLama | https://github.com/containers/ramalama |
| ↩ | https://redirect.github.com/vonchenplus/llama.cpp#user-content-fnref-3-d29bca9ac93f9e17559b36676e3741c6 |
| Readme | https://redirect.github.com/vonchenplus/llama.cpp#readme-ov-file |
| License | https://redirect.github.com/vonchenplus/llama.cpp#License-1-ov-file |
| Contributing | https://redirect.github.com/vonchenplus/llama.cpp#contributing-ov-file |
| Security policy | https://redirect.github.com/vonchenplus/llama.cpp#security-ov-file |
| Activity | https://redirect.github.com/vonchenplus/llama.cpp/activity |
| 0 forks | https://redirect.github.com/vonchenplus/llama.cpp/forks |
| Report repository | https://redirect.github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fvonchenplus%2Fllama.cpp&report=vonchenplus+%28user%29 |
|
| https://github.com |
| Terms | https://docs.github.com/site-policy/github-terms/github-terms-of-service |
| Privacy | https://docs.github.com/site-policy/privacy-policies/github-privacy-statement |
| Security | https://github.com/security |
| Status | https://www.githubstatus.com/ |
| Community | https://github.community/ |
| Docs | https://docs.github.com/ |
| Contact | https://support.github.com?tags=dotcom-footer |