| react-profiling | 0 |
| react-import-map | react,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322 |
| route-pattern | /:user_id/:repository |
| route-controller | files |
| route-action | disambiguate |
| fetch-nonce | v2:a85fac27-13c1-64d3-3f06-667e0be89acc |
| current-catalog-service-hash | f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb |
| request-id | 89A2:2880D7:1078AC7:14BC583:6AFF8A79 |
| html-safe-nonce | 7ee383ce4fbcc75439341f8d8544305a614292aef0cd441e418265137846c8cd |
| visitor-payload | eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI4OUEyOjI4ODBENzoxMDc4QUM3OjE0QkM1ODM6NkFGRjhBNzkiLCJ2aXNpdG9yX2lkIjoiODYzNDk2OTU1OTI3NjY4NTk0NiIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9 |
| visitor-hmac | 891a6e17e4ddcb094da2df0306d88ddc58d398fc3117cde153236f8b18dfc383 |
| hovercard-subject-tag | repository:1398949688 |
| github-keyboard-shortcuts | repository,copilot |
| google-site-verification | Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I |
| octolytics-url | https://collector.github.com/github/collect |
| analytics-location | // |
| fb:app_id | 1401488693436528 |
| apple-itunes-app | app-id=1477376905, app-argument=https://github.com/xxyykkxx/llama-cpp-python |
| twitter:image | https://opengraph.githubassets.com/b3f53d19c1166f24414ec5e97926fd54762176692d71e42f88fadb03f0d552a2/xxyykkxx/llama-cpp-python |
| twitter:card | summary_large_image |
| og:image | https://opengraph.githubassets.com/b3f53d19c1166f24414ec5e97926fd54762176692d71e42f88fadb03f0d552a2/xxyykkxx/llama-cpp-python |
| og:image:alt | Efficient Python bindings for the llama.cpp library - xxyykkxx/llama-cpp-python |
| og:image:width | 1200 |
| og:image:height | 600 |
| og:site_name | GitHub |
| og:type | object |
| hostname | github.com |
| expected-hostname | github.com |
| None | 587ad99a3e443b2e1cb0d3df059ec49ab54389861f13310766dfdeb615fed90d |
| turbo-cache-control | no-cache |
| go-import | github.com/xxyykkxx/llama-cpp-python git https://github.com/xxyykkxx/llama-cpp-python.git |
| octolytics-dimension-user_id | 46990259 |
| octolytics-dimension-user_login | xxyykkxx |
| octolytics-dimension-repository_id | 1398949688 |
| octolytics-dimension-repository_nwo | xxyykkxx/llama-cpp-python |
| octolytics-dimension-repository_public | true |
| octolytics-dimension-repository_is_fork | true |
| octolytics-dimension-repository_parent_id | 920296412 |
| octolytics-dimension-repository_parent_nwo | JamePeng/llama-cpp-python |
| octolytics-dimension-repository_network_root_id | 617868717 |
| octolytics-dimension-repository_network_root_nwo | abetlen/llama-cpp-python |
| turbo-body-classes | logged-out env-production page-responsive |
| disable-turbo | false |
| browser-stats-url | https://api.github.com/_private/browser/stats |
| browser-errors-url | https://api.github.com/_private/browser/errors |
| release | 5bb0fb68d64881d1344f5863d809515d496801e5 |
| ui-target | full |
| theme-color | #1e2327 |
| color-scheme | light dark |
| Skip to content | https://github.com/xxyykkxx/llama-cpp-python#start-of-content |
| https://github.com/ |
| Sign in | https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fxxyykkxx%2Fllama-cpp-python |
| GitHubRepository hosting, code review, and collaboration | https://github.com/platform |
| CopilotAI coding agent for tasks, projects, and automated work | https://github.com/features/copilot |
| ActionsCI/CD and automation | https://github.com/features/actions |
| Code QualityQuality analysis, fixes, and enforcement | https://github.com/features/code-quality |
| CodespacesCloud development environments | https://github.com/features/codespaces |
| Code SecurityVulnerability detection and remediation | https://github.com/security/advanced-security/code-security |
| Secret ProtectionSecret scanning and leak prevention | https://github.com/security/advanced-security/secret-protection |
| All Features | https://github.com/features |
| Documentation | https://docs.github.com/en |
| Training | https://learn.github.com/skills |
| Certifications | https://learn.github.com/credentials |
| Build with Copilot | https://github.com/features/ai |
| Customize Copilot | https://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview |
| Integrations | https://github.com/marketplace |
| Trending repos | https://github.com/trending |
| GitHub Sponsors | https://github.com/open-source/sponsors |
| Developer articles | https://github.com/resources/articles |
| ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place. | https://github.blog/changelog/ |
| Enterprises | https://github.com/enterprise |
| Team | https://github.com/team |
| Startups | https://github.com/enterprise/startups |
| Nonprofits | https://github.com/solutions/industry/nonprofits |
| Healthcare | https://github.com/solutions/industry/healthcare |
| Financial services | https://github.com/solutions/industry/financial-services |
| Manufacturing | https://github.com/solutions/industry/manufacturing |
| Government | https://github.com/solutions/industry/government |
| See all industries | https://github.com/solutions/industry |
| Why GitHub | https://github.com/why-github |
| Customer Stories | https://github.com/customer-stories |
| Business insights | https://github.com/solutions/executive-insights |
| Trust Center | https://github.com/trust-center |
| GitHub Advanced SecurityEnterprise-grade code security | https://github.com/security/advanced-security |
| Copilot for businessEnterprise-grade AI features | https://github.com/features/copilot/copilot-business |
| Premium SupportEnterprise-grade 24/7 support | https://github.com/enterprise/premium-support |
| Documentation | https://docs.github.com/en |
| Resource center | https://github.com/resources |
| Blog | https://github.blog/ |
| Events & webinars | https://github.com/resources/events |
| Community Forum | https://github.com/orgs/community/discussions |
| Customer Stories | https://github.com/customer-stories |
| Integrations | https://github.com/marketplace |
| Customer support | https://help.github.com/ |
| Training | https://learn.github.com/skills |
| GitHub Status | https://www.githubstatus.com/ |
| About | https://github.com/about |
| Partners | https://github.com/partners |
| Changelog | https://github.blog/changelog/ |
| GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub. | https://githubuniverse.com/ |
| Pricing | https://github.com/pricing |
| Sign in | https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fxxyykkxx%2Fllama-cpp-python |
| Sign up | https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=xxyykkxx%2Fllama-cpp-python |
| Reload | https://github.com/xxyykkxx/llama-cpp-python |
| Reload | https://github.com/xxyykkxx/llama-cpp-python |
| Reload | https://github.com/xxyykkxx/llama-cpp-python |
| xxyykkxx | https://github.com/xxyykkxx |
| llama-cpp-python | https://github.com/xxyykkxx/llama-cpp-python |
| Code | https://github.com/xxyykkxx/llama-cpp-python |
| Pull requests | https://github.com/xxyykkxx/llama-cpp-python/pulls |
| Actions | https://github.com/xxyykkxx/llama-cpp-python/actions |
| Projects | https://github.com/xxyykkxx/llama-cpp-python/projects |
| Security and quality | https://github.com/xxyykkxx/llama-cpp-python/security |
| Insights | https://github.com/xxyykkxx/llama-cpp-python/pulse |
| https://github.com/xxyykkxx/llama-cpp-python |
| llama-cpp-python | https://github.com/xxyykkxx/llama-cpp-python |
| JamePeng/llama-cpp-python | https://github.com/JamePeng/llama-cpp-python |
| Notifications | https://github.com/login?return_to=%2Fxxyykkxx%2Fllama-cpp-python |
| Fork0 (0) | https://github.com/login?return_to=%2Fxxyykkxx%2Fllama-cpp-python |
| Star0 (0) | https://github.com/login?return_to=%2Fxxyykkxx%2Fllama-cpp-python |
| github.com/JamePeng/llama-cpp-python/wiki | https://github.com/JamePeng/llama-cpp-python/wiki |
| Readme | https://github.com/xxyykkxx/llama-cpp-python#readme-ov-file |
| License | https://github.com/xxyykkxx/llama-cpp-python#License-1-ov-file |
| Activity | https://github.com/xxyykkxx/llama-cpp-python/activity |
| 0 forks | https://github.com/xxyykkxx/llama-cpp-python/forks |
| Report repository | https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fxxyykkxx%2Fllama-cpp-python&report=xxyykkxx+%28user%29 |
| Branches | https://github.com/xxyykkxx/llama-cpp-python/branches |
| Tags | https://github.com/xxyykkxx/llama-cpp-python/tags |
| https://github.com/xxyykkxx/llama-cpp-python/branches |
| https://github.com/xxyykkxx/llama-cpp-python/tags |
| 2,980 Commits | https://github.com/xxyykkxx/llama-cpp-python/commits/main/ |
| https://github.com/xxyykkxx/llama-cpp-python/commits/main/ |
| .github | https://github.com/xxyykkxx/llama-cpp-python/tree/main/.github |
| .github | https://github.com/xxyykkxx/llama-cpp-python/tree/main/.github |
| docker | https://github.com/xxyykkxx/llama-cpp-python/tree/main/docker |
| docker | https://github.com/xxyykkxx/llama-cpp-python/tree/main/docker |
| docs | https://github.com/xxyykkxx/llama-cpp-python/tree/main/docs |
| docs | https://github.com/xxyykkxx/llama-cpp-python/tree/main/docs |
| examples | https://github.com/xxyykkxx/llama-cpp-python/tree/main/examples |
| examples | https://github.com/xxyykkxx/llama-cpp-python/tree/main/examples |
| llama_cpp | https://github.com/xxyykkxx/llama-cpp-python/tree/main/llama_cpp |
| llama_cpp | https://github.com/xxyykkxx/llama-cpp-python/tree/main/llama_cpp |
| scripts | https://github.com/xxyykkxx/llama-cpp-python/tree/main/scripts |
| scripts | https://github.com/xxyykkxx/llama-cpp-python/tree/main/scripts |
| tests | https://github.com/xxyykkxx/llama-cpp-python/tree/main/tests |
| tests | https://github.com/xxyykkxx/llama-cpp-python/tree/main/tests |
| tools/abi | https://github.com/xxyykkxx/llama-cpp-python/tree/main/tools/abi |
| tools/abi | https://github.com/xxyykkxx/llama-cpp-python/tree/main/tools/abi |
| vendor | https://github.com/xxyykkxx/llama-cpp-python/tree/main/vendor |
| vendor | https://github.com/xxyykkxx/llama-cpp-python/tree/main/vendor |
| .dockerignore | https://github.com/xxyykkxx/llama-cpp-python/blob/main/.dockerignore |
| .dockerignore | https://github.com/xxyykkxx/llama-cpp-python/blob/main/.dockerignore |
| .gitignore | https://github.com/xxyykkxx/llama-cpp-python/blob/main/.gitignore |
| .gitignore | https://github.com/xxyykkxx/llama-cpp-python/blob/main/.gitignore |
| .gitmodules | https://github.com/xxyykkxx/llama-cpp-python/blob/main/.gitmodules |
| .gitmodules | https://github.com/xxyykkxx/llama-cpp-python/blob/main/.gitmodules |
| CHANGELOG.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/CHANGELOG.md |
| CHANGELOG.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/CHANGELOG.md |
| CMakeLists.txt | https://github.com/xxyykkxx/llama-cpp-python/blob/main/CMakeLists.txt |
| CMakeLists.txt | https://github.com/xxyykkxx/llama-cpp-python/blob/main/CMakeLists.txt |
| LICENSE.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/LICENSE.md |
| LICENSE.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/LICENSE.md |
| Makefile | https://github.com/xxyykkxx/llama-cpp-python/blob/main/Makefile |
| Makefile | https://github.com/xxyykkxx/llama-cpp-python/blob/main/Makefile |
| README.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/README.md |
| README.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/README.md |
| pyproject.toml | https://github.com/xxyykkxx/llama-cpp-python/blob/main/pyproject.toml |
| pyproject.toml | https://github.com/xxyykkxx/llama-cpp-python/blob/main/pyproject.toml |
| README | https://github.com/xxyykkxx/llama-cpp-python |
| License | https://github.com/xxyykkxx/llama-cpp-python |
| https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/icon.png |
| llama.cpp | https://github.com/ggml-org/llama.cpp |
| https://github.com/xxyykkxx/llama-cpp-python#efficient-python-bindings-for-llamacpp-library |
| https://github.com/JamePeng/llama-cpp-python/actions/workflows/test.yaml |
| https://camo.githubusercontent.com/0026f4c164c97907dccf892a7d855514d9d14a6b559f659a1592113081ff192c/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f762f7461672f4a616d6550656e672f6c6c616d612d6370702d707974686f6e |
| https://pypi.org/project/llama-cpp-python/ |
| https://pepy.tech/projects/llama-cpp-python |
| https://github.com/JamePeng/llama-cpp-python/releases |
| llama.cpp | https://github.com/ggml-org/llama.cpp |
| Low-level API | https://github.com/xxyykkxx/llama-cpp-python#low-level-api |
| Low-level tutorial and examples | https://github.com/xxyykkxx/llama-cpp-python/blob/main/examples/low_level_api/README.md |
| llama_cpp_lib | https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/llama_cpp.py |
| mtmd_cpp_lib | https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/mtmd_cpp.py |
| ggml_cpp_lib | https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/_ggml.py |
| llama_types.py | https://github.com/JamePeng/llama-cpp-python/blob/main/llama_cpp/llama_types.py |
| High-level API | https://github.com/xxyykkxx/llama-cpp-python#high-level-api |
| Continuing Assistant Responses (Prefill) | https://github.com/JamePeng/llama-cpp-python#continuing-assistant-responses-prefill |
| Dynamic LoRA Routing & Control Vectors (Multi-Tenant Serving) | https://github.com/JamePeng/llama-cpp-python#dynamic-lora-routing--control-vectors-multi-tenant-serving |
| Dynamic LoRA Example | https://github.com/JamePeng/llama-cpp-python#dynamic-lora-example |
| Control Vector Injection (Representation Engineering) | https://github.com/JamePeng/llama-cpp-python#control-vector-injection-representation-engineering |
| Sampling Configuration & Usage (LlamaSamplingParams) | https://github.com/JamePeng/llama-cpp-python#sampling-configuration--usage-llamasamplingparams |
| How to use the ReasoningBudgetSampler | https://github.com/JamePeng/llama-cpp-python#reasoning-budget-first-reasoning-block |
| Speculative Decoding | https://github.com/JamePeng/llama-cpp-python#speculative-decoding |
| MTP speculative decoding | https://github.com/JamePeng/llama-cpp-python#mtp-speculative-decoding |
| DFlash, DFlash2, and DSpark speculative decoding | https://github.com/JamePeng/llama-cpp-python#dflash-dflash2-and-dspark-speculative-decoding |
| N-gram speculative decoding | https://github.com/JamePeng/llama-cpp-python#n-gram-speculative-decoding |
| Multi-modal Models Support | https://github.com/JamePeng/llama-cpp-python#multi-modal-models |
| Introducing Generic MTMD Chat Handler | https://github.com/JamePeng/llama-cpp-python#generic-mtmd-chat-handler |
| Loading a Local Video With Generic MTMD | https://github.com/JamePeng/llama-cpp-python#loading-a-local-video-with-generic-mtmd |
| Loading a Local Image With Qwen3VL(Thinking/Instruct) | https://github.com/JamePeng/llama-cpp-python#loading-a-local-image-with-qwen3vlthinkinginstruct |
| Speech Recognition With Qwen3-ASR (Speech-to-Text) | https://github.com/JamePeng/llama-cpp-python#speech-recognition-with-qwen3-asr-speech-to-text |
| Speech Synthesis With MTMD (Text-to-Speech) | https://github.com/xxyykkxx/llama-cpp-python#speech-synthesis-with-mtmd-text-to-speech |
| Comprehensive Omni MultiModal Example: Gemma-4 (Vision + Audio + Video + Text) | https://github.com/JamePeng/llama-cpp-python#comprehensive-omni-multimodal-example-gemma-4-vision--audio--video--text |
| Embeddings & Reranking (GGUF) | https://github.com/JamePeng/llama-cpp-python#embeddings--reranking-gguf |
| 1. Text Embeddings (Vector Search) | https://github.com/JamePeng/llama-cpp-python#1-text-embeddings-vector-search |
| 2. Reranking (Cross-Encoder Scoring) | https://github.com/JamePeng/llama-cpp-python#2-reranking-cross-encoder-scoring |
| 3. Normalization | https://github.com/JamePeng/llama-cpp-python#3-normalization |
| FAQ | https://github.com/JamePeng/llama-cpp-python#faq |
| docs/wiki | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/index.md |
| project wiki | https://github.com/JamePeng/llama-cpp-python/wiki |
| https://github.com/xxyykkxx/llama-cpp-python#discussions |
| GitHub Discussions | https://github.com/JamePeng/llama-cpp-python/discussions |
| docs/wiki | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/index.md |
| https://github.com/xxyykkxx/llama-cpp-python#installation |
| docs/wiki/install.md | https://github.com/JamePeng/llama-cpp-python/blob/main/docs/wiki/install.md |
| Visual Studio 2022 Build Tools | https://download.visualstudio.microsoft.com/download/pr/6efb3484-905b-485c-8b5f-9d3a5f39e731/07908cd6d91e75b8ea4339d8f2cfa6e8d8bb4fd706af7b918ae391cd6fc2a066/vs_BuildTools.exe |
| https://github.com/xxyykkxx/llama-cpp-python#installation-configuration |
| llama.cpp build docs | https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md |
| https://github.com/xxyykkxx/llama-cpp-python#supported-backends |
| https://developer.nvidia.com/cuda-toolkit-archive | https://developer.nvidia.com/cuda-toolkit-archive |
| https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#cuda | https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#cuda |
| https://github.com/JamePeng/llama-cpp-python/releases | https://github.com/JamePeng/llama-cpp-python/releases |
| https://github.com/xxyykkxx/llama-cpp-python#install-openvino-runtime |
| Linux | https://docs.openvino.ai/2026/get-started/install-openvino/install-openvino-archive-linux.html |
| Windows | https://docs.openvino.ai/2026/get-started/install-openvino/install-openvino-archive-windows.html |
| https://github.com/xxyykkxx/llama-cpp-python#supported-devices |
| Intel hardware | https://docs.openvino.ai/2026/about-openvino/release-notes-openvino/system-requirements.html |
| https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/OPENVINO.md | https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/OPENVINO.md |
| https://github.com/JamePeng/llama-cpp-python/releases | https://github.com/JamePeng/llama-cpp-python/releases |
| ROCm Quick Start (Linux) | https://rocm.docs.amd.com/projects/install-on-linux/en/latest/tutorial/quick-start.html#rocm-install-quick |
| https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#hip | https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#hip |
| Vulkan SDK | https://vulkan.lunarg.com/sdk/home#windows |
| Getting Started with the Linux Tarball Vulkan SDK | https://vulkan.lunarg.com/doc/sdk/latest/linux/getting_started.html |
| Getting Started with the MacOS Vulkan SDK | https://vulkan.lunarg.com/doc/sdk/latest/mac/getting_started.html |
| https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#vulkan | https://github.com/ggml-org/llama.cpp/blob/master/docs/build.md#vulkan |
| https://github.com/xxyykkxx/llama-cpp-python#supported-os |
| https://github.com/xxyykkxx/llama-cpp-python#intel-gpu |
| oneAPI supported GPU | https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-base-toolkit-system-requirements.html#inpage-nav-1-1 |
| OpenCL guide | https://github.com/xxyykkxx/llama-cpp-python/blob/main/vendor/llama.cpp/docs/backend/OPENCL.md |
| https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md | https://github.com/ggml-org/llama.cpp/blob/master/docs/backend/SYCL.md |
| llama.cpp releases | https://github.com/ggml-org/llama.cpp/releases |
| Multimodal Models with RPC | https://github.com/xxyykkxx/llama-cpp-python#multimodal-models-with-rpc |
| https://github.com/xxyykkxx/llama-cpp-python#install-notes |
| https://github.com/xxyykkxx/llama-cpp-python#windows-notes |
| https://github.com/xxyykkxx/llama-cpp-python#wheel-packaging-checklist |
| https://github.com/xxyykkxx/llama-cpp-python#upgrading-and-reinstalling |
| https://github.com/xxyykkxx/llama-cpp-python#low-level-api |
| Low-level tutorial and examples | https://github.com/xxyykkxx/llama-cpp-python/blob/main/examples/low_level_api/README.md |
| ctypes | https://docs.python.org/3/library/ctypes.html |
| llama_cpp/llama_cpp.py | https://github.com/xxyykkxx/llama-cpp-python/blob/main/llama_cpp/llama_cpp.py |
| llama.h | https://github.com/xxyykkxx/llama-cpp-python/blob/main/vendor/llama.cpp/include/llama.h |
| llama_cpp/mtmd_cpp.py | https://github.com/xxyykkxx/llama-cpp-python/blob/main/llama_cpp/mtmd_cpp.py |
| llama_cpp/_ggml.py | https://github.com/xxyykkxx/llama-cpp-python/blob/main/llama_cpp/_ggml.py |
| examples/low_level_api | https://github.com/xxyykkxx/llama-cpp-python/blob/main/examples/low_level_api |
| https://github.com/xxyykkxx/llama-cpp-python#high-level-api |
| API Reference | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/core/Llama.md |
| Llama | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/core/Llama.md |
| https://github.com/xxyykkxx/llama-cpp-python#quick-start |
| Llama.from_pretrained | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/core/Llama.md |
| Qwen3-0.6B-GGUF | https://huggingface.co/Qwen/Qwen3-0.6B-GGUF |
| https://github.com/xxyykkxx/llama-cpp-python#loading-a-local-gguf |
| Llama class guide | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/core/Llama.md |
| https://github.com/xxyykkxx/llama-cpp-python#streaming-chat-completion |
| https://github.com/xxyykkxx/llama-cpp-python#more-high-level-features |
| Llama guide | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/core/Llama.md |
| https://github.com/xxyykkxx/llama-cpp-python#continuing-assistant-responses-prefill |
| https://github.com/xxyykkxx/llama-cpp-python#dynamic-lora-routing--control-vectors-multi-tenant-serving |
| https://github.com/xxyykkxx/llama-cpp-python#dynamic-lora-example |
| https://github.com/xxyykkxx/llama-cpp-python#control-vector-injection-representation-engineering |
| https://github.com/xxyykkxx/llama-cpp-python#sampling-configuration--usage-llamasamplingparams |
| https://github.com/xxyykkxx/llama-cpp-python#core-sampling-parameters |
| https://github.com/xxyykkxx/llama-cpp-python#advanced--experimental-samplers |
| https://github.com/xxyykkxx/llama-cpp-python#target-entropy-mirostat |
| https://github.com/xxyykkxx/llama-cpp-python#repetition-penalties |
| https://github.com/xxyykkxx/llama-cpp-python#constraints--callbacks |
| https://github.com/xxyykkxx/llama-cpp-python#reasoning-budget-first-reasoning-block |
| https://github.com/xxyykkxx/llama-cpp-python#️-usage-example |
| https://github.com/xxyykkxx/llama-cpp-python#speculative-decoding |
| llama-cpp-python | https://github.com/JamePeng/llama-cpp-python |
| Llama Speculative Decoding | https://github.com/JamePeng/llama-cpp-python/blob/main/docs/wiki/modules/LlamaSpeculative.md |
| DFlash2 Speculative Decoding | https://github.com/JamePeng/llama-cpp-python/blob/main/docs/wiki/examples/dflash2-speculative-decoding.md |
| https://github.com/xxyykkxx/llama-cpp-python#mtp-speculative-decoding |
| https://github.com/xxyykkxx/llama-cpp-python#built-in-mtp |
| https://github.com/xxyykkxx/llama-cpp-python#external-mtp-model |
| https://github.com/xxyykkxx/llama-cpp-python#dflash-dflash2-and-dspark-speculative-decoding |
| https://github.com/xxyykkxx/llama-cpp-python#n-gram-speculative-decoding |
| https://github.com/xxyykkxx/llama-cpp-python#runtime-statistics |
| https://github.com/xxyykkxx/llama-cpp-python#notes-and-limitations |
| state reuse guide | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/features/caching.md |
| https://github.com/xxyykkxx/llama-cpp-python#multi-modal-models |
| Text-to-Speech | https://github.com/xxyykkxx/llama-cpp-python#speech-synthesis-with-mtmd-text-to-speech |
| llava-v1.5-7b | https://huggingface.co/mys/ggml_llava-v1.5-7b |
| llava-v1.6-34b | https://huggingface.co/cjpais/llava-v1.6-34B-gguf |
| moondream2 | https://huggingface.co/vikhyatk/moondream2 |
| nanollava | https://huggingface.co/abetlen/nanollava-gguf |
| llama-3-vision-alpha | https://huggingface.co/abetlen/llama-3-vision-alpha-gguf |
| minicpm-v-2.6 | https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf |
| minicpm-v-4.5 | https://huggingface.co/openbmb/MiniCPM-V-4_5-gguf |
| minicpm-v-4.6 | https://huggingface.co/openbmb/MiniCPM-V-4.6-gguf |
| gemma3 | https://huggingface.co/unsloth/gemma-3-27b-it-GGUF |
| gemma4 | https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF |
| glm4.1v | https://huggingface.co/unsloth/GLM-4.1V-9B-Thinking-GGUF |
| glm4.6v | https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF |
| granite-docling | https://huggingface.co/ibm-granite/granite-docling-258M-GGUF |
| lfm2-vl | https://huggingface.co/LiquidAI/LFM2-VL-3B-GGUF |
| lfm2.5-vl | https://huggingface.co/LiquidAI/LFM2.5-VL-1.6B-GGUF |
| deepseek-ocr | https://huggingface.co/JamePeng2023/DeepSeek-OCR-2-GGUF |
| mineru2.5-pro | https://huggingface.co/JamePeng2023/MinerU2.5-Pro-2605-1.2B-GGUF |
| paddleocr-vl-1.5 | https://huggingface.co/JamePeng2023/PaddleOCR-VL-1.5-GGUF |
| qwen2.5-vl | https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-GGUF |
| qwen3-asr | https://huggingface.co/JamePeng2023/Qwen3-ASR-1.7B-GGUF |
| qwen3-vl | https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-GGUF |
| qwen3.5 | https://huggingface.co/unsloth/Qwen3.5-27B-GGUF |
| qwen3.6 | https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF |
| qwen3.8 | https://huggingface.co/unsloth/Qwen3.8-27B-GGUF |
| step3-vl | https://huggingface.co/JamePeng2023/Step3-VL-10B-GGUF |
| https://github.com/xxyykkxx/llama-cpp-python#generic-mtmd-chat-handler-recommended |
| https://github.com/xxyykkxx/llama-cpp-python#multimodal-models-with-rpc |
| https://github.com/xxyykkxx/llama-cpp-python#chat-template-resolution-order |
| https://github.com/xxyykkxx/llama-cpp-python#passing-extra-template-arguments |
| https://github.com/xxyykkxx/llama-cpp-python#loading-a-local-video-with-generic-mtmd |
| official FFmpeg download page | https://ffmpeg.org/download.html |
| examples/high_level_api/mtmd_video_chat.py | https://github.com/xxyykkxx/llama-cpp-python/blob/main/examples/high_level_api/mtmd_video_chat.py |
| https://github.com/xxyykkxx/llama-cpp-python#loading-a-local-image-with-qwen3vlthinkinginstruct |
| https://github.com/xxyykkxx/llama-cpp-python#speech-recognition-with-qwen3-asr-speech-to-text |
| https://github.com/xxyykkxx/llama-cpp-python#how-it-works |
| https://github.com/xxyykkxx/llama-cpp-python#speech-synthesis-with-mtmd-text-to-speech |
| Qwen3-TTS-12Hz-Base-GGUF | https://huggingface.co/JamePeng2023/Qwen3-TTS-12Hz-Base-GGUF |
| TTS API guide and limitations | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki/examples/audio/audio-tts.md |
| CLI examples: reference voices, multilingual and batch synthesis | https://github.com/xxyykkxx/llama-cpp-python/blob/main/examples/high_level_api/mtmd_tts.py |
| Streamlit playground: upload, recording, playback and downloads | https://github.com/xxyykkxx/llama-cpp-python/blob/main/examples/streamlit_tts/README.md |
| https://github.com/xxyykkxx/llama-cpp-python#comprehensive-omni-multimodal-example-gemma-4-vision--audio--video--text |
| https://github.com/xxyykkxx/llama-cpp-python#embeddings--reranking-gguf |
| https://github.com/xxyykkxx/llama-cpp-python#key-features |
| https://github.com/xxyykkxx/llama-cpp-python#support-embeddings--rerank-model |
| bge-m3-GGUF | https://huggingface.co/gpustack/bge-m3-GGUF |
| jina-embeddings-v2-base-zh-GGUF | https://huggingface.co/gpustack/jina-embeddings-v2-base-zh-GGUF |
| jina-embeddings-v3-GGUF | https://huggingface.co/second-state/jina-embeddings-v3-GGUF |
| bge-reranker-v2-m3-GGUF | https://huggingface.co/gpustack/bge-reranker-v2-m3-GGUF |
| Qwen3-Reranker-GGUF | https://huggingface.co/JamePeng2023/Qwen3-Reranker-GGUF |
| https://github.com/xxyykkxx/llama-cpp-python#todojamepeng-needs-more-extensive-testing-with-various-embedding-and-rerank-models- |
| https://github.com/xxyykkxx/llama-cpp-python#1-text-embeddings-vector-search |
| https://github.com/xxyykkxx/llama-cpp-python#2-reranking-cross-encoder-scoring |
| https://github.com/xxyykkxx/llama-cpp-python#3-normalization |
| https://github.com/xxyykkxx/llama-cpp-python#using-the-standard-llama-class |
| https://github.com/xxyykkxx/llama-cpp-python#docker-image |
| https://github.com/JamePeng/llama-cpp-python/tree/main/docker#cuda_simple | https://github.com/JamePeng/llama-cpp-python/tree/main/docker#cuda_simple |
| https://github.com/xxyykkxx/llama-cpp-python#openai-compatible-web-server-deprecated |
| http://localhost:8000/docs | http://localhost:8000/docs |
| llama_cpp/llama_chat_format.py | https://github.com/xxyykkxx/llama-cpp-python/blob/main/llama_cpp/llama_chat_format.py |
| https://github.com/xxyykkxx/llama-cpp-python#web-server-features |
| Local Copilot replacement | https://llama-cpp-python.readthedocs.io/en/latest/server/#code-completion |
| Function Calling support | https://llama-cpp-python.readthedocs.io/en/latest/server/#function-calling |
| Vision API support | https://llama-cpp-python.readthedocs.io/en/latest/server/#multimodal-models |
| Multiple Models | https://llama-cpp-python.readthedocs.io/en/latest/server/#configuration-and-multi-model-support |
| https://github.com/xxyykkxx/llama-cpp-python#documentation |
| docs/wiki | https://github.com/xxyykkxx/llama-cpp-python/blob/main/docs/wiki |
| project wiki | https://github.com/JamePeng/llama-cpp-python/wiki |
| https://github.com/xxyykkxx/llama-cpp-python#development |
| tests/README.md | https://github.com/xxyykkxx/llama-cpp-python/blob/main/tests/README.md |
| https://github.com/xxyykkxx/llama-cpp-python#faq |
| https://github.com/xxyykkxx/llama-cpp-python#are-there-pre-built-binaries--binary-wheels-available |
| https://github.com/xxyykkxx/llama-cpp-python#how-does-this-compare-to-other-python-bindings-of-llamacpp |
| https://github.com/xxyykkxx/llama-cpp-python#oserror-libcudartsoxxcudart64_xxdll-cannot-open-shared-object-file-no-such-file-or-directory |
| https://github.com/xxyykkxx/llama-cpp-python#filenotfounderror-could-not-find-module-like-ggmldll-ggml-cpudll-ggml-cudadll |
| https://github.com/xxyykkxx/llama-cpp-python#why-are-libraries-compiled-by-other-authors-only-around-100mb-while-your-pre-compiled-versions-are-300mb-or-larger |
| https://github.com/xxyykkxx/llama-cpp-python#quick-tips-for-developuser-continuously-updated |
| https://github.com/xxyykkxx/llama-cpp-python#any-suggestions-contributions-and-modifications-to-this-package-will-be-directed-toward-building-a-user-friendly-efficient-and-secure-python-library |
| https://github.com/xxyykkxx/llama-cpp-python#license |
| github.com/JamePeng/llama-cpp-python/wiki | https://github.com/JamePeng/llama-cpp-python/wiki |
| Readme | https://github.com/xxyykkxx/llama-cpp-python#readme-ov-file |
| License | https://github.com/xxyykkxx/llama-cpp-python#License-1-ov-file |
| Activity | https://github.com/xxyykkxx/llama-cpp-python/activity |
| 0 forks | https://github.com/xxyykkxx/llama-cpp-python/forks |
| Report repository | https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Fxxyykkxx%2Fllama-cpp-python&report=xxyykkxx+%28user%29 |
|
| https://github.com |
| Terms | https://docs.github.com/site-policy/github-terms/github-terms-of-service |
| Privacy | https://docs.github.com/site-policy/privacy-policies/github-privacy-statement |
| Security | https://github.com/security |
| Status | https://www.githubstatus.com/ |
| Community | https://github.community/ |
| Docs | https://docs.github.com/ |
| Contact | https://support.github.com?tags=dotcom-footer |