René's URL Explorer Experiment


Title: History for llama_cpp/server - chynggi/llama-cpp-python · GitHub

Open Graph Title: History for llama_cpp/server - chynggi/llama-cpp-python

X Title: History for llama_cpp/server - chynggi/llama-cpp-python

Description: Python bindings for llama.cpp. Contribute to chynggi/llama-cpp-python development by creating an account on GitHub.

Open Graph Description: Python bindings for llama.cpp. Contribute to chynggi/llama-cpp-python development by creating an account on GitHub.

X Description: Python bindings for llama.cpp. Contribute to chynggi/llama-cpp-python development by creating an account on GitHub.

Opengraph URL: https://github.com/chynggi/llama-cpp-python

X: @github

direct link

Domain: github.com

react-profiling0
react-import-mapreact,react-dom,react-dom/client,react-dom/profiling,react-is,react/compiler-runtime,react/jsx-dev-runtime,react/jsx-runtime,scheduler@b8488ee2c322
route-pattern/:user_id/:repository/commits(/*name)
route-controllercommits
route-actionshow
fetch-noncev2:1b640beb-ec83-8987-cb07-81095fc38fae
current-catalog-service-hashf3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb
request-idAEEA:354B:F38EDF:13F2BC3:6AFFF700
html-safe-nonce96fbd3bc0fdce7341a4b0cf1ee49346ec92ae945455232dc1a4fc9d640e94c81
visitor-payloadeyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBRUVBOjM1NEI6RjM4RURGOjEzRjJCQzM6NkFGRkY3MDAiLCJ2aXNpdG9yX2lkIjoiMzQ0MjIxNjkyMTQ1MjI0Njc4NCIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9
visitor-hmac38abb5ae0fb93fb5fcb4361fa6c3fdd013a7e8d28b9e70896ff55868e911769b
hovercard-subject-tagrepository:1349320893
github-keyboard-shortcutsrepository,commit-list,copilot
google-site-verificationApib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I
octolytics-urlhttps://collector.github.com/github/collect
analytics-location///commits/show
fb:app_id1401488693436528
apple-itunes-appapp-id=1477376905, app-argument=https://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server
twitter:imagehttps://opengraph.githubassets.com/d78b9d765f008d8f3ad9cb1831dd0d37c08fa3a99a917c0ea009edda52400c10/chynggi/llama-cpp-python
twitter:cardsummary_large_image
og:imagehttps://opengraph.githubassets.com/d78b9d765f008d8f3ad9cb1831dd0d37c08fa3a99a917c0ea009edda52400c10/chynggi/llama-cpp-python
og:image:altPython bindings for llama.cpp. Contribute to chynggi/llama-cpp-python development by creating an account on GitHub.
og:image:width1200
og:image:height600
og:site_nameGitHub
og:typeobject
hostnamegithub.com
expected-hostnamegithub.com
None0a0d8686a27478df474dd7415776a7a080cde8838b3cd92ea7601187572e5729
turbo-cache-controlno-cache
go-importgithub.com/chynggi/llama-cpp-python git https://github.com/chynggi/llama-cpp-python.git
octolytics-dimension-user_id47375675
octolytics-dimension-user_loginchynggi
octolytics-dimension-repository_id1349320893
octolytics-dimension-repository_nwochynggi/llama-cpp-python
octolytics-dimension-repository_publictrue
octolytics-dimension-repository_is_forktrue
octolytics-dimension-repository_parent_id617868717
octolytics-dimension-repository_parent_nwoabetlen/llama-cpp-python
octolytics-dimension-repository_network_root_id617868717
octolytics-dimension-repository_network_root_nwoabetlen/llama-cpp-python
turbo-body-classeslogged-out env-production page-responsive
disable-turbofalse
browser-stats-urlhttps://api.github.com/_private/browser/stats
browser-errors-urlhttps://api.github.com/_private/browser/errors
release16a11879678aaf97e2a42c33ee70830aa084fa37
ui-targetfull
theme-color#1e2327
color-schemelight dark
react-app-namecommits

Links:

Skip to contenthttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server#start-of-content
https://github.com/
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fchynggi%2Fllama-cpp-python%2Fcommits%2Fmain%2Fllama_cpp%2Fserver
GitHubRepository hosting, code review, and collaborationhttps://github.com/platform
CopilotAI coding agent for tasks, projects, and automated workhttps://github.com/features/copilot
ActionsCI/CD and automationhttps://github.com/features/actions
Code QualityQuality analysis, fixes, and enforcementhttps://github.com/features/code-quality
CodespacesCloud development environmentshttps://github.com/features/codespaces
Code SecurityVulnerability detection and remediationhttps://github.com/security/advanced-security/code-security
Secret ProtectionSecret scanning and leak preventionhttps://github.com/security/advanced-security/secret-protection
All Featureshttps://github.com/features
Documentationhttps://docs.github.com/en
Traininghttps://learn.github.com/skills
Certificationshttps://learn.github.com/credentials
Build with Copilothttps://github.com/features/ai
Customize Copilothttps://docs.github.com/en/copilot/how-tos/copilot-on-github/customize-copilot/customize-copilot-overview
Integrationshttps://github.com/marketplace
Trending reposhttps://github.com/trending
GitHub Sponsorshttps://github.com/open-source/sponsors
Developer articleshttps://github.com/resources/articles
ChangelogYour source for the latest features, improvements, and fixes across GitHub products, all in one place.https://github.blog/changelog/
Enterpriseshttps://github.com/enterprise
Teamhttps://github.com/team
Startupshttps://github.com/enterprise/startups
Nonprofitshttps://github.com/solutions/industry/nonprofits
Healthcarehttps://github.com/solutions/industry/healthcare
Financial serviceshttps://github.com/solutions/industry/financial-services
Manufacturinghttps://github.com/solutions/industry/manufacturing
Governmenthttps://github.com/solutions/industry/government
See all industrieshttps://github.com/solutions/industry
Why GitHubhttps://github.com/why-github
Customer Storieshttps://github.com/customer-stories
Business insightshttps://github.com/solutions/executive-insights
Trust Centerhttps://github.com/trust-center
GitHub Advanced SecurityEnterprise-grade code securityhttps://github.com/security/advanced-security
Copilot for businessEnterprise-grade AI featureshttps://github.com/features/copilot/copilot-business
Premium SupportEnterprise-grade 24/7 supporthttps://github.com/enterprise/premium-support
Documentationhttps://docs.github.com/en
Resource centerhttps://github.com/resources
Bloghttps://github.blog/
Events & webinarshttps://github.com/resources/events
Community Forumhttps://github.com/orgs/community/discussions
Customer Storieshttps://github.com/customer-stories
Integrationshttps://github.com/marketplace
Customer supporthttps://help.github.com/
Traininghttps://learn.github.com/skills
GitHub Statushttps://www.githubstatus.com/
Abouthttps://github.com/about
Partnershttps://github.com/partners
Changeloghttps://github.blog/changelog/
GitHub UniverseThe event for developers, by developers — talks, workshops, and what’s next for GitHub.https://githubuniverse.com/
Pricinghttps://github.com/pricing
Sign inhttps://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Fchynggi%2Fllama-cpp-python%2Fcommits%2Fmain%2Fllama_cpp%2Fserver
Sign uphttps://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E%2Fcommits%2Fshow&source=header-repo&source_repo=chynggi%2Fllama-cpp-python
Reloadhttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server
Reloadhttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server
Reloadhttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server
chynggihttps://github.com/chynggi
llama-cpp-pythonhttps://github.com/chynggi/llama-cpp-python
Codehttps://github.com/chynggi/llama-cpp-python
Pull requestshttps://github.com/chynggi/llama-cpp-python/pulls
Actionshttps://github.com/chynggi/llama-cpp-python/actions
Projectshttps://github.com/chynggi/llama-cpp-python/projects
Security and qualityhttps://github.com/chynggi/llama-cpp-python/security
Insightshttps://github.com/chynggi/llama-cpp-python/pulse
https://github.com/chynggi/llama-cpp-python
llama-cpp-pythonhttps://github.com/chynggi/llama-cpp-python/commits
llama_cpphttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp
serverhttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server
mainhttps://github.com/chynggi/llama-cpp-python/tree/main
fix(server): show falsey defaults in CLI help (#2355)https://github.com/chynggi/llama-cpp-python/commit/d8a4a1e4835b36974154393c44dc852dae357d84
d8a4a1ehttps://github.com/chynggi/llama-cpp-python/commit/d8a4a1e4835b36974154393c44dc852dae357d84
https://github.com/chynggi/llama-cpp-python/blob/d8a4a1e4835b36974154393c44dc852dae357d84/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/d8a4a1e4835b36974154393c44dc852dae357d84
feat: Generic Multimodal Chat Handler (#2256)https://github.com/chynggi/llama-cpp-python/commit/2dae4770db6a2c34bf3a5f7f76b4d4bc8045bfd0
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
2dae477https://github.com/chynggi/llama-cpp-python/commit/2dae4770db6a2c34bf3a5f7f76b4d4bc8045bfd0
https://github.com/chynggi/llama-cpp-python/blob/2dae4770db6a2c34bf3a5f7f76b4d4bc8045bfd0/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/2dae4770db6a2c34bf3a5f7f76b4d4bc8045bfd0
feat: add Gemma 4 multimodal chat support (#2241)https://github.com/chynggi/llama-cpp-python/commit/a9b480f8b1ac20279a563d12c5d0f3567c0af960
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
a9b480fhttps://github.com/chynggi/llama-cpp-python/commit/a9b480f8b1ac20279a563d12c5d0f3567c0af960
https://github.com/chynggi/llama-cpp-python/blob/a9b480f8b1ac20279a563d12c5d0f3567c0af960/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/a9b480f8b1ac20279a563d12c5d0f3567c0af960
server types: Move 'model' parameter to clarify it is used (#1786)https://github.com/chynggi/llama-cpp-python/commit/c3adb354e71ec964dd503a5aae81d16ca947e377
https://github.com/domdomegg
domdomegghttps://github.com/chynggi/llama-cpp-python/commits?author=domdomegg
c3adb35https://github.com/chynggi/llama-cpp-python/commit/c3adb354e71ec964dd503a5aae81d16ca947e377
https://github.com/chynggi/llama-cpp-python/blob/c3adb354e71ec964dd503a5aae81d16ca947e377/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/c3adb354e71ec964dd503a5aae81d16ca947e377
fix: correct typo in comments and settings description (#2121)https://github.com/chynggi/llama-cpp-python/commit/33bf9d2480351d28d4a5e21626bf29f35a2e9838
https://github.com/thecaptain789
thecaptain789https://github.com/chynggi/llama-cpp-python/commits?author=thecaptain789
33bf9d2https://github.com/chynggi/llama-cpp-python/commit/33bf9d2480351d28d4a5e21626bf29f35a2e9838
https://github.com/chynggi/llama-cpp-python/blob/33bf9d2480351d28d4a5e21626bf29f35a2e9838/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/33bf9d2480351d28d4a5e21626bf29f35a2e9838
feat(server): add model-load chat_template_kwargs (#2168)https://github.com/chynggi/llama-cpp-python/commit/7257ba95fbbf65201fd5bf4b7f0bdd1c701e1345
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
7257ba9https://github.com/chynggi/llama-cpp-python/commit/7257ba95fbbf65201fd5bf4b7f0bdd1c701e1345
https://github.com/chynggi/llama-cpp-python/blob/7257ba95fbbf65201fd5bf4b7f0bdd1c701e1345/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/7257ba95fbbf65201fd5bf4b7f0bdd1c701e1345
misc: Add Ruff formatting (#2148)https://github.com/chynggi/llama-cpp-python/commit/a9b4a067300c89857334195518e0bb9430d1c059
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
a9b4a06https://github.com/chynggi/llama-cpp-python/commit/a9b4a067300c89857334195518e0bb9430d1c059
https://github.com/chynggi/llama-cpp-python/blob/a9b4a067300c89857334195518e0bb9430d1c059/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/a9b4a067300c89857334195518e0bb9430d1c059
feat: Add support for new mtmd api, add Qwen2.5-VL chat handlerhttps://github.com/chynggi/llama-cpp-python/commit/cd548bd0f14210627798237d5c2ea78acfb88ccb
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
cd548bdhttps://github.com/chynggi/llama-cpp-python/commit/cd548bd0f14210627798237d5c2ea78acfb88ccb
https://github.com/chynggi/llama-cpp-python/blob/cd548bd0f14210627798237d5c2ea78acfb88ccb/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/cd548bd0f14210627798237d5c2ea78acfb88ccb
fix: streaming resource lock (#1879)https://github.com/chynggi/llama-cpp-python/commit/e8f14ceff9b3279c67368d85f9457bb27168d126
gjpowerhttps://github.com/chynggi/llama-cpp-python/commits?author=gjpower
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
e8f14cehttps://github.com/chynggi/llama-cpp-python/commit/e8f14ceff9b3279c67368d85f9457bb27168d126
https://github.com/chynggi/llama-cpp-python/blob/e8f14ceff9b3279c67368d85f9457bb27168d126/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/e8f14ceff9b3279c67368d85f9457bb27168d126
fix: add missing await statements for async exit_stack handling (#1858)https://github.com/chynggi/llama-cpp-python/commit/afedfc888462f9a6e809dc9455eb3b663764cc3f
https://github.com/gjpower
gjpowerhttps://github.com/chynggi/llama-cpp-python/commits?author=gjpower
afedfc8https://github.com/chynggi/llama-cpp-python/commit/afedfc888462f9a6e809dc9455eb3b663764cc3f
https://github.com/chynggi/llama-cpp-python/blob/afedfc888462f9a6e809dc9455eb3b663764cc3f/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/afedfc888462f9a6e809dc9455eb3b663764cc3f
fix: Avoid thread starvation on many concurrent requests by making use of asyncio to lock llama_proxy context (#1798)https://github.com/chynggi/llama-cpp-python/commit/9bd0c95424aad6a80a7305761b465006af3dde0d
gjpowerhttps://github.com/chynggi/llama-cpp-python/commits?author=gjpower
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
9bd0c95https://github.com/chynggi/llama-cpp-python/commit/9bd0c95424aad6a80a7305761b465006af3dde0d
https://github.com/chynggi/llama-cpp-python/blob/9bd0c95424aad6a80a7305761b465006af3dde0d/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/9bd0c95424aad6a80a7305761b465006af3dde0d
fix: added missing exit_stack.close() to /v1/chat/completions (#1796)https://github.com/chynggi/llama-cpp-python/commit/073b7e421e419ec389d8c064bbdae26c482ec17c
https://github.com/Ian321
Ian321https://github.com/chynggi/llama-cpp-python/commits?author=Ian321
073b7e4https://github.com/chynggi/llama-cpp-python/commit/073b7e421e419ec389d8c064bbdae26c482ec17c
https://github.com/chynggi/llama-cpp-python/blob/073b7e421e419ec389d8c064bbdae26c482ec17c/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/073b7e421e419ec389d8c064bbdae26c482ec17c
feat: Add option to configure n_ubatchhttps://github.com/chynggi/llama-cpp-python/commit/6c44a3f36b089239cb6396bb408116aad262c702
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
6c44a3fhttps://github.com/chynggi/llama-cpp-python/commit/6c44a3f36b089239cb6396bb408116aad262c702
https://github.com/chynggi/llama-cpp-python/blob/6c44a3f36b089239cb6396bb408116aad262c702/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/6c44a3f36b089239cb6396bb408116aad262c702
feat: Add server chat_format minicpm-v-2.6 for MiniCPMv26ChatHandlerhttps://github.com/chynggi/llama-cpp-python/commit/97d527eec2b879f38e9eed599f8c7d2ae3a0b23e
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
97d527ehttps://github.com/chynggi/llama-cpp-python/commit/97d527eec2b879f38e9eed599f8c7d2ae3a0b23e
https://github.com/chynggi/llama-cpp-python/blob/97d527eec2b879f38e9eed599f8c7d2ae3a0b23e/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/97d527eec2b879f38e9eed599f8c7d2ae3a0b23e
fix(server): Use split_mode from model settings (#1594)https://github.com/chynggi/llama-cpp-python/commit/66d5cddbc31976636cf560e5b0b86e4333e85f2b
grider-withouraihttps://github.com/chynggi/llama-cpp-python/commits?author=grider-withourai
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
66d5cddhttps://github.com/chynggi/llama-cpp-python/commit/66d5cddbc31976636cf560e5b0b86e4333e85f2b
https://github.com/chynggi/llama-cpp-python/blob/66d5cddbc31976636cf560e5b0b86e4333e85f2b/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/66d5cddbc31976636cf560e5b0b86e4333e85f2b
fix(misc): Formathttps://github.com/chynggi/llama-cpp-python/commit/c1ae81554f52f81b6027e36ae0a056dca9819c3c
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
c1ae815https://github.com/chynggi/llama-cpp-python/commit/c1ae81554f52f81b6027e36ae0a056dca9819c3c
https://github.com/chynggi/llama-cpp-python/blob/c1ae81554f52f81b6027e36ae0a056dca9819c3c/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/c1ae81554f52f81b6027e36ae0a056dca9819c3c
fix(misc): Fix type errorshttps://github.com/chynggi/llama-cpp-python/commit/387d01d94581dbcb1288304094bb1acee5ca24ae
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
387d01dhttps://github.com/chynggi/llama-cpp-python/commit/387d01d94581dbcb1288304094bb1acee5ca24ae
https://github.com/chynggi/llama-cpp-python/blob/387d01d94581dbcb1288304094bb1acee5ca24ae/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/387d01d94581dbcb1288304094bb1acee5ca24ae
fix(server): Fix bug in FastAPI streaming response where dependency was released before request completes causing SEGFAULThttps://github.com/chynggi/llama-cpp-python/commit/296304b60bb83689659883c9cc24f4c074dd88ff
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
296304bhttps://github.com/chynggi/llama-cpp-python/commit/296304b60bb83689659883c9cc24f4c074dd88ff
https://github.com/chynggi/llama-cpp-python/blob/296304b60bb83689659883c9cc24f4c074dd88ff/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/296304b60bb83689659883c9cc24f4c074dd88ff
fix(server): Update `embeddings=False` by default. Embeddings should be enabled by default for embedding models.https://github.com/chynggi/llama-cpp-python/commit/bf5e0bb4b151f4ca2f5a21af68eb832a96a79d75
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
bf5e0bbhttps://github.com/chynggi/llama-cpp-python/commit/bf5e0bb4b151f4ca2f5a21af68eb832a96a79d75
https://github.com/chynggi/llama-cpp-python/blob/bf5e0bb4b151f4ca2f5a21af68eb832a96a79d75/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/bf5e0bb4b151f4ca2f5a21af68eb832a96a79d75
feat: Add `.close()` method to `Llama` class to explicitly free model from memory (#1513)https://github.com/chynggi/llama-cpp-python/commit/320a5d7ea5ebcc5a8dc114d9bdf7c584537b04b0
jkawamotohttps://github.com/chynggi/llama-cpp-python/commits?author=jkawamoto
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
320a5d7https://github.com/chynggi/llama-cpp-python/commit/320a5d7ea5ebcc5a8dc114d9bdf7c584537b04b0
https://github.com/chynggi/llama-cpp-python/blob/320a5d7ea5ebcc5a8dc114d9bdf7c584537b04b0/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/320a5d7ea5ebcc5a8dc114d9bdf7c584537b04b0
feat: adding `rpc_servers` parameter to `Llama` class (#1477)https://github.com/chynggi/llama-cpp-python/commit/d634efcdd91a0f79ae0fac13cfdf46eb47d0602d
chraachttps://github.com/chynggi/llama-cpp-python/commits?author=chraac
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
d634efchttps://github.com/chynggi/llama-cpp-python/commit/d634efcdd91a0f79ae0fac13cfdf46eb47d0602d
https://github.com/chynggi/llama-cpp-python/blob/d634efcdd91a0f79ae0fac13cfdf46eb47d0602d/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/d634efcdd91a0f79ae0fac13cfdf46eb47d0602d
fix: fix string value kv_overrides. Closes #1487https://github.com/chynggi/llama-cpp-python/commit/df45a4b3fe46e72664bda87301b318210c6d4782
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
df45a4bhttps://github.com/chynggi/llama-cpp-python/commit/df45a4b3fe46e72664bda87301b318210c6d4782
https://github.com/chynggi/llama-cpp-python/blob/df45a4b3fe46e72664bda87301b318210c6d4782/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/df45a4b3fe46e72664bda87301b318210c6d4782
feat: add MinTokensLogitProcessor and min_tokens argument to server (#1333)https://github.com/chynggi/llama-cpp-python/commit/5212fb08ae69a721b2ced4e5e8b96ce642219e16
https://github.com/twaka
twakahttps://github.com/chynggi/llama-cpp-python/commits?author=twaka
5212fb0https://github.com/chynggi/llama-cpp-python/commit/5212fb08ae69a721b2ced4e5e8b96ce642219e16
https://github.com/chynggi/llama-cpp-python/blob/5212fb08ae69a721b2ced4e5e8b96ce642219e16/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/5212fb08ae69a721b2ced4e5e8b96ce642219e16
feat(server): Add support for setting root_path. Closes #1420https://github.com/chynggi/llama-cpp-python/commit/0318702cdc860999ee70f277425edbbfe0e60419
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
0318702https://github.com/chynggi/llama-cpp-python/commit/0318702cdc860999ee70f277425edbbfe0e60419
https://github.com/chynggi/llama-cpp-python/blob/0318702cdc860999ee70f277425edbbfe0e60419/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/0318702cdc860999ee70f277425edbbfe0e60419
Merge branch 'main' of github.com:abetlen/llama_cpp_python into mainhttps://github.com/chynggi/llama-cpp-python/commit/f9b7221c8ff42c6220a2b60a3fc37eab6010a048
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
f9b7221https://github.com/chynggi/llama-cpp-python/commit/f9b7221c8ff42c6220a2b60a3fc37eab6010a048
https://github.com/chynggi/llama-cpp-python/blob/f9b7221c8ff42c6220a2b60a3fc37eab6010a048/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/f9b7221c8ff42c6220a2b60a3fc37eab6010a048
feat(server): Remove temperature bounds checks for server. Closes #1384https://github.com/chynggi/llama-cpp-python/commit/0a454bebe67d12a446981eb16028c168ca5faa81
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
0a454behttps://github.com/chynggi/llama-cpp-python/commit/0a454bebe67d12a446981eb16028c168ca5faa81
https://github.com/chynggi/llama-cpp-python/blob/0a454bebe67d12a446981eb16028c168ca5faa81/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/0a454bebe67d12a446981eb16028c168ca5faa81
fix(server): Propagate `flash_attn` to model load. (#1424)https://github.com/chynggi/llama-cpp-python/commit/2138561fab5e60672c63b6c446b62a8bc26e17c4
https://github.com/dthuerck
dthuerckhttps://github.com/chynggi/llama-cpp-python/commits?author=dthuerck
2138561https://github.com/chynggi/llama-cpp-python/commit/2138561fab5e60672c63b6c446b62a8bc26e17c4
https://github.com/chynggi/llama-cpp-python/blob/2138561fab5e60672c63b6c446b62a8bc26e17c4/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/2138561fab5e60672c63b6c446b62a8bc26e17c4
feat: Add llama-3-vision-alpha chat formathttps://github.com/chynggi/llama-cpp-python/commit/31b1d95a6c19f5b615a3286069f181a415f872e8
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
31b1d95https://github.com/chynggi/llama-cpp-python/commit/31b1d95a6c19f5b615a3286069f181a415f872e8
https://github.com/chynggi/llama-cpp-python/blob/31b1d95a6c19f5b615a3286069f181a415f872e8/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/31b1d95a6c19f5b615a3286069f181a415f872e8
feat: Add option to enable `flash_attn` to Lllama params and ModelSettingshttps://github.com/chynggi/llama-cpp-python/commit/22d77eefd2edaf0148f53374d0cac74d0e25d06e
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
22d77eehttps://github.com/chynggi/llama-cpp-python/commit/22d77eefd2edaf0148f53374d0cac74d0e25d06e
https://github.com/chynggi/llama-cpp-python/blob/22d77eefd2edaf0148f53374d0cac74d0e25d06e/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/22d77eefd2edaf0148f53374d0cac74d0e25d06e
feat: Generic Chat Formats, Tool Calling, and Huggingface Pull Support for Multimodal Models (Obsidian, LLaVA1.6, Moondream) (#1147)https://github.com/chynggi/llama-cpp-python/commit/fe2da0953895000e20655d2c991308f702508494
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
fe2da09https://github.com/chynggi/llama-cpp-python/commit/fe2da0953895000e20655d2c991308f702508494
https://github.com/chynggi/llama-cpp-python/blob/fe2da0953895000e20655d2c991308f702508494/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/fe2da0953895000e20655d2c991308f702508494
fix: pydantic deprecation warninghttps://github.com/chynggi/llama-cpp-python/commit/fcfea66857c73c9b95326835debb80dbc0b76f17
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
fcfea66https://github.com/chynggi/llama-cpp-python/commit/fcfea66857c73c9b95326835debb80dbc0b76f17
https://github.com/chynggi/llama-cpp-python/blob/fcfea66857c73c9b95326835debb80dbc0b76f17/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/fcfea66857c73c9b95326835debb80dbc0b76f17
feat(server): Provide ability to dynamically allocate all threads if desired using `-1` (#1364)https://github.com/chynggi/llama-cpp-python/commit/53ebcc8bb5beb912bda4baa44ae27a1e3a7eabd0
https://github.com/sean-bailey
sean-baileyhttps://github.com/chynggi/llama-cpp-python/commits?author=sean-bailey
53ebcc8https://github.com/chynggi/llama-cpp-python/commit/53ebcc8bb5beb912bda4baa44ae27a1e3a7eabd0
https://github.com/chynggi/llama-cpp-python/blob/53ebcc8bb5beb912bda4baa44ae27a1e3a7eabd0/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/53ebcc8bb5beb912bda4baa44ae27a1e3a7eabd0
feat: add `disable_ping_events` flag (#1257)https://github.com/chynggi/llama-cpp-python/commit/b73c73c0c67f559fd9c7d620ad3d4e24d5c4bc4c
https://github.com/khimaros
khimaroshttps://github.com/chynggi/llama-cpp-python/commits?author=khimaros
b73c73chttps://github.com/chynggi/llama-cpp-python/commit/b73c73c0c67f559fd9c7d620ad3d4e24d5c4bc4c
https://github.com/chynggi/llama-cpp-python/blob/b73c73c0c67f559fd9c7d620ad3d4e24d5c4bc4c/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/b73c73c0c67f559fd9c7d620ad3d4e24d5c4bc4c
feat: Use all available CPUs for batch processing (#1345)https://github.com/chynggi/llama-cpp-python/commit/c96b2daebf7b5dbe3cfd9194b249ab579a28d633
ddh0https://github.com/chynggi/llama-cpp-python/commits?author=ddh0
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
c96b2dahttps://github.com/chynggi/llama-cpp-python/commit/c96b2daebf7b5dbe3cfd9194b249ab579a28d633
https://github.com/chynggi/llama-cpp-python/blob/c96b2daebf7b5dbe3cfd9194b249ab579a28d633/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/c96b2daebf7b5dbe3cfd9194b249ab579a28d633
feat: Add support for yaml based configshttps://github.com/chynggi/llama-cpp-python/commit/060bfa64d529ade2af9b1f4e207a3937bbc4138f
https://github.com/abetlen
abetlenhttps://github.com/chynggi/llama-cpp-python/commits?author=abetlen
060bfa6https://github.com/chynggi/llama-cpp-python/commit/060bfa64d529ade2af9b1f4e207a3937bbc4138f
https://github.com/chynggi/llama-cpp-python/blob/060bfa64d529ade2af9b1f4e207a3937bbc4138f/llama_cpp/server
https://github.com/chynggi/llama-cpp-python/tree/060bfa64d529ade2af9b1f4e207a3937bbc4138f
Nexthttps://github.com/chynggi/llama-cpp-python/commits/main/llama_cpp/server?after=3befb793a168eaa328de0df969b2279adda972ee+34
https://github.com
Termshttps://docs.github.com/site-policy/github-terms/github-terms-of-service
Privacyhttps://docs.github.com/site-policy/privacy-policies/github-privacy-statement
Securityhttps://github.com/security
Statushttps://www.githubstatus.com/
Communityhttps://github.community/
Docshttps://docs.github.com/
Contacthttps://support.github.com?tags=dotcom-footer

Viewport: width=device-width


URLs of crawlers that visited me.