René's URL Explorer Experiment

{"@context":"https://schema.org","@type":"DiscussionForumPosting","headline":"merge API server example / module","articleBody":"Currently there exist two server implementations:\r\n\r\n- `llama_cpp/server/__main__.py`, the module that's runnable by consumers of the library with `python3 -m llama_cpp.server`\r\n- `examples/high_level_api/fastapi_server.py`, which is probably a copy-pasted example by folks hacking around\r\n\r\nIMO this is confusing. As a new user of the library I see they've both been updated relatively recently but looking side-by-side there's a diff.\r\n\r\nThe one in the module seems better:\r\n- supports logits_all\r\n- supports use_mmap\r\n- has experimental cache support (with some mutex thing going on)\r\n- some stuff with streaming support was moved around more recently than fastapi_server.py\r\n\r\nSo IMO the example server should go away (perhaps just import the module's server and run it after #1 is done)\r\n\r\n\r\n\r\n","author":{"url":"https://github.com/Stonelinks","@type":"Person","name":"Stonelinks"},"datePublished":"2023-04-29T05:06:29.000Z","interactionStatistic":{"@type":"InteractionCounter","interactionType":"https://schema.org/CommentAction","userInteractionCount":0},"url":"https://github.com/2/llama-cpp-python/issues/2"}