What is llama-swap?
llama-swap is an open-source project that runs multiple generative AI models on a local machine and switches between them on demand. llama-swap works with OpenAI and Anthropic API-compatible servers, providing model switching for local AI workflows. The project is written in Go and distributed under the MIT license.
llama-swap supports inference servers including llama.cpp and its forks, vllm, stable-diffusion.cpp, audio.cpp and ComfyUI. The application is distributed as a standalone binary with a configuration file and no external dependencies. A built-in web interface adds testing, model controls and monitoring alongside the API endpoints.
What llama-swap can do
llama-swap combines model routing with controls for loading, unloading and observing local inference servers. Supported API endpoints cover text generation, embeddings, image generation and editing, speech and transcription. Additional endpoints expose model status, logs, health checks and system metrics, while configuration options control routing and request parameters.
- llama-swap switches local models on demand.
- llama-swap supports OpenAI and Anthropic API endpoints.
- llama-swap provides a browser playground with token metrics and request inspection.
- llama-swap changes model routing at runtime through profiles.
- llama-swap supports concurrent models through a custom swap matrix.
- llama-swap unloads models automatically after a configured timeout.
- llama-swap restricts API access through configured keys.
- llama-swap exposes system and GPU metrics for Prometheus.
How llama-swap works
llama-swap uses a binary and configuration file to manage local model servers. Users can configure runtime routing profiles, startup hooks for preloading models, and filters that strip or set inference parameters. Docker and Podman support uses the cmd and cmdStop configuration options to control server processes.
llama-swap exposes a web interface at /ui for testing models, inspecting requests and responses, viewing token metrics and streaming logs. API endpoints also allow users to list running models, unload models individually or together, and pause inference requests until service is resumed. Direct access to upstream servers is available through a model-specific route.
Who llama-swap is for
llama-swap serves developers and users who run generative AI models locally and need to switch between compatible servers. The documented capabilities cover language, image and audio workflows rather than text generation alone. The playground and monitoring features also support people testing model behavior and inspecting local inference activity.
- llama-swap supports local chat, completion and embedding workflows.
- llama-swap routes image generation and editing requests to supported servers.
- llama-swap supports speech and transcription endpoints.
- llama-swap helps users inspect model requests, responses and logs.
llama-swap pricing and plans
llama-swap is available as an MIT-licensed open-source project rather than a documented subscription service. The repository provides installation paths through Docker, Homebrew, MacPorts, WinGet, release binaries and source builds. Release binaries are available for Linux, Mac, Windows and FreeBSD; MacPorts and WinGet packages are maintained by community contributors.
llama-swap offers unified container images that include llama-server, ik-llama-server, stable-diffusion.cpp, whisper.cpp and audio.cpp alongside llama-swap. The repository recommends these unified images and provides CUDA and Vulkan options. A legacy image instead adds llama-swap to the llama.cpp server container and includes only the components supplied by that base image.
Strengths and limitations of llama-swap
llama-swap combines broad local-server compatibility with a standalone application, a web playground and operational controls. Runtime profiles, automatic unloading, request filters and live logs provide configurable model management. Deployment still involves selecting compatible inference servers and supplying configuration, while source builds and container choices introduce specific requirements and differences.
- llama-swap requires Go and Node.js when building from source.
- llama-swap legacy containers omit image generation, speech components and LLAMA_SWAP_* environment variable support.
- llama-swap requires a model query parameter for the llama-server /props endpoint and ignores its autoload parameter.
llama-swap FAQ
What is llama-swap?
llama-swap is an open-source tool for running multiple generative AI models locally and switching between them on demand. llama-swap works with OpenAI and Anthropic API-compatible servers and includes a web interface for testing models, inspecting requests and viewing logs.
Is llama-swap free?
llama-swap is available as an MIT-licensed open-source project. The repository provides source code, release binaries, container images and package-manager installation options, rather than describing paid plans or a hosted subscription service.
Which local AI servers does llama-swap support?
llama-swap supports local servers including llama.cpp and its forks, vllm, stable-diffusion.cpp, audio.cpp and ComfyUI. llama-swap also works with OpenAI and Anthropic API-compatible servers, with documented endpoints for text, embeddings, images, audio and other supported operations.
How do you install llama-swap?
llama-swap can be installed through Docker, Homebrew, MacPorts, WinGet, release binaries or source builds. Release binaries cover Linux, Mac, Windows and FreeBSD. Building from source requires Go and Node.js, while MacPorts and WinGet installations are maintained by community contributors.
Does llama-swap have a web interface?
llama-swap includes a real-time web interface with a playground for testing local models. The interface provides detailed token metrics, request and response inspection, manual model loading and unloading, and live log streaming.
Can llama-swap run models concurrently?
llama-swap supports concurrent models through a configurable custom DSL swap matrix. llama-swap also provides runtime routing profiles, automatic unloading after a configured timeout, startup hooks for preloading models and API endpoints for manually unloading models.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




