What is LMCache?
LMCache is an open-source KV cache management layer for scalable LLM inference. LMCache turns cache data from temporary inference state into reusable information that can be stored persistently and shared across serving engines. The project serves developers, researchers, and infrastructure teams working on language model serving, particularly long-context agents, multi-turn conversations, and knowledge-augmented workloads such as RAG.
LMCache separates cache management from the inference engine and provides interfaces for storage, transport, monitoring, and transformation. The repository describes LMCache as vendor-neutral, allowing users to change serving engines or storage vendors while retaining reusable KV caches. The codebase is licensed under Apache License 2.0.
What LMCache can do
LMCache focuses on reducing repeated prefill computation by moving KV caches beyond GPU memory and making those caches reusable. LMCache also provides observability and extension interfaces for production operations and research. The documented capabilities cover persistent storage, cache reuse beyond prompt prefixes, transfers between inference workers, and custom transformations of cached data.
- LMCache offloads KV caches to CPU memory, local disk, and remote storage backends.
- LMCache reuses cached data across requests, sessions, and engine instances.
- LMCache reports health, performance, prefix-hit, cache lifecycle, and user-specific usage metrics.
- LMCache uses CacheBlend for non-prefix reuse with selective token recomputation.
- LMCache transfers KV caches between prefill and decode workers over NVLink, RDMA, or TCP.
- LMCache exposes a SERDE interface for compression, token dropping, and custom serialization.
How LMCache works
LMCache runs as a standalone daemon process that manages KV caches independently from the inference engine process. This separation means cached state is not lost solely because the inference engine crashes. LMCache moves caches into a tiered storage hierarchy and retrieves reusable data across requests, sessions, and engine instances instead of repeating the same prefill computation.
LMCache uses unified interfaces for pluggable storage and KV transfer backends. The repository lists CPU RAM, local SSD storage, Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, and GDS among supported backends. LMCache also documents multimodal model support in vLLM and an integration with NVIDIA Dynamo.
Who LMCache is for
LMCache is intended for developers, researchers, and industry teams building efficient LLM inference systems. LMCache is particularly relevant when workloads repeatedly use context, when caches need to survive engine restarts, or when infrastructure needs to move cached data between storage tiers and serving components. Researchers can also use the transformation interface to explore cache compression and serialization.
- LMCache supports cache reuse for long-context agentic workloads.
- LMCache reduces repeated prefill work in multi-turn conversation workloads.
- LMCache supports knowledge-augmented workloads such as RAG.
- LMCache enables KV cache transfers between separate prefill and decode workers.
LMCache licensing and access
LMCache is available as an independent open-source project, with its codebase licensed under Apache License 2.0. LMCache can be obtained through the public repository or installed using pip install lmcache. The repository provides links to an installation guide, quickstart, recipes, CLI reference, benchmarking guide, production deployment documentation, and FAQ.
LMCache strengths and setup considerations
LMCache combines persistent cache reuse, engine-independent operation, and a choice of storage and transport backends. These capabilities support cache management across different serving environments without tying stored caches to an individual engine process. LMCache requires installation and setup within an inference environment; the repository directs users to setup examples and production deployment guidance rather than presenting a standalone consumer application.
LMCache FAQ
What is LMCache?
LMCache is an open-source KV cache management layer for LLM inference. LMCache persistently stores and reuses caches across requests, sessions, and serving engines to reduce repeated prefill computation, lower time-to-first-token, and improve throughput.
Is LMCache open source?
LMCache is an independent open-source project licensed under Apache License 2.0. LMCache is available through its public repository and can be installed using pip install lmcache, with installation and deployment documentation linked from the repository.
Which storage backends does LMCache support?
LMCache supports CPU RAM, local disk storage, Redis/Valkey, Mooncake, InfiniStore, S3-compatible object storage, NIXL, and GDS. LMCache provides a unified interface for integrating remote storage and KV transfer backends for cache offloading and sharing.
Who is LMCache for?
LMCache serves developers, researchers, and teams building LLM inference systems. LMCache targets long-context agentic workloads, multi-turn conversations, and knowledge-augmented workloads such as RAG, where cache reuse can reduce repeated computation.
Does LMCache support non-prefix cache reuse?
LMCache supports reusing cached KV blocks at any position in a prompt, rather than only matching prefixes. LMCache uses CacheBlend to selectively recompute tokens for quality recovery when applying this non-prefix reuse.
How do you install LMCache?
LMCache can be installed with pip install lmcache. LMCache also provides links to installation instructions, a quickstart, recipes, a CLI reference, a benchmarking guide, and production deployment documentation for further setup.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




