What is LiteLLM?
LiteLLM is an open-source AI gateway that gives developers and organizations a unified interface for calling 100+ LLM providers in OpenAI format. LiteLLM can run as a Python library inside an application or as a self-hosted Proxy Server that centralizes model access for a team.
LiteLLM addresses differences between provider SDKs, authentication patterns, request formats and error types. Supported providers named in the repository include OpenAI, Anthropic, Gemini, Bedrock, Azure, VertexAI, vLLM and Nvidia NIM. The repository describes a Rust core alongside the Python SDK.
What LiteLLM can do
LiteLLM combines provider access with gateway controls for operating model-backed applications. LiteLLM supports multiple endpoint types, including chat completions, responses, embeddings, images, audio, batches and reranking. The SDK and gateway also expose automatic routing, MCP tool connections and A2A agent invocation.
- LiteLLM calls LLM providers through a unified OpenAI-compatible interface.
- LiteLLM manages virtual keys and tracks model spending.
- LiteLLM provides guardrails, load balancing, logging and an admin dashboard.
- LiteLLM routes prompts to models using configured complexity tiers.
- LiteLLM connects MCP servers and loads tools in OpenAI format.
- LiteLLM invokes A2A agents from supported providers.
How LiteLLM works
LiteLLM offers a direct SDK workflow and a centralized gateway workflow. For SDK integration, the repository shows installing litellm, setting provider API keys and calling the completion function with a provider-qualified model name. For gateway use, developers install the proxy package, start a model endpoint and point an OpenAI client at the proxy URL.
LiteLLM also documents connecting MCP servers to the gateway and calling their tools through chat completions. A Cursor IDE configuration example uses the gateway's MCP endpoint. For A2A access, developers add an agent to the gateway and call that agent through an A2A SDK using a master key or virtual key.
Who LiteLLM is for
LiteLLM is for developers integrating models into applications and teams that want shared infrastructure for model access. The Python SDK supports direct library integration, while the gateway provides an organizational service with access controls and spending visibility. MCP and A2A support extend those workflows to tools and agents.
- Developers can switch model providers without rewriting OpenAI-compatible application code.
- Teams can centralize access through a shared Proxy Server.
- Organizations can track spending and issue virtual keys.
- Agent developers can connect MCP tools and invoke A2A agents.
LiteLLM pricing and access options
LiteLLM is available as an open-source, self-hosted project. The repository also links to a Hosted Proxy and an Enterprise Tier, but the supplied text does not specify subscription prices or plan entitlements. Developers can obtain the SDK or proxy through the installation commands documented in the repository.
LiteLLM strengths and limitations
LiteLLM brings provider compatibility, routing, spending visibility and tool connectivity into a shared SDK and gateway offering. The choice between library integration and centralized deployment supports different application architectures. Setup remains technical: the documented workflows require package installation, provider credentials and configuration of models, gateway clients or connected services.
- LiteLLM requires separate environments for litellm and litellm-core because their Python files overlap.
- LiteLLM's litellm-core distribution omits optional extras, CLI entry points and the bundled dashboard.
- LiteLLM's core distribution build requires Git, uv and the Rust build toolchain.
- LiteLLM MCP connections may require provider-approved, pre-registered OAuth application credentials.
LiteLLM FAQ
What is LiteLLM?
LiteLLM is an open-source AI gateway and Python SDK that provides a unified OpenAI-compatible interface for 100+ LLM providers. LiteLLM supports direct application integration or deployment as a centralized, self-hosted Proxy Server for teams and organizations.
How can developers access LiteLLM?
LiteLLM is available as a self-hosted open-source project with documented installation paths for the Python SDK and Proxy Server. LiteLLM also links to a Hosted Proxy and an Enterprise Tier, although the supplied repository text does not provide prices.
Which LLM providers does LiteLLM support?
LiteLLM supports 100+ LLM providers through a unified interface. Providers named in the repository include OpenAI, Anthropic, Gemini, Bedrock, Azure, VertexAI, vLLM and Nvidia NIM. LiteLLM uses OpenAI-compatible requests so developers can change providers without rewriting their application code.
Does LiteLLM support MCP tools and A2A agents?
LiteLLM supports MCP server connections and A2A agent invocation through the Python SDK and AI Gateway. LiteLLM can load MCP tools in OpenAI format, expose gateway tools through chat completions and connect Cursor IDE to the gateway's MCP endpoint.
What is the difference between litellm and litellm-core?
LiteLLM provides a separate litellm-core distribution with the same import API and runtime dependencies, but without optional extras, CLI entry points or the bundled dashboard. LiteLLM requires installing only one SDK distribution per environment because litellm and litellm-core own overlapping Python files.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




