Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

LiteLLM

Open-source AI gateway and Python SDK for calling LLM providers through a unified OpenAI-compatible interface.

LiteLLM is a self-hosted, open-source AI gateway and Python SDK that connects applications to 100+ LLM providers through an OpenAI-compatible interface. LiteLLM serves developers and organizations needing centralized model access, routing and spend tracking, with open-source access plus linked Hosted Proxy and Enterprise Tier options.

Updated 8 Nov 20266 sections

LiteLLM website screenshot

What is LiteLLM?

LiteLLM is an open-source AI gateway that gives developers and organizations a unified interface for calling 100+ LLM providers in OpenAI format. LiteLLM can run as a Python library inside an application or as a self-hosted Proxy Server that centralizes model access for a team.

LiteLLM addresses differences between provider SDKs, authentication patterns, request formats and error types. Supported providers named in the repository include OpenAI, Anthropic, Gemini, Bedrock, Azure, VertexAI, vLLM and Nvidia NIM. The repository describes a Rust core alongside the Python SDK.

What LiteLLM can do

LiteLLM combines provider access with gateway controls for operating model-backed applications. LiteLLM supports multiple endpoint types, including chat completions, responses, embeddings, images, audio, batches and reranking. The SDK and gateway also expose automatic routing, MCP tool connections and A2A agent invocation.

  • LiteLLM calls LLM providers through a unified OpenAI-compatible interface.
  • LiteLLM manages virtual keys and tracks model spending.
  • LiteLLM provides guardrails, load balancing, logging and an admin dashboard.
  • LiteLLM routes prompts to models using configured complexity tiers.
  • LiteLLM connects MCP servers and loads tools in OpenAI format.
  • LiteLLM invokes A2A agents from supported providers.

How LiteLLM works

LiteLLM offers a direct SDK workflow and a centralized gateway workflow. For SDK integration, the repository shows installing litellm, setting provider API keys and calling the completion function with a provider-qualified model name. For gateway use, developers install the proxy package, start a model endpoint and point an OpenAI client at the proxy URL.

LiteLLM also documents connecting MCP servers to the gateway and calling their tools through chat completions. A Cursor IDE configuration example uses the gateway's MCP endpoint. For A2A access, developers add an agent to the gateway and call that agent through an A2A SDK using a master key or virtual key.

Who LiteLLM is for

LiteLLM is for developers integrating models into applications and teams that want shared infrastructure for model access. The Python SDK supports direct library integration, while the gateway provides an organizational service with access controls and spending visibility. MCP and A2A support extend those workflows to tools and agents.

  • Developers can switch model providers without rewriting OpenAI-compatible application code.
  • Teams can centralize access through a shared Proxy Server.
  • Organizations can track spending and issue virtual keys.
  • Agent developers can connect MCP tools and invoke A2A agents.

LiteLLM pricing and access options

LiteLLM is available as an open-source, self-hosted project. The repository also links to a Hosted Proxy and an Enterprise Tier, but the supplied text does not specify subscription prices or plan entitlements. Developers can obtain the SDK or proxy through the installation commands documented in the repository.

LiteLLM strengths and limitations

LiteLLM brings provider compatibility, routing, spending visibility and tool connectivity into a shared SDK and gateway offering. The choice between library integration and centralized deployment supports different application architectures. Setup remains technical: the documented workflows require package installation, provider credentials and configuration of models, gateway clients or connected services.

  • LiteLLM requires separate environments for litellm and litellm-core because their Python files overlap.
  • LiteLLM's litellm-core distribution omits optional extras, CLI entry points and the bundled dashboard.
  • LiteLLM's core distribution build requires Git, uv and the Rust build toolchain.
  • LiteLLM MCP connections may require provider-approved, pre-registered OAuth application credentials.

LiteLLM FAQ

What is LiteLLM?

LiteLLM is an open-source AI gateway and Python SDK that provides a unified OpenAI-compatible interface for 100+ LLM providers. LiteLLM supports direct application integration or deployment as a centralized, self-hosted Proxy Server for teams and organizations.

How can developers access LiteLLM?

LiteLLM is available as a self-hosted open-source project with documented installation paths for the Python SDK and Proxy Server. LiteLLM also links to a Hosted Proxy and an Enterprise Tier, although the supplied repository text does not provide prices.

Which LLM providers does LiteLLM support?

LiteLLM supports 100+ LLM providers through a unified interface. Providers named in the repository include OpenAI, Anthropic, Gemini, Bedrock, Azure, VertexAI, vLLM and Nvidia NIM. LiteLLM uses OpenAI-compatible requests so developers can change providers without rewriting their application code.

Does LiteLLM support MCP tools and A2A agents?

LiteLLM supports MCP server connections and A2A agent invocation through the Python SDK and AI Gateway. LiteLLM can load MCP tools in OpenAI format, expose gateway tools through chat completions and connect Cursor IDE to the gateway's MCP endpoint.

What is the difference between litellm and litellm-core?

LiteLLM provides a separate litellm-core distribution with the same import API and runtime dependencies, but without optional extras, CLI entry points or the bundled dashboard. LiteLLM requires installing only one SDK distribution per environment because litellm and litellm-core own overlapping Python files.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like LiteLLM

One API

www.github.com

Open-source LLM API management and key redistribution through a unified OpenAI-compatible interface.

Free

New API

www.github.com

Self-hosted AI gateway for model routing, API access control, usage tracking, and cost accounting.

Free

AxonHub

www.github.com

Open-source AI gateway for model routing, SDK compatibility, request tracing and cost control.

Free

Switchyard

www.github.com

Open-source model routing for AI agents, with Rust embedding and gateway integrations.

Free