What is Switchyard?
Switchyard is an open-source model-routing library for AI agents and LLM applications. Switchyard selects which model handles a request, combining efficient models with more capable models according to the configured routing strategy. The project is published in the NVIDIA-NeMo GitHub repository and supports native OpenAI and Anthropic API compatibility.
Switchyard separates routing decisions from the service that executes model calls. Developers can embed the Rust core, libsy, in a gateway or agent harness while retaining ownership of model clients, credentials, and service operation. This design supports evaluating task success, cost, and latency across a chosen model pool.
What Switchyard can do
Switchyard provides routing choices based on the request, task assessment, and an agent's recent activity. Some strategies call a model to judge the task, while execution-based routing uses tool activity and outcome signals. Routing decisions can change as an agent continues working, rather than remaining fixed for the entire task.
- Switchyard offers an Auto preset using efficient-first Execution routing without a classifier call.
- Switchyard uses Task routing to ask a model whether an efficient model can handle a task.
- Switchyard uses Execution routing to respond to recent tool activity and outcome signals.
- Switchyard combines task classification and execution signals through Composite routing.
- Switchyard provides benchmark configurations and guidance for baseline comparisons.
How Switchyard works
Switchyard starts with a configured model pool and routing algorithm. The algorithm examines the request or recent tool activity, then a gateway or agent sends the request to the selected model. Embedded integrations supply the algorithm, model pool, request, and state; application-owned clients handle any model calls requested during routing.
Switchyard can be accessed through OpenRouter, integrated through LiteLLM or NeMo Relay, or embedded directly through libsy. OpenRouter hosts its integration, while LiteLLM and NeMo Relay run in the user's own deployment. The repository links Rust quickstarts, Python bindings, TypeScript bindings, and local evaluation instructions.
Who Switchyard is for
Switchyard is intended for developers operating AI agents, gateways, and agent harnesses who need control over model selection. Switchyard also supports local evaluation with a developer's own model pool. The documentation recommends evaluating the complete agent and routing configuration because a cheaper model call does not necessarily produce a cheaper successfully completed task.
- Agent developers can adjust model selection as tools run.
- Gateway developers can embed routing in a service they operate.
- Evaluation teams can compare routed workloads with a single-model baseline.
- Integration developers can test routing through supported gateways and bindings.
Switchyard pricing and access
Switchyard is available as an open-source project under the Apache-2.0 license through its GitHub repository. Access options include embedding libsy, using an available gateway integration, or running the demo server for local evaluation. Switchyard's documentation assigns model clients, credentials, and service operation to the developer rather than supplying those responsibilities through the routing library.
Switchyard strengths and limitations
Switchyard combines configurable routing strategies with an embedded architecture that leaves model execution under application control. Gateway integrations offer additional adoption paths, and evaluation guidance supports comparisons against an existing baseline. However, the repository describes components with different stability levels and advises developers to pin the version they integrate because APIs, configuration, and routing behavior can change.
- Switchyard's standalone server supports demonstrations and local evaluation, not production deployment.
- Switchyard's LiteLLM example is experimental and checkout-only.
- Switchyard's LiteLLM integration cannot service intermediate calls needed by classifier or escalation algorithms.
- Switchyard's embedded core requires developers to manage model calls, credentials, and retries.
Switchyard FAQ
What is Switchyard?
Switchyard is an open-source library that helps AI agents choose which model handles each request. Switchyard routes across models and providers to balance task success, cost, and latency, with native OpenAI and Anthropic API compatibility.
Is Switchyard free?
Switchyard is available as an open-source project under the Apache-2.0 license. Switchyard supplies routing decisions, while developers remain responsible for model clients, credentials, and service operation; the repository does not specify model-provider or gateway pricing.
Which gateways does Switchyard integrate with?
Switchyard documents integrations with OpenRouter, LiteLLM, and NeMo Relay. OpenRouter hosts its integration, while LiteLLM and NeMo Relay run in the user's own deployment. Algorithm support varies by integration, and the LiteLLM example is experimental.
How does Switchyard choose a model?
Switchyard chooses models using a configured routing algorithm and model pool. Task routing uses a model to judge task suitability, Execution routing examines recent tool activity and outcomes, and Composite routing combines task classification with execution signals.
Can Switchyard run as a production proxy?
Switchyard's standalone server is intended only for demonstrations and local evaluation, not supported production deployment. Switchyard instead documents embedding libsy in a developer-operated gateway or agent harness, alongside available gateway integrations with their own deployment requirements.
Does Switchyard guarantee lower costs?
Switchyard does not guarantee that a cheaper model call results in a cheaper successfully completed task. Switchyard recommends evaluating the complete agent, model pool, and routing configuration against a single-model baseline to assess task success, cost, and latency.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




