Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

guidance

An open-source Python library for controlling language model output with grammars, regex and JSON schemas.

guidance is a Python library for controlling large language models through constrained generation and programmable workflows. guidance helps developers produce structured output using regex, grammars, predefined choices and JSON schemas. guidance is available as an open-source project under the MIT license and can be installed through PyPI.

Updated 9 Nov 20266 sections

guidance website screenshot

What is guidance?

guidance is an open-source Python project for controlling large language models and structuring their generated output. guidance serves developers who want to combine model generation with familiar programming constructs rather than rely on prompting alone. The repository describes a programming paradigm that interleaves generation with conditionals, loops and tool use.

guidance exposes model interactions through a Pythonic interface, including system, user and assistant contexts. Developers can capture generated text by name and retrieve the result from a model object. The examples demonstrate conversational responses, multiple-choice answers, simple webpage generation and JSON output constrained by a schema.

What guidance can do

guidance provides controls for both the content boundaries and syntax of generated responses. Constraints range from a regular expression on a generation call to reusable functions that compose a context-free grammar. guidance also includes dedicated JSON generation support and local tools for checking constraints before running them against a model.

  • guidance constrains generated text with regular expressions.
  • guidance restricts answers to supplied options through select().
  • guidance composes reusable functions into context-free grammars.
  • guidance generates JSON using a Pydantic model as the schema.
  • guidance validates candidate strings against a grammar locally.
  • guidance tests grammar execution with a local Mock model.

How guidance works

guidance is installed from PyPI with pip install guidance once the required model backend is available. Supported backends include Transformers, llama.cpp and OpenAI. The Python examples create a model object, add messages within role contexts, and append generation or selection operations to construct a response.

guidance functions use the @guidance decorator and can be reused alongside built-in generation operations. Stateless functions can be composed into grammars. When a grammar makes upcoming tokens predictable, guidance can insert those tokens directly, avoiding model forward passes. In Jupyter notebooks, a widget provides a richer view of generation.

Who guidance is for

guidance is suited to Python developers building language model workflows that need explicit output constraints. The repository examples focus on extracting named responses, answering multiple-choice questions, producing structured text and generating schema-constrained JSON. Developers working on grammars can also check candidate strings and use a mock model without making model API calls.

  • guidance supports multiple-choice workflows with a fixed set of permitted answers.
  • guidance supports structured JSON generation based on Pydantic models.
  • guidance supports reusable grammar functions for simple webpage generation.
  • guidance supports local testing while developers iterate on generation constraints.

guidance pricing and access

guidance is an open-source project distributed under the MIT license, with source code available in its public GitHub repository. guidance is also available as a Python package through PyPI. The installation instructions assume that the backend required for the chosen model is already available; the page does not list hosted subscription plans or model usage prices.

guidance strengths and limitations

guidance combines Python control flow, reusable grammars and structured generation within the same programming interface. Offline grammar checks help developers test constraints without model API calls, while token fast-forwarding can reduce GPU usage by avoiding unnecessary forward passes. Backend compatibility remains important: enforcing arbitrary context-free grammars requires a backend with full guidance support.

  • guidance requires a suitable backend for the selected model.
  • guidance depends on full backend support for arbitrary context-free grammar constraints.
  • guidance demonstrates simple webpage generation rather than a full HTML implementation.

guidance FAQ

What is guidance?

guidance is an open-source Python project for steering large language models and controlling output structure. guidance combines generation with Python control flow and supports constraints based on regular expressions, predefined choices, context-free grammars and JSON schemas.

Is guidance free and open source?

guidance is available as an open-source project under the MIT license and can be installed from PyPI. The repository does not list subscription plans or model usage prices, and the selected model requires a suitable backend.

Which model backends does guidance support?

guidance supports backends including Transformers, llama.cpp and OpenAI. The installation instructions assume the required backend is already available. Full support for arbitrary context-free grammar constraints depends on whether the selected backend fully supports guidance.

Can guidance generate structured JSON?

guidance provides dedicated JSON generation support that uses a schema to constrain language model output. The repository demonstrates passing a Pydantic model to the JSON generation function, capturing the result and validating the generated JSON with Pydantic.

Can guidance test grammars without model API calls?

guidance can validate candidate strings directly against a grammar and test a complete grammar run using a local Mock model. These capabilities let developers debug generation constraints locally without making model API calls.

How do you install guidance?

guidance can be installed from PyPI using pip install guidance after the required model backend is available. The repository demonstrates importing role contexts and generation functions, creating a backend model object and capturing generated responses in Python.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like guidance

LMQL

www.lmql.ai

A programming language for LLM prompting with types, templates, constraints and Python control flow.

Freemium

Marvin

www.askmarvin.ai

Marvin turns unstructured input into Python values and runs language-model tasks with tools.

Freemium

ClassifyAI

www.github.com

Open-source Python REST API for classifying text with custom JSON models and OpenAI.

Free

GGML

www.ggml.ai

A cross-platform tensor library for running large machine learning models on commodity hardware.

Free