What is Candle?
Candle is an open-source machine learning framework for Rust, designed around performance and ease of use. Candle supports developers who need tensor operations, model training, and inference in Rust applications. The repository identifies lightweight deployment and removing Python from production workloads as reasons to use the framework.
Candle organizes its functionality into crates for core operations, neural network tools, datasets, transformer utilities, and model examples. Candle also includes CUDA kernels and ONNX model evaluation. This structure gives developers separate components for working with tensors, building models, loading data, and exploring implementations of existing models.
What Candle can do
Candle provides model implementations and examples across language, vision, and audio tasks, alongside lower-level tools for custom machine learning development. Candle supports CPU, GPU, and browser execution, with quantization options for supported models. The repository distinguishes online demos that run entirely in the browser from command-line examples.
- Candle runs text generation examples with models including LLaMA, Falcon, and Gemma.
- Candle supports speech recognition with Whisper and text-to-speech with Parler-TTS.
- Candle provides image generation examples using Stable Diffusion.
- Candle supports object detection, pose estimation, and image segmentation.
- Candle loads safetensors, npz, ggml, and PyTorch model files.
- Candle supports custom operations and kernels, including flash attention.
How Candle works
Candle applications use Rust code to create tensors, choose an execution device, and perform operations such as matrix multiplication. The introductory example uses candle-core on the CPU and runs through Cargo. After installing CUDA support, developers can change the device selection to execute the same workflow on a GPU.
Candle command-line examples run through Cargo, with CUDA enabled through a feature flag. Developers with cuDNN installed can enable the corresponding feature for additional speedups. Candle also provides WebAssembly examples that can be built with trunk; the documented local language-model example requires a WebAssembly target, model weights, and a tokenizer file.
Who Candle is for
Candle is for Rust developers and machine learning engineers building applications around pretrained models or custom model code. Candle is particularly relevant to workloads where lightweight binaries, serverless CPU deployment, or avoiding Python in production are requirements. Browser demos also provide a way to explore supported tasks before integrating the library.
- Candle helps developers deploy language model inference in Rust applications.
- Candle supports teams building speech transcription and audio generation workflows.
- Candle provides examples for image captioning, translation, and optical character recognition.
- Candle supports developers experimenting with models that run inside a browser.
Candle pricing and access
Candle is available as an open-source project through its public GitHub repository, which lists Apache-2.0 and MIT licenses. Candle access centers on source code, installation documentation, model examples, and online demos rather than a listed subscription plan. The supplied repository text does not describe paid tiers or hosted service pricing.
Candle strengths and setup limitations
Candle combines PyTorch-like syntax with Rust deployment and several execution backends. Candle supports optimized CPU execution with optional MKL or Accelerate support, CUDA execution, and multi-GPU distribution through NCCL. These capabilities sit alongside browser execution and quantized model support, allowing developers to explore different deployment approaches within the same framework.
- Candle application development requires installing the library and working with Rust code.
- Candle CUDA examples require CUDA support and explicit feature selection.
- Candle local WebAssembly examples require build tools and model assets.
Candle FAQ
What is Candle?
Candle is a minimalist open-source machine learning framework for Rust focused on performance and ease of use. Candle provides tensor operations, model training, pretrained model examples, and execution options spanning CPUs, CUDA GPUs, and browsers through WebAssembly.
Is Candle free and open source?
Candle is an open-source project available through a public GitHub repository under Apache-2.0 and MIT licenses. Candle's repository provides source code, installation guidance, examples, and browser demos; the supplied page does not list subscription plans or paid tiers.
Does Candle support GPU acceleration?
Candle supports a CUDA backend for GPU execution and multi-GPU distribution through NCCL. Candle requires installation with CUDA support and an appropriate feature flag for CUDA examples; developers with cuDNN installed can enable a separate feature for additional speedups.
Can Candle run models in a browser?
Candle supports browser-based model execution through WebAssembly. Candle's online demos include speech recognition, text generation, object recognition, image segmentation, and image captioning. Candle also provides WebAssembly examples that developers can build locally with trunk.
What model file formats does Candle support?
Candle supports loading models from safetensors, npz, ggml, and PyTorch files. Candle also supports quantization using llama.cpp quantized types and includes quantized language model examples in the repository.
Can Candle train models or only run inference?
Candle supports model training as well as inference. Candle includes core tensor operations, tools for building models, datasets and data loaders, and support for user-defined operations and kernels, alongside examples for running pretrained language, audio, and vision models.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




