What is Miles?
Miles is an open-source reinforcement learning framework for enterprise-facing LLM and VLM post-training. Miles was forked from slime and continues to co-evolve with that project. The framework pairs SGLang rollout generation with scalable training infrastructure, bringing together model training, rollout execution, precision controls, and resilience features for large-scale reinforcement learning workloads.
Miles serves teams building and post-training models rather than providing a consumer chat interface. The repository describes support for model labs, agent infrastructure, and academic work. Alongside language and vision-language model training, the project links to Miles-diffusion for diffusion-model reinforcement learning and supervised fine-tuning.
What Miles can do
Miles combines rollout generation, training recipes, and correctness mechanisms within a distributed post-training framework. SGLang handles generation, while supported training backends include Megatron-LM, FSDP2, and torchtitan. Miles also provides configurable asynchronous scheduling and mechanisms intended to keep rollout data, model weights, and mixture-of-experts routing aligned during training.
- Miles decouples rollout and training workers for fully asynchronous reinforcement learning.
- Miles supports GRPO, GSPO, PPO, REINFORCE++, supervised fine-tuning, and on-policy distillation.
- Miles supports low-precision training alongside LoRA and multi-LoRA adapters.
- Miles preserves token sequences between rollout and training through token-in-token-out processing.
- Miles replays rollout expert routing during the trainer's forward pass.
- Miles recovers failed SGLang engines and resumes the training run in place.
How Miles works
Miles routes generation requests across SGLang engines, preserving per-request metadata and checking engine health. Rollout and training workers operate separately, with configurable on-policy and off-policy schedules. Updated weights transfer back to generation engines during the run, with P2P RDMA available as the fast path for disaggregated setups.
Miles can run through Ray or deploy as a Helm release with Kubernetes scheduling the workers. The repository provides installation guidance, a launch-script walkthrough, and training-backend documentation. Supported hardware includes NVIDIA and AMD GPUs, with separate installation status by GPU and ROCm image guidance for AMD deployments.
Who Miles is for
Miles targets enterprise teams conducting large-scale model post-training, including teams working on language models, vision-language models, and agent behavior. Agentic environment connectors include Harbor, HUD, NeMo Gym, OpenEnv, and Verifiers. Task sandboxes can use AgentENV, Daytona, E2B, or Modal, depending on the training environment.
- Miles supports post-training language and vision-language models.
- Miles supports training coding and computer-use agents.
- Miles supports adapter-based training with LoRA and multi-LoRA.
- Miles supports supervised fine-tuning and on-policy distillation workflows.
Miles pricing and access
Miles is available as an open-source project through its public GitHub repository under the Apache-2.0 license. The supplied repository text does not list Miles subscription tiers or paid plans. Access information instead points to installation instructions, a quick start, supported-model documentation, core concepts, and a contribution guide.
Miles strengths and limitations
Miles brings together multiple training backends, asynchronous rollout execution, engine recovery, and agent-environment connectors. These capabilities support teams managing distributed post-training workloads across supported NVIDIA or AMD hardware. Miles also documents correctness mechanisms for token handling and mixture-of-experts routing, addressing specific sources of mismatch between rollout and training.
- Miles requires technical setup across installation, training backends, and worker deployment.
- Miles hardware compatibility depends on the GPU, with per-GPU status covered in the installation documentation.
- Miles AMD deployments have separate ROCm image guidance.
Miles FAQ
What is Miles?
Miles is an open-source reinforcement learning framework for large-scale LLM and VLM post-training. Miles is enterprise-facing, was forked from slime, and combines SGLang rollout generation with training backends including Megatron-LM, FSDP2, and torchtitan.
Is Miles free?
Miles is available as an open-source project under the Apache-2.0 license through a public GitHub repository. The supplied Miles repository text does not state subscription prices or describe paid Miles plans.
Who is Miles for?
Miles is for enterprise model-training teams working on large-scale language and vision-language model post-training. Miles also supports coding and computer-use agent training, with connectors to environments including Harbor, HUD, NeMo Gym, OpenEnv, and Verifiers.
What training methods does Miles support?
Miles supports GRPO, GSPO, PPO, and REINFORCE++ for reinforcement learning, alongside score centering, supervised fine-tuning, and on-policy distillation. Miles also supports LoRA and multi-LoRA adapters and multiple low-precision training formats.
Can Miles run on Kubernetes?
Miles supports Kubernetes deployment as a Helm release, allowing Kubernetes to schedule the workers in a training run. Miles also supports Ray, and the repository states that the same launch script works with these deployment options.
What GPUs does Miles support?
Miles lists support for NVIDIA GB300, GB200, B300, B200, H200, H100, and A100 GPUs, alongside AMD MI355X, MI350X, MI325X, and MI300X. Miles directs users to installation documentation for per-GPU status and AMD ROCm documentation for images.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




