What is autoresearch?
autoresearch is an open-source project for letting AI agents experiment autonomously with LLM training on a single GPU. The repository provides a simplified implementation of nanochat rather than a hosted research service. Researchers and developers can use the project to investigate training changes while directing the agent through written instructions instead of manually editing the training code.
autoresearch follows a repeating experiment loop: an agent modifies code, trains a model, checks whether the result improved, and keeps or discards the change. The README describes leaving this process running overnight and reviewing the experiment log afterward. A better model is a possible outcome, not a guaranteed result.
What autoresearch can do
autoresearch gives the agent control over the training implementation while keeping data preparation and evaluation utilities fixed. The editable script contains the GPT model, a Muon and AdamW optimizer setup, and the training loop. This structure supports experiments across architecture and training settings without spreading agent edits throughout the repository.
- autoresearch lets agents change model architecture, hyperparameters, optimizers, and batch size.
- autoresearch runs training for a fixed 5-minute budget, excluding startup and compilation.
- autoresearch evaluates results using validation bits per byte, where lower values are better.
- autoresearch downloads training data and trains a BPE tokenizer through prepare.py.
- autoresearch stores human-written agent instructions in program.md.
How autoresearch works
autoresearch requires a single NVIDIA GPU, Python 3.10+, and uv, with the repository tested on H100. The documented setup installs dependencies with uv sync, prepares data with uv run prepare.py, and verifies training with uv run train.py. These steps establish a working local training setup before autonomous experiments begin.
autoresearch then uses a coding agent such as Claude or Codex running inside the repository. The README directs the user to point the agent at program.md to begin setup and experimentation. Humans iterate on program.md, the agent edits train.py, and prepare.py remains unchanged during the intended research workflow.
Who autoresearch is for
autoresearch is intended for people exploring autonomous research on LLM training who can operate a local GPU development environment. The project suits experiments where the question is which model or training changes perform better within the same time budget on the available hardware, rather than comparisons between different compute platforms.
- Researchers can run unattended training experiments overnight.
- Developers can test architecture and optimizer changes in a contained script.
- Agent experimenters can revise Markdown instructions to change research behavior.
autoresearch pricing and access
autoresearch is available as an open-source GitHub repository under the MIT license. The README provides local installation and execution instructions rather than product subscription plans. Access to the source code is separate from the hardware and software prerequisites: running experiments requires the supported GPU environment, dependencies, prepared data, and a coding agent.
autoresearch strengths and limitations
autoresearch keeps the experiment surface small: the agent edits a single file, training uses a fixed time budget, and evaluation relies on a vocabulary-size-independent metric. The README describes this approach as keeping diffs reviewable and experiments comparable within the same hardware environment, without distributed training or complex configuration files.
- autoresearch currently requires an NVIDIA GPU in the main repository.
- autoresearch results are not directly comparable across different compute platforms.
- autoresearch provides bare-bones baseline agent instructions that users can iterate on.
- autoresearch points users toward community forks for MacOS, Windows, and AMD alternatives.
autoresearch FAQ
What is autoresearch?
autoresearch is an open-source project that lets AI agents run autonomous experiments on single-GPU LLM training. The agent edits training code, trains a model, evaluates the result, and keeps or discards changes before repeating the process.
Is autoresearch free?
autoresearch is available as source code under the MIT license, rather than as a subscription product. Running autoresearch requires a supported NVIDIA GPU, Python, uv, and a coding agent; the repository does not specify costs for those resources.
What hardware does autoresearch require?
autoresearch requires a single NVIDIA GPU and was tested on H100. The documented software requirements include Python 3.10+ and uv. The main repository does not currently support CPU or MPS execution, while the README links to community forks for alternative platforms.
How does autoresearch evaluate experiments?
autoresearch evaluates experiments using val_bpb, or validation bits per byte, with lower values indicating better results. Training has a fixed 5-minute budget excluding startup and compilation. The metric is vocabulary-size-independent, but results are not directly comparable across different compute platforms.
Can autoresearch use Claude or Codex?
autoresearch documents running Claude, Codex, or another coding agent inside the repository and directing the agent to program.md. Humans edit the Markdown instructions, while the agent modifies train.py to conduct training experiments.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




