Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

openTPU

Open-source AI accelerator with hardware design, a compiler, a simulator and profiling tools in one repository.

openTPU is an open-source AI accelerator project that combines hardware design, a compiler, a bit-exact simulator and profiling tools. openTPU supports language model inference on a Kintex-7 PCIe card and simulator-based experimentation for developers and learners, with source code available under the Apache-2.0 license.

Updated 9 Nov 20266 sections

openTPU website screenshot

What is openTPU?

openTPU is an open-source AI accelerator project developed by AI, with hardware and software maintained together in a small repository. openTPU is also a learning project for understanding accelerator design, connecting Python matrix operations to instructions and hardware execution. The repository includes SystemVerilog hardware, an instruction set, a simulator, a kernel language, a compiler and software that drives a physical PCIe card.

What openTPU can do

openTPU supports language model inference, execution analysis and hardware development within the same project. The repository reports models running with real weights on an Inspur YPCB-00338 card using a Xilinx Kintex-7 device. Reported configurations produce the same tokens as the simulator, while profiling tools expose instruction activity and memory behavior rather than hiding execution behind automatic scheduling.

  • openTPU runs supported Qwen3, Qwen3.5 and LFM2.5 models.
  • openTPU compiles kernels written in its ol language.
  • openTPU simulates instruction execution with bit-exact checks against RTL.
  • openTPU displays rooflines, timelines and instruction tables through Lens.
  • openTPU streams missing mixture-of-experts weights from host storage.
  • openTPU includes an ASIC layout flow using Yosys and OpenROAD.

How openTPU works

openTPU compiles kernels into accelerator instructions that run through the simulator or hardware implementation. A sequencer issues instructions to units for data movement, matrix multiplication, vector mathematics and quantization. Explicit data-movement instructions make execution traces inspectable. Lens records runs from RTL, the simulator or the card and opens the results in a browser.

openTPU provides a laptop workflow that installs the repository and dependencies, runs tests, downloads a model checkpoint and starts otpu-chat with the ISA backend. Physical card use follows a separate path: build the bitstream in the board directory, load the image over JTAG, run the host setup command and launch chat with the board backend.

Who openTPU is for

openTPU is for people studying how AI accelerators work and developers exploring the relationship between kernels, instructions, hardware and inference performance. The repository can be read end to end, making the project relevant to hardware and compiler experimentation. The documented ASIC timing loop also provides a workflow and a ready-to-fill prompt for delegating physical-design work to a coding agent.

  • Learners can trace a Python matrix operation down to hardware execution.
  • Developers can compare simulator behavior with RTL and card results.
  • Performance researchers can inspect memory waits and instruction timing.
  • Hardware designers can study the documented ASIC timing-repair loop.

openTPU access and licensing

openTPU is available as an open-source GitHub repository under the Apache-2.0 license. The supplied page describes installation from the repository rather than hosted subscription plans. The simulator path allows experimentation on a laptop without the accelerator card. Running inference on physical hardware additionally requires the documented board workflow, including bitstream preparation, JTAG loading and host configuration.

openTPU strengths and limitations

openTPU combines inspectable hardware, software and performance tools with token-level comparisons between simulation and physical execution. The explicit instruction model and browser-based profiler support investigation of execution costs. Hardware use still involves board setup, RTL testing needs an additional tool, and reduced-precision weights trade lower memory traffic for a measurable perplexity cost documented separately for each model.

  • Physical card execution requires a compatible board and JTAG loading.
  • RTL tests additionally require Verilator 5.
  • Quantized weights introduce a measurable perplexity trade-off.
  • Models larger than card memory can depend on host-storage expert streaming.

openTPU FAQ

What is openTPU?

openTPU is an open-source AI accelerator project developed by AI. openTPU combines SystemVerilog hardware, an instruction set, a bit-exact simulator, a kernel language, a compiler and host software, and also serves as a learning resource for understanding AI accelerator execution.

Is openTPU free and open source?

openTPU is available as an open-source repository under the Apache-2.0 license. The repository provides a laptop-based simulator workflow, while running the physical accelerator requires a compatible FPGA card and the documented bitstream and host setup process.

Can openTPU run without an FPGA card?

openTPU supports a simulator workflow on a laptop without the FPGA card. The documented steps install the project and dependencies, run tests, download a model checkpoint and launch otpu-chat using the ISA backend.

Which language models does openTPU support?

openTPU reports card execution for models from families including LFM2.5, LFM2, Qwen3, Qwen3.5, Gemma 4, SmolLM3 and Phi-4-mini. The repository also describes mixture-of-experts inference that streams missing experts from host storage when models exceed card memory.

What profiling tools does openTPU include?

openTPU includes Lens, a profiler that records runs from RTL, the simulator or the physical card and displays results in a browser. Lens provides a roofline, an execution timeline and per-instruction tables to show unit activity and waiting behavior.

What hardware does openTPU run on?

openTPU reports language model inference on an Inspur YPCB-00338 PCIe card with a Xilinx Kintex-7 xc7k480t device and two DDR3 channels. Card use requires building a bitstream, loading the image over JTAG and configuring the host before launching the board backend.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like openTPU

BitNet

www.github.com

Open-source inference framework for running 1-bit language models on CPUs and GPUs.

Free

llama.cpp

www.github.com

Open-source LLM and vision-language model inference in C/C++ for local and cloud hardware.

Free

llmfit

www.github.com

Find local LLMs that fit your CPU, RAM, GPU, and VRAM, with hardware detection and performance estimates.

Free

Open-source model optimization library for quantization, pruning, distillation and faster inference.

Free