What is HFlow?
HFlow is an open-source SDK for building multimodal data pipelines in robotics and physical AI. Developed by Hebbian Robotics, HFlow helps teams verify the quality of data used for AI model training. Recordings can combine video, state, actions, timestamps, and metadata from different collection systems.
HFlow addresses the difficulty of tracking processing work across fragmented scripts as a corpus grows. HFlow manages orchestration, storage, versioning, and curation around user-defined processing steps. The project is pre-v1, with the core lifecycle working end to end and local execution available for evaluation.
What HFlow can do
HFlow combines processing, quality evidence, and dataset curation in a shared workflow. Teams can start with built-in checks or supply their own transformations, labels, and enrichments. Measurements remain queryable, allowing dataset consumers to apply different quality thresholds without running the underlying media processing again.
- HFlow connects existing Python processing functions through small adapters.
- HFlow records schema, pipeline, and tool versions inside processed episodes.
- HFlow renders Airflow DAGs for monitoring task status, logs, retries, and reruns.
- HFlow stores metadata, measurements, tags, and artifact locations in a Parquet catalog.
- HFlow uses DuckDB SQL to produce version-pinned dataset manifests.
How HFlow works
HFlow accepts supported standard MCAP episodes directly and imports supported LeRobot Dataset v3 repositories through its CLI. Processing steps are plain Python functions in the team's own environment. Local development runs in-process, while scheduled execution uses generated Airflow 3 DAGs through the included Docker Compose runtime or an existing Airflow environment.
HFlow can be installed from PyPI with uv add hflow. The repository's bundled quickstart synthesizes a small episode containing camera and state streams when no input file is supplied, runs the pipeline locally, and writes outputs under the data directory. This quickstart needs neither Docker nor Airflow.
Who HFlow is for
HFlow serves robotics and physical-AI teams managing multimodal training corpora, including teams using human-worn cameras, teleoperated robots, or autonomous policies. Data from these collection systems must be represented as supported MCAP episodes. HFlow also supports contributors who do not have robot hardware.
- Robotics teams can investigate frozen cameras, missing topics, synchronization drift, and duplicate recordings.
- Dataset developers can select episodes using catalog measurements and SQL.
- Pipeline maintainers can trace processing outcomes to source data and step versions.
HFlow access and licensing
HFlow is available as an open-source project under the Apache-2.0 license, with source code on GitHub and SDK installation through PyPI. The repository includes a single-tenant Docker Compose runtime. A hosted platform waitlist is listed, but the hosted control plane is not implemented in the repository. Access to the hosted semantic-check API requires contacting the team.
HFlow strengths and limitations
HFlow keeps processing code in ordinary Python and uses MCAP, Parquet, and Airflow at its boundaries. Provenance and reusable quality measurements support auditing and dataset curation without reopening recordings. Deployment remains workspace-based rather than a built-in account-management service, and the project explicitly identifies remaining work before its initial release milestone.
- HFlow is pre-v1, with implementation details and remaining work tracked in the repository.
- HFlow requires supported MCAP data or conversion through the supported LeRobot importer.
- HFlow has no built-in user accounts, RBAC, or multi-tenant control plane.
HFlow FAQ
What is HFlow?
HFlow is an open-source SDK for scalable multimodal data pipelines in robotics and physical AI. HFlow helps teams run quality checks, process recordings, track provenance, and curate training datasets using a queryable catalog.
Is HFlow open source?
HFlow is an open-source project licensed under Apache-2.0, with source code on GitHub and SDK installation through PyPI. HFlow includes local execution and a Docker Compose runtime; the repository does not implement the proposed hosted control plane.
What data formats does HFlow support?
HFlow uses supported standard MCAP episodes as its input and output boundary. HFlow also imports a supported subset of LeRobot Dataset v3 repositories, writes catalog data in Parquet, and uses DuckDB SQL for curation.
Does HFlow require Docker or Airflow?
HFlow does not require Docker or Airflow for its bundled local quickstart, which runs the pipeline in-process. HFlow supports scheduled execution through generated Airflow 3 DAGs, using the included Docker Compose runtime or an existing Airflow environment.
Can HFlow use existing Python quality checks?
HFlow accepts existing Python processing code through small adapters rather than requiring a rewrite into a proprietary framework. HFlow supports transformations, quality checks, labels, and enrichments, recording results as queryable measurements linked to processing versions.
Does HFlow offer a hosted service?
HFlow lists a hosted platform waitlist, but the hosted control plane is not implemented in the repository or committed for the pre-v1 release. HFlow also documents hosted semantic checks whose API access requires contacting the team.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




