Skip to content
Sign in
  • Browse all toolsFilter by category and pricing
  • Browse categoriesWriting, code, image, audio and more
  • Compare toolsPut up to three side by side
  • Read the blogGuides and comparisons

HFlow

Open-source Python SDK for robotics data quality checks, multimodal pipelines, and dataset curation.

HFlow is a Python SDK for multimodal robotics data pipelines that runs quality checks, tracks provenance, and curates training datasets through SQL. HFlow serves robotics and physical-AI teams and is available as an open-source project under the Apache-2.0 license through GitHub and PyPI.

Updated 9 Nov 20266 sections

HFlow website screenshot

What is HFlow?

HFlow is an open-source SDK for building multimodal data pipelines in robotics and physical AI. Developed by Hebbian Robotics, HFlow helps teams verify the quality of data used for AI model training. Recordings can combine video, state, actions, timestamps, and metadata from different collection systems.

HFlow addresses the difficulty of tracking processing work across fragmented scripts as a corpus grows. HFlow manages orchestration, storage, versioning, and curation around user-defined processing steps. The project is pre-v1, with the core lifecycle working end to end and local execution available for evaluation.

What HFlow can do

HFlow combines processing, quality evidence, and dataset curation in a shared workflow. Teams can start with built-in checks or supply their own transformations, labels, and enrichments. Measurements remain queryable, allowing dataset consumers to apply different quality thresholds without running the underlying media processing again.

  • HFlow connects existing Python processing functions through small adapters.
  • HFlow records schema, pipeline, and tool versions inside processed episodes.
  • HFlow renders Airflow DAGs for monitoring task status, logs, retries, and reruns.
  • HFlow stores metadata, measurements, tags, and artifact locations in a Parquet catalog.
  • HFlow uses DuckDB SQL to produce version-pinned dataset manifests.

How HFlow works

HFlow accepts supported standard MCAP episodes directly and imports supported LeRobot Dataset v3 repositories through its CLI. Processing steps are plain Python functions in the team's own environment. Local development runs in-process, while scheduled execution uses generated Airflow 3 DAGs through the included Docker Compose runtime or an existing Airflow environment.

HFlow can be installed from PyPI with uv add hflow. The repository's bundled quickstart synthesizes a small episode containing camera and state streams when no input file is supplied, runs the pipeline locally, and writes outputs under the data directory. This quickstart needs neither Docker nor Airflow.

Who HFlow is for

HFlow serves robotics and physical-AI teams managing multimodal training corpora, including teams using human-worn cameras, teleoperated robots, or autonomous policies. Data from these collection systems must be represented as supported MCAP episodes. HFlow also supports contributors who do not have robot hardware.

  • Robotics teams can investigate frozen cameras, missing topics, synchronization drift, and duplicate recordings.
  • Dataset developers can select episodes using catalog measurements and SQL.
  • Pipeline maintainers can trace processing outcomes to source data and step versions.

HFlow access and licensing

HFlow is available as an open-source project under the Apache-2.0 license, with source code on GitHub and SDK installation through PyPI. The repository includes a single-tenant Docker Compose runtime. A hosted platform waitlist is listed, but the hosted control plane is not implemented in the repository. Access to the hosted semantic-check API requires contacting the team.

HFlow strengths and limitations

HFlow keeps processing code in ordinary Python and uses MCAP, Parquet, and Airflow at its boundaries. Provenance and reusable quality measurements support auditing and dataset curation without reopening recordings. Deployment remains workspace-based rather than a built-in account-management service, and the project explicitly identifies remaining work before its initial release milestone.

  • HFlow is pre-v1, with implementation details and remaining work tracked in the repository.
  • HFlow requires supported MCAP data or conversion through the supported LeRobot importer.
  • HFlow has no built-in user accounts, RBAC, or multi-tenant control plane.

HFlow FAQ

What is HFlow?

HFlow is an open-source SDK for scalable multimodal data pipelines in robotics and physical AI. HFlow helps teams run quality checks, process recordings, track provenance, and curate training datasets using a queryable catalog.

Is HFlow open source?

HFlow is an open-source project licensed under Apache-2.0, with source code on GitHub and SDK installation through PyPI. HFlow includes local execution and a Docker Compose runtime; the repository does not implement the proposed hosted control plane.

What data formats does HFlow support?

HFlow uses supported standard MCAP episodes as its input and output boundary. HFlow also imports a supported subset of LeRobot Dataset v3 repositories, writes catalog data in Parquet, and uses DuckDB SQL for curation.

Does HFlow require Docker or Airflow?

HFlow does not require Docker or Airflow for its bundled local quickstart, which runs the pipeline in-process. HFlow supports scheduled execution through generated Airflow 3 DAGs, using the included Docker Compose runtime or an existing Airflow environment.

Can HFlow use existing Python quality checks?

HFlow accepts existing Python processing code through small adapters rather than requiring a rewrite into a proprietary framework. HFlow supports transformations, quality checks, labels, and enrichments, recording results as queryable measurements linked to processing versions.

Does HFlow offer a hosted service?

HFlow lists a hosted platform waitlist, but the hosted control plane is not implemented in the repository or committed for the pre-v1 release. HFlow also documents hosted semantic checks whose API access requires contacting the team.

From the community

No reviews yet

Be the first to review this tool. What you write helps the next person choose.

More tools like HFlow

Roboto AI

www.roboto.ai

Robotics data analytics for automated failure analysis, fleet reliability tracking, and training dataset curation.

Freemium

Dataloop

www.dataloop.ai

AI data infrastructure for unstructured data, multimodal pipelines, model deployment, and human feedback.

Freemium

datasets

www.github.com

Open-source Python library for loading, streaming, and preprocessing machine learning datasets.

Free

DataFlow

www.github.com

Open-source LLM data preparation with reusable operators, visual pipelines, and data generation and cleaning workflows.

Free