Open motion generation research

RoMo

An open ecosystem of large-scale, Richly Organized Motion data and shared tools

Full-body and hand-motion datasets built on one collection stack, one semantic taxonomy, and one toolbox for measuring, converting, and viewing motion.

Explore the datasets Hugging Face Motion Toolbox

2 open datasets

1.36M text-paired motion sequences

2,122 h of motion

1 shared toolbox

Why RoMo exists

Motion generation deserves the data that moved language, image, and video

Generative models in other fields scaled because the community shared large, well-curated data and agreed on how to measure progress. 3D human motion has lagged behind.

The data gap

Small and clean, or large and noisy

Researchers choose between high-fidelity motion capture that is too small for modern generative models and in-the-wild collections dominated by static or low-quality sequences. Hands are often missing entirely, or treated as rigid end-effectors.

The comparison gap

Every paper, its own pipeline

Formats, representations, and evaluators are re-implemented from project to project. Aggregate scores hide category-specific failures, and results across papers are hard to compare or reproduce.

Our goal

Enable open motion generation research by giving the community large-scale, carefully curated data and shared tools to build on, compare against, and extend.

01 · Open data

Scale with quality gates

In-the-wild motion distilled through aggressive filtering, then released on Hugging Face for research use.

02 · Shared organization

One taxonomy, rich language

A three-level semantic taxonomy and detailed captions make it possible to evaluate per category, not just on average.

03 · Shared tools

Comparable by default

The Motion Toolbox standardizes metrics, format conversion, and visualization so results line up across papers.

Datasets

Two datasets, one foundation

Each dataset has its own project page, paper, and Hugging Face release. Both share RoMo's collection stack and taxonomy.

CVPR 2026

RoMo

Full-body human motion

In-the-wild SMPL motion distilled from about 125K hours of web video, where only about 1% survives a taxonomy-aware filtering pipeline. Every sequence carries detailed captions and sits in a three-level semantic taxonomy.

813,938 sequences 1,238 h of motion 54 → 2,065 → 28,874 taxonomy nodes

SMPLHumanML3D 263-D272-D

Project page Paper

NeurIPS 2026

RoMo Hands

Bimanual hand motion

In-the-wild bimanual MANO motion with five caption levels, from action tags to paragraphs about finger articulation, a 393-D hand representation, and a benchmark that scores both semantic alignment and physical plausibility.

541,929 sequences 884 h of motion 5 caption levels

393-D R_handLitData

Project page Paper Dataset

Code

Motion Toolbox

One open-source library to measure, convert, and view motion across the ecosystem, so a result on RoMo means the same thing in every paper.

Standard metrics FID, R-Precision, diversity, multimodality, and per-category scores computed the same way everywhere.
Format conversion SMPL, HumanML3D 263-D, 272-D, and MHR body formats with consistent root-motion handling.
Visualization Interactive browser viewers and GLB export, the same pipeline behind the RoMo motion gallery.

View on GitHub

quickstart

# install
git clone https://github.com/RoMoDataset/motion-toolbox
cd motion-toolbox && uv sync

# visualize any supported motion format
from motion_toolbox.visualization import Visualizer

Visualizer().visualize(motion, "motion.html")

How it fits together

One collection stack, many kinds of motion

Every RoMo dataset starts from the same curated web video, temporal segmentation, and semantic taxonomy. New motion types branch from that shared foundation, and the Motion Toolbox ties them back together.

Source Curated web video Meta, scene, and human filters

Shared foundation RoMo collection stack Semantic segmentation · three-level taxonomy · captioning

RoMo GVHMR → SMPL body motion

RoMo Hands WiLoR → MANO hand motion

Shared tools Motion Toolbox Metrics · conversion · visualization

Open research Your model Trained, compared, and shared

Build with RoMo

Use it, compare on it, extend it

The ecosystem gets more useful as more people train on it, report against it, and contribute back.

Train

Start from scale

Download a dataset from Hugging Face in the representation your model expects: SMPL, HumanML3D 263-D, 272-D, or the 393-D hand representation.

Compare

Report on shared ground

Evaluate with the Motion Toolbox and report per-category results, so improvements are visible where they actually happen.

Contribute

Grow the toolbox

Add converters, metrics, and viewers on GitHub so the next dataset and the next paper start further ahead.

Citation

Cite the work you use

If you use a RoMo dataset or the Motion Toolbox, please cite the corresponding paper.

RoMo

@inproceedings{Zhang2026RoMo,
  author    = {Zhang, Jiahao and Liu, Joseph and Lee, Young-Yoon and Moon, Seonghyeon and Zordan, Victor and Tevet, Guy and Liu, Karen and Gould, Stephen and Jacob, Oren and Jiang, Haomiao and Kapadia, Mubbasir and Ben-Shabat, Yizhak},
  title     = {RoMo: A Large-Scale, Richly Organized Dataset and Semantic Taxonomy for Human Motion Generation},
  booktitle = {Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026},
}

RoMo Hands

@inproceedings{BenShabat2026RoMoHands,
  author    = {Ben-Shabat, Yizhak and Zhang, Jiahao and Liu, Joseph and Moon, Seonghyeon and Lee, Young-Yoon and Jiang, Haomiao and Jacob, Oren and Kapadia, Mubbasir},
  title     = {{RoMo Hands}: A Large Scale Richly Organized Text to Hand Motion Dataset},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  year      = {2026},
}