EgoMimic: Scaling Imitation Learning through Egocentric Video

Dataset Overview

EgoMimic contains human and robot operation data for imitation learning. The dataset includes samples of the following tasks:

• Grocery handling (Groceries)
• Laundry handling (Laundry)
• Bowl placement (Bowlplace)

Data Structure

The dataset is stored in HDF5 format and includes the following files:

• groceries_human.hdf5
• groceries_robot.hdf5
• smallclothfold_human.hdf5
• smallclothfold_robot.hdf5
• bowlplace_human.hdf5
• bowlplace_robot.hdf5

Data Processing

Processing scripts are located in the following directories:

• egomimic/scripts/aloha_process/: Converts raw Aloha‑style data to robomimic‑style HDF5 files.
• egomimic/scripts/aria_process/: Converts human data captured with Aria glasses to robomimic‑style HDF5 files.

Data Download

Sample data can be downloaded with the following commands:

mkdir datasets
cd datasets

Groceries

wget https://huggingface.co/datasets/gatech/EgoMimic/resolve/main/groceries_human.hdf5 wget https://huggingface.co/datasets/gatech/EgoMimic/resolve/main/groceries_robot.hdf5

Laundry

wget https://huggingface.co/datasets/gatech/EgoMimic/resolve/main/smallclothfold_human.hdf5 wget https://huggingface.co/datasets/gatech/EgoMimic/resolve/main/smallclothfold_robot.hdf5

Bowlplace

wget https://huggingface.co/datasets/gatech/EgoMimic/resolve/main/bowlplace_human.hdf5 wget https://huggingface.co/datasets/gatech/EgoMimic/resolve/main/bowlplace_robot.hdf5

Training and Evaluation

Quick Start

Run the following command to train on the sample data:

python scripts/pl_train.py --config configs/egomimic_oboo.json --dataset /path/to/bowlplace_robot.hdf5 --dataset_2 /path/to/bowlplace_human.hdf5 --debug

Detailed Experiment Commands

See experiment_launch.md for full experiment commands.

Offline Evaluation

Run the following command for offline evaluation:

python scripts/pl_train.py --dataset <dataset> --ckpt_path <ckpt> --eval

Custom Data Processing

Refer to data_processing.md for detailed instructions on processing custom data.