Scaling Open-World Egocentric Human Data
for Humanoid Loco-Manipulation

ReportGitHubHugging FaceSoonBibTeX
Explore the tasks Watch video

Research overview

Human experience.Whole-body intelligence.

Diverse human activities reveal how locomotion, posture, and dexterous hand movements work together across varied scenes, objects, and interaction contexts. Stepping closer, bending to reach, and maintaining balance during contact offer rich, naturally occurring examples of whole-body coordination.

λ₀ builds on diverse human experience. HumanVerse-500 captures everyday activities through synchronized egocentric vision, body motion, and hand motion, preserving the coordination between locomotion, posture, and dexterous interaction. Our three-stage training recipe learns general whole-body priors from these human activities, then acquires manipulation-specific biases through robot demonstrations.

Core idea: we learn whole-body priors from diverse human data, then manipulation-specific biases from robot demonstrations.

Real-world demonstrations

Move. Reach. Manipulate.

Whole-body loco-manipulation across the laboratory, break room, and visitor center.

Laboratory

Break room

Visitor center

Zero-Shot Transfer on Unitree G1

Whole-body coordination.
Long-range interaction.

Aligned human task demonstrations support G1 control without target-task robot demonstrations. These evaluations validate the robot-aligned data pipeline across two representative task families.

EgoAlign

76.2% progress across relocation, navigation, and foot interaction, averaging seen and unseen target locations. Videos also illustrate targets moved during execution.

Basket placement

0:00 / 0:00
Closer look: end-effector alignment

Whole-Body Loco-Manipulation

EgoHumanoid-V2

51.7% progress across four coordinated whole-body tasks. Compare the human demonstration, aligned execution, and unaligned replay.

Human demonstration
0:00 / 0:00
Aligned execution
0:00 / 0:00
Unaligned replay
0:00 / 0:00

HumanVerse-500

Capture the body.
Keep the context.

500 hours of robot-free capture across open-world scenes and tasks, pairing egocentric video with synchronized whole-body and hand motion.

HumanVerse-500: wearable capture setup, skill frequencies, five-ring activity distribution, and synchronized image–pose examples
HumanVerse-500 spans everyday scenes, objects, and coordinated whole-body activities.
HumanVerse in motionSMPL + MANO + RGB
Loading recorded motion…
0:00 / 0:02
From human motion to G1Synchronized replay
Human motion SMPL-X + fitted hands + RGBG1 + Revo2 GMR retargeting → SONIC
Loading recorded motion…
0:00 / 0:08

Egocentric vision

A neck-mounted GoPro records first-person RGB at a height matched to the robot camera, retaining the objects and surroundings that give actions their context.

24 body joints

Wearable PICO tracking captures global displacement, posture, and coordinated body motion on a synchronized timeline.

21 joints per hand

Image-based hand reconstruction estimates articulated hand geometry, complementing directly tracked body motion.

Robot-Aligned Data Pipeline

Observation alignment matches the egocentric camera to the robot view. For action-space alignment, the three steps below correct the human reference, fit a robot reference through whole-body IK and SONIC rollout, then refine motion with dynamics-aware optimization.

Three-stage alignment pipeline: human-reference alignment, robot-reference alignment, and dynamics-aware optimization
The alignment pipeline turns human motion into robot-compatible motion through human-reference correction, robot-reference alignment, and dynamics-aware refinement.
Efficient Robot-Free Collection

Estimated time per demo: approximately 70 s robot / 33 s human for whole-body manipulation, and 130 s robot / 25 s human for long-range loco-manipulation. Equal weighting gives approximately 100 s versus 29 s, or 3.4× collection throughput across these representative families.

Separate data for embodiment grounding.

An additional 600 task-aligned human demonstrations—150 per task—are reserved for Stage III and excluded from the 500-hour mid-training pool. Sessions are split before temporal windows are constructed.

The λ₀ training recipe

From interaction priors
to humanoid actions.

A shared vision–language backbone and action expert learn across human and robot data, while embodiment-specific interfaces preserve their different physical meanings.

Stage I · Egocentric Hand-Motion Pre-training

Learn interaction

Pretrain on seven egocentric human datasets to learn object interaction and articulated hand motion across diverse activities.

Stage II · Robot-Aligned Whole-body Human Mid-training

Learn coordination

Mid-train on robot-aligned whole-body human data from HumanVerse-500. The robot-aligned data pipeline converts human demonstrations into the downstream robot observation and action spaces.

Stage III · Robot Post-training

Ground in the robot

Post-train with task-aligned human and native robot demonstrations, grounding shared representations in executable humanoid actions.

Updated three-stage training pipeline with egocentric hand-motion pretraining, whole-body human mid-training, and robot post-training
The shared model learns from human interactions, whole-body motion, and embodiment-specific demonstrations.

At deployment

The policy takes the robot’s camera image, language instruction, and proprioception. It predicts whole-body motion tokens and hand commands; SONIC executes the body motion. Human tracking is used during data collection, not at deployment.

Evaluation results

Coordinated action.
Measurable progress.

We report both completed tasks and milestone-weighted progress. The two metrics distinguish terminal success from useful partial execution.

Robot hardware and evaluation scenes

The Unitree G1 uses BrainCo Revo 2 hands and a head-mounted GoPro. PICO body tracking and HexaCercle M11 gloves provide teleoperation demonstrations, with SONIC translating whole-body motion into robot control.

Teleoperation hardware, four real-world task sequences, and evaluation objects
The paper’s hardware setup, task sequences, and robot-seen and robot-unseen objects.
62.5%

Real-world success

25 of 40 trials completed; +22.5 percentage points over the strongest external baseline.

80.4%

Real-world progress

Average over four tasks; +25.8 percentage points over the strongest external baseline.

90.0%

SIMPLE success

Six core tasks, three difficulty levels, 180 trials. Uses the best recorded configuration per task.

Real-world comparison

Higher is better · four tasks, 40 trials per method

Real-world success and task progress
MethodSuccess (%)Task progress (%)
GR00T N1.6
7.5
21.3
Ψ₀
10.0
31.8
StarVLA
27.5
54.6
π₀.₅
40.0
54.0
λ₀ (Ours)
62.5
80.4
Training-stage ablations

All variants retain vision–language initialization and Stage III. Removing Stage II causes a larger real-world success drop than removing Stage I.

Real-world training-stage ablations
Training recipeSuccess (%)Progress (%)
w/o human-data pretrainingNo Stage I or II15.054.0
w/o Stage II mid-trainingStages I + III22.559.9
w/o Stage I pretrainingStages II + III50.075.9
Full recipeStages I + II + III62.580.4

Human Data Scaling

More experience.
Lower validation loss.

With a fixed 2B model and 100k Stage-II updates, nested human-data subsets let us study how data quantity changes learning.

Human-data scaling: validation lossRaw representative and clean validation curves at 5, 10, 25, 50, and 100 percent of training data, followed by their full-window minimum losses. The left axes use logarithmic loss; the right chart uses logarithmic data fraction. Dashed lines are descriptive fits, not extrapolations.(a) Mid-training Validation LossValidation Loss (log scale)Representative Validation0.0050.0060.0070.010.0150.02025k50k75k100kLate-Stage Loss IncreaseClean Validation025k50k75k100kLate-Stage Loss Increase5%10%25%50%100%(b) Validation Loss Scaling0.0060.0080.0100.012Similar Trends5102550100RepresentativeCleanTraining Data (%, log scale)Best Validation Loss
Validation-loss trend
Increasing the human-data fraction lowers held-out loss and delays late-stage degradation at smaller scales.

Two fixed holdouts

From 5% to 100% data, minimum loss decreases by 35.5% on the representative holdout and 40.6% on the clean holdout. Both are disjoint from training sessions.

Downstream task progress

No Stage II59.9%
+20.5 pts
Full recipe80.4%

The measured real-world endpoints show a gain from Stage II. Evaluations at 5%, 10%, 25%, and 50% are pending, so intermediate-scale trends remain unestablished.

BibTeX

Cite λ₀, EgoHumanoid-V2, and EgoAlign using their official arXiv entries.

Download .bib
@misc{xu2026generalhumanoidlocomanipulationmodel,
      title={Towards a General Humanoid Loco-Manipulation Model via Egocentric Whole-Body Human Data Pretraining},
      author={Chongyang Xu and Zhao Wu and Jin Chen and Yiming Jiang and Jinhui Ye and Yuming Jiang and Shifeng Zhang and Ziliang Feng and Mu Xu and Yilun Chen and Li Lu and Steven C. H. Hoi},
      year={2026},
      eprint={2610.00438},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2610.00438},
}

@misc{chen2026egohumanoidv2humantohumanoidtransfercoordinated,
      title={EgoHumanoid-V2: Human-to-Humanoid Transfer of Coordinated Whole-Body Skills for Loco-Manipulation},
      author={Jin Chen and Yiming Jiang and Chongyang Xu and Modi Shi and Shijia Peng and Li Chen and Tianyu Li and Mu Xu and Yilun Chen and Steven Hoi and Hongyang Li},
      year={2026},
      eprint={2609.37181},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2609.37181},
}

@misc{jiang2026egoalignbridginghumanhumanoidgap,
      title={EgoAlign: Bridging the Human-Humanoid Gap for Long-Range Loco-Manipulation},
      author={Yiming Jiang and Jin Chen and Chongyang Xu and Yilun Chen and Aimin Hao and Yisheng He},
      year={2026},
      eprint={2609.38046},
      archivePrefix={arXiv},
      primaryClass={cs.RO},
      url={https://arxiv.org/abs/2609.38046},
}

Explore the research

Human motion as a foundation
for humanoid control.

Read the full method, evaluation protocols, and experimental findings in the manuscript.

Read the report