HuGeHUMANOID GENERALIST
HUMAN EXPERIENCE. HUMANOID INTELLIGENCE.

HuGe: Scaling Humanoid Generalist
Vision-Language-Action Models
with Egocentric Experience

Learning from egocentric experience—first through large-scale pre-training, then through task-specific adaptation.

Read the paper Explore the code
FROM EXPERIENCE TO ACTION 01 / G1
The Unitree G1 humanoid next to a table with a cola bottle and a nearby bin in the real-world experiment.
Learning to act in the physical world.Unitree G1 · Real-world setup
2,000+hours of human & robot experience
3architectures, one action interface
7HumanoidArena evaluation tasks
Explore the research

Yang Liu1*Jiacheng Liu1,2,3*Hengtao Li4*Yuheng Huang2Zijian Yang5Ji Dai5Maoxin Wang5Yi Gao6Qishen Wang7Baohua Yin8Peiyang Li5Shuanghao Bai9Mingyang Sun1,2,3Pengxiang Ding1,2Jianqin Yin5Lixin Yang3,10†Donglin Wang1†

  1. Westlake University
  2. Zhejiang University
  3. Shanghai Innovation Institute
  4. Hebei University of Technology
  5. Beijing University of Posts and Telecommunications
  6. Southwest University of Finance and Economics
  7. University of Science and Technology of China
  8. University of Sussex
  9. Xi'an Jiaotong University
  10. Shanghai Jiao Tong University

* Equal contribution · † Corresponding authors

01 / THE IDEA

Human experience.
Whole-body intelligence.

Everyday human activity offers a rich source of experience. How can a humanoid learn from it?

Robot demonstrations are expensive to collect, especially when a task requires coordinated movement of the entire body. Egocentric human recordings offer scale, but humans and robots look different—and human video does not directly provide robot proprioception.

We investigate two roles for egocentric data. First, we refine a pre-training recipe through training-strategy, pseudo-proprioception, and data-scale studies. We then compare the complete recipe against existing baselines. Finally, we study how task-specific human demonstrations and visual adaptation support downstream fine-tuning on a real humanoid.

PART I / PRE-TRAININGBuild the recipe, one experiment at a time.Training strategy → Pseudo-proprioception → Data scale → Baseline comparison PART II / DOWNSTREAMBring human task experience to a real robot.Visual adaptation → Fine-tuning with ego data → Real-world demonstrations
02 / THE METHOD

Different experience.
One common interface.

Three representative VLA architectures.
A shared path from perception to action.

FIGURE 01 HUGE FRAMEWORK
Swipe to explore

FROM HUMAN EXPERIENCE TO HUMANOID CONTROL

Learn. Unify. Act.

01

A universal action space

Different embodiments. Shared motion tokens.

SONIC FROZEN MOTION PRIOR
Robot motion in simulation
ROBOT EXPERIENCE

Robot action

Robot motion with tracked VR keypoints
HYBRID EXPERIENCE

VR keypoints
+ robot leg action

Human motion in a shared 3D space
HUMAN EXPERIENCE

Human motion

Robot encoder
Hybrid encoder
Human encoder
SHARED LATENT SPACE

Universal motion token

64 dimensions

WHOLE-BODY CONTROL

Robot control
decoder

Robot state

Fixed SONIC controller

EXECUTION

Whole-body action

Humanoid coordinating its arms and legsBODY + HANDS
Motion token per step 64-D Human motion → simulated robot experience Hand controls2-D / action
02

From experience to policy

Reconstruct the state. Learn the action.

SONIC ENCODE / DECODE

Simulation playback

Parallel humanoid simulation reconstructs robot states from human motion

Replay human motion on G1

STATE RECONSTRUCTION

Pseudo-
proprioception

34-D

29 joints + 2 hands
+ 3 gravity components

HUMANOID GENERALIST VLATRAINABLE

HuGe-Pi

Qwen3-VL 2B · vision + languageDiT

Layer-wise backbone features

ImageLanguageStateNoise

Flow matching · layer-wise conditioning

VLA OUTPUT

Action chunk

50×66

time steps × dimensions / action

Per action: 64-D motion token+ 2 hand controls

50 Hz action sequence

03

A progressive training recipe

Scale experience, then specialize.

STAGE 011,669.61 h
Diverse egocentric human and simulated robot datasets

Egocentric
pre-training

Human demonstrations &
simulated robot experience

EgoDex · EgoVerse · Xperience-10M · BEHAVIOR-1K

STAGE 02417.58 h
Real humanoid demonstration datasets

Humanoid
adaptation

Real robot observations &
proprioceptive states

Humanoid Everyday · UniFoLM-WBT · HIW-500

STAGE 03TASK-SPECIFIC
Human teleoperating a humanoid robot
Original human viewVisually adapted robot view

Downstream
fine-tuning

Task demonstrations &
visually adapted ego data

HumanoidArena · Real-world cola pick-and-drop

Shared action targets across embodiments · Full VLA fine-tuning · Frozen low-level controller
Motion & learning State reconstruction Hand controls

Hover or focus a step to trace its connections

Human and robot experience share a SONIC motion space. Simulation supplies missing robot states; through three training stages, HuGe learns to predict 50-step action chunks, with a 64-D motion token and 2 hand controls in each 66-D action.

THREE ARCHITECTURES, ONE FAMILY

HuGe-OFTParallel action regression

An MLP predicts continuous action chunks from action-token hidden states.

HuGe-GR00TFlow matching + cross-attention

An action transformer generates motion conditioned on vision-language features.

HuGe-PiLayer-wise action conditioning

A flow-matching action expert attends to multiple backbone layers.

Qwen3-VL-2B-Instruct64-D motion token + 2 hand controls34-D proprioception50-step chunks at 50 Hz

03 / EGO DATA FOR PRE-TRAINING

A recipe built
through three questions.

First establish how to train. Then ask what human experience should contain—and what happens as it grows.

Shared evaluation: one multi-task policy per variant, 100 demonstrations per task, and 60 evaluation episodes per task across three seed groups of 20. Success rates are unweighted averages over seven HumanoidArena tasks.

EXPLORATION 01 / TRAINING STRATEGY

How much of the
backbone should adapt?

Before introducing egocentric pre-training, we compare Frozen, LoRA, and Full training on HumanoidArena. Each starts from the pre-trained Qwen3-VL backbone.

  • FrozenKeep the vision-language backbone fixed.
  • LoRAAdapt the backbone with low-rank updates.
  • FullFine-tune the backbone and action modules.
WHAT WE CARRY FORWARD

Full fine-tuning wins across all three variants. It exceeds LoRA by 19.29–38.81 percentage points, so we adopt Full in the following experiments.

Choose the training strategy

AVERAGE SUCCESS (%) ↑
No HuGe stage pre-training in this comparison; the VLM is pre-trained.
VariantFrozenLoRAFull
HuGe-OFT0.0016.1955.00
HuGe-GR00T24.0534.0558.33
HuGe-Pi30.2440.9560.24

The action modules are trained in every setting. The low-level SONIC controller remains fixed.

With Full selected, what should ego data provide?
EXPLORATION 02 / PSEUDO-PROPRIOCEPTION

Can human experience
provide a robot state?

Human video does not contain the corresponding robot proprioception. Simulated G1 playback reconstructs a 34-dimensional state to accompany egocentric demonstrations.

29joint positions2hand states3gravity components

We vary whether these states are provided during egocentric pre-training, then evaluate downstream control with robot-state inputs.

WHAT WE LEARN

Robot-state context helps GR00T and Pi. OFT declines; the paper associates this with truncation of its textual state representation.

Add pseudo-proprioception

AVERAGE SUCCESS (%) ↑
Pseudo-proprioceptive states during egocentric pre-training.
VariantWithoutWithChange
HuGe-OFT56.4354.05−2.38 pp
HuGe-GR00T61.6766.67+5.00 pp
HuGe-Pi62.1465.00+2.86 pp
POSTURE-SENSITIVE EXAMPLE / SITSOFA

61.67 → 78.33%HuGe-GR00T

68.33 → 83.33%HuGe-Pi

With state context, does more experience help?
EXPLORATION 03 / SCALING EGOCENTRIC DATA

What changes when
experience scales?

We compare no HuGe pre-training, 10% of the egocentric pre-training corpus, and the full corpus, measuring transfer to downstream humanoid control.

HuGe-GR00T and HuGe-Pi improve at each evaluated scale. With the full corpus, they reach 66.67% and 65.00% average success.

SCALING TREND

More egocentric experience benefits the two flow-matching variants. OFT falls at the largest scale, consistent with the textual-state truncation issue.

These are observed trends at three evaluated scales; the study does not fit a mathematical scaling law.

Average success (%)

Select a model to highlight
Egocentric pre-training scale and downstream success Three equally spaced experimental settings: no HuGe pre-training, 10%, and 100% of the egocentric corpus. Average success rates: HuGe-GR00T, 58.33, 64.05, 66.67%; HuGe-Pi, 60.24, 62.14, 65.00%; HuGe-OFT, 55.00, 55.67, 54.05%. The vertical axis runs from 50 to 70%. Lines connect measured points and are not a fitted scaling law.
Equal spacing represents experimental settings, not proportional data volumes. “No pre-train.” means no HuGe stage pre-training; the backbone starts from Qwen3-VL.

Exact values at each scale

Average success (%). The three columns are experimental settings, not equally spaced data volumes.
VariantNo pre-train.10%100%
HuGe-OFT55.0055.6754.05
HuGe-GR00T58.3364.0566.67
HuGe-Pi60.2462.1465.00
FROM EXPLORATION TO COMPARISON

Now evaluate the complete recipe.

Full fine-tuning · pseudo-proprioceptive pre-training · full egocentric corpus
04 / COMPARISON WITH EXISTING BASELINES

The complete recipe.
Against existing baselines.

Building on the three investigations, we compare HuGe with existing policies across seven HumanoidArena tasks.

HUMANOIDARENA · OVERALL SUCCESS
66.67%

HuGe-GR00T

+12.86 percentage points over the strongest listed baseline, π0.5.

HuGe-Pi reaches 65.00%. One multi-task policy per variant is evaluated across seven tasks.

Evaluation protocol

100 training demonstrations per task. 60 evaluation episodes per task: three seed groups of 20. Overall is the unweighted mean across seven tasks.

Compare success rates

Unweighted average across all 7 tasks

HuGe models Baselines † Reported by HumanoidArena

All task-level results

Success rate (%). † Results retrieved from the original HumanoidArena paper; other results evaluated in this work.
ModelFootballDoubleDeskP&PBoxOpenDoorSitSofaBoxingVisNaviOverall
ACT †20.0030.0060.0058.3021.7058.3028.3039.51
DP †53.3035.000.0083.3083.3021.7050.0046.66
FM †21.7020.000.0076.7048.3018.3038.3031.90
π0.5 †10.0030.0075.0085.0070.0071.7035.0053.81
GR00T (scratch)0.000.0075.0011.6713.3368.3336.6729.29
GR00T (N1.7)13.330.0076.678.3375.0011.6726.6730.24
HuGe-OFT48.330.0053.3375.0055.0073.3373.3354.05
HuGe-GR00T71.6715.0081.6795.0078.3368.3356.6766.67
HuGe-Pi63.3311.6776.6788.3383.3368.3363.3365.00

Performance varies by task: DoubleDesk remains challenging for all HuGe variants (0–15% success). HOI comprises Football, DoubleDesk, and P&PBox; HSI comprises the remaining four tasks.

05 / EGO DATA FOR DOWNSTREAM TASKS

A second role for ego data.
Learning the task at hand.

Pre-training builds transferable experience. Can task-specific human demonstrations also help when real robot data is limited?

We move to a real Unitree G1, hold the robot budget at 110 demonstrations, and add 400 human demonstrations of cola pick-and-drop. First, we study the visual gap. Then, we measure full-task success.

DOWNSTREAM STUDY 01 / VISUAL ADAPTATION

The same experience.
A new embodiment.

Human hands become robot hands. The task, motion, and supervision stay aligned.

For downstream cola pick-and-drop, visual adaptation inpaints human arms and hands and composites rendered robotic counterparts, while preserving actions, states, and language.

ApproachPick upDrop
Human observation ORIGINAL
Robot embodiment ADAPTED

Paired frames from the visual adaptation figure in the paper.

Do adapted observations look more like robot experience?

FRÉCHET DISTANCE TO REAL ROBOT FEATURES ↓

Wan2.2

2.0311 1.3499

33.54% lower

VideoMAE V2

7.5697 5.5204

27.07% lower

V-JEPA 2

305.5143 263.2697

13.83% lower

Raw → visually adapted ego observations. Distances decrease within all three feature spaces; their absolute scales differ.

FEATURE DISTRIBUTIONS

Bridging the visual gap.

One shared projection.
Real robot experience as the reference.

Raw ego + real robot

Visually adapted + real robot

Click a group to show / hide

Hover or select a point to inspect a sample.

FRÉCHET DISTANCE ↓

To real robot features

Wan2.2 · original feature space

Raw ego
2.0311
Visually adapted
1.3499
33.54%lower after adaptation
Each point is a measured feature sample, L2-normalized and projected with a PCA fit shared by all three groups. The before / after view uses identical axes and repeats the blue robot samples as a common reference. Density contours show Gaussian KDE (bandwidth 0.4), relative to each group’s peak. Reported Fréchet distances are measured in the original feature spaces, not in this 2D projection. Projection method ↗

DOWNSTREAM STUDY 02 / TASK FINE-TUNING

Does human task experience
improve robot control?

“Walk forward, grab the cola
and throw into the trash bin”

UNITREE G1 + INSPIRE HANDS

More human experience.
Better task completion.

With 110 robot demonstrations held fixed, adding 400 human demonstrations improves HuGe-Pi on a whole-body pick-and-drop task. Visually adapting those human observations adds two successful trials out of 30.

30 trials per condition · randomized bottle placement · HuGe-Pi only

Robot data only43.3% 13/30
+ Raw human experience60.0% 18/30
+ Visually adapted experience66.7% 20/30

The two augmented conditions use the same 400 human demonstrations, with raw or adapted visuals.

Real-world experiment figure with the G1 setup, varied cola placements, success rates and sequential photographs of the robot completing the task.
Approach → pick up → crouch → throw into the bin.
Full experiment figure from the paper.
06 / REAL-WORLD DEMONSTRATIONS

One task.
Different conditions.

From a knocked-over bottle to different tabletop placements: three real-world rollouts reveal how the policy adapts and coordinates its body.

SHARED INSTRUCTION

“Walk forward, grab the cola and throw into the trash bin”

HuGe-Pi · Unitree G1
01 / OBJECT DISTURBANCE10.6 s

Recover after
a disturbance.

The bottle is deliberately knocked over during grasping—a perturbation absent from fine-tuning data. The robot re-grasps it and completes the task.

WATCH FOR / RECOVERY

Bottle tips → re-grasp → successful drop

02 / TABLETOP PLACEMENT A10.0 s

Adapt to the
bottle position.

At one tabletop placement, the robot approaches, picks up the bottle, and turns toward the bin. Compare its approach and reach with the next rollout.

WATCH FOR / SPATIAL ADAPTATION

Approach → reach and grasp → turn

03 / TABLETOP PLACEMENT B12.5 s

Coordinate the
full sequence.

With the bottle at another tabletop position, the robot links pickup, transport, crouching, and release through coordinated arm and lower-body motion.

WATCH FOR / WHOLE-BODY CONTROL

Carry the bottle → crouch → release

These rollouts provide qualitative evidence of adaptation to an object disturbance and spatial generalization across the evaluated bottle placements. Reported success rates use 30 evaluation trials per data configuration.

07 / BUILD ON HUGE

Experience worth
building on.

If HuGe supports your research,
please consider citing our work.

Read the paperExplore the repository Contact the authors
BIBTEX
@article{liu2026huge,
  author    = {Yang Liu and Jiacheng Liu and Hengtao Li and Yuheng Huang and Zijian Yang and Ji Dai and Maoxin Wang and Yi Gao and Qishen Wang and Baohua Yin and Peiyang Li and Shuanghao Bai and Mingyang Sun and Pengxiang Ding and Jianqin Yin and Lixin Yang and Donglin Wang},
  title     = {{HuGe}: Scaling Humanoid Generalist Vision-Language-Action Models with Egocentric Experience},
  journal   = {CoRR},
  volume    = {abs/2610.34186},
  year      = {2026},
  url       = {https://arxiv.org/abs/2610.34186}
}
arXiv:2610.34186Download .bib