HoloMotion-1: Sparse MoE Foundation Model for Whole-Body Humanoid

What if a single AI policy could make a humanoid robot dance, perform martial arts, crawl across the floor, and stand up from a seated position — all without ever seeing a single frame of real-robot data during training?

That's exactly what HoloMotion-1 achieves. In May 2026, Horizon Robotics (one of China's leading AI chip and robotics companies) published the HoloMotion-1 Technical Report introducing a Sparse Mixture-of-Experts (MoE) foundation model for whole-body humanoid motion tracking. The system learns from millions of frames of human video, then transfers zero-shot to a Unitree G1 humanoid robot — no fine-tuning, no real-world data collection required.

The numbers speak for themselves: 40% lower tracking error compared to the previous state-of-the-art (SONIC), at 2-3× higher inference speed.

The Core Problem: Why Whole-Body Control Is Hard

Imagine teaching a robot to dance. You can't just say "raise your right arm" — you need to coordinate 29 joints simultaneously: upper body, lower body, hands, feet, head, and center of mass. If any part falls out of sync, the whole motion collapses.

For a humanoid like the Unitree G1 with 29 degrees of freedom (DoF), the challenges compound:

Action space explosion: 29 joints, each with position, velocity, and torque targets — the action vector easily reaches hundreds of dimensions.

Dynamic coupling: Moving a leg shifts the hip, which propagates through the torso, affecting the arms. Everything is interconnected through rigid-body physics.

Sim-to-real gap: Policies trained in simulation routinely fail on real hardware due to actuator delays, friction differences, and imperfect physics models.

Data scarcity: High-quality motion capture (MoCap) data is expensive and time-consuming to collect at scale.

Previous methods like NVIDIA's SONIC addressed parts of this problem but were fundamentally limited by MoCap data volume and simpler policy architectures.

What Is HoloMotion-1?

HoloMotion-1 is a motion foundation model — a general-purpose policy for whole-body humanoid motion tracking — developed by a team at Horizon Robotics: Maiyue Chen, Kaihui Wang, Bo Zhang, Xihan Ma, Zhiyuan Yang, Yi Ren, Qijun Huang, Zihao Zhu, Yucheng Wang, and Zhizhong Su.

The key differences from prior work:

Architecture: Sparse MoE Transformer + KV-Cache

This is where HoloMotion-1 gets technically interesting. It's not just a bigger Transformer — it's a fundamentally different architecture designed for real-time robot control.

What Is Sparse MoE?

In a standard Dense Transformer, every input token passes through all layers — computationally uniform but wasteful, since many operations are irrelevant for any given input.

Mixture-of-Experts solves this by organizing expert sub-networks, each specializing in different patterns. A router selects the top-K most relevant experts per token, rather than using the full model.

HoloMotion-1's Sparse MoE design has ~400M total parameters but activates only ~7M parameters per control step (~1.7%). This is why it achieves 200-300 FPS on the G1's onboard ARM64 hardware — extraordinary for a model of this scale.

Input: joint state + target motion sequence
        ↓
Token embedding + positional encoding
        ↓
MoE Transformer blocks:
  [Router] → select top-K experts
  [Expert FFNs] → process in parallel
  [Sparse combine] → merge outputs
        ↓
Action head: joint position targets (29 DoF)
        ↓
PD controller → torque commands → robot actuators

KV-Cache for Real-Time Inference

The policy operates on motion sequences — not just the current frame but temporal history across many steps. A naive Transformer would recompute full attention at every step: O(T²) cost that becomes prohibitive at robot control frequencies.

HoloMotion-1 integrates KV-cache: keys and values from prior steps are cached, and each new step only computes attention for the new token before concatenating with the cache. This technique — standard in LLM inference — is applied here to robot motion control at scale for the first time.

Result: per-step latency drops from O(T²) to O(T), enabling sustained high-frequency control without sacrificing temporal context.

Data Pipeline: From YouTube to Robot G1

This is HoloMotion-1's "secret sauce." Instead of relying exclusively on expensive MoCap, the team built a hybrid motion corpus from three complementary sources:

Source 1: In-the-Wild Video (primary diversity)

MotionMillion — motions reconstructed from monocular in-the-wild video using SMPL-based pose estimation. This provides the bulk of behavioral diversity: walking, dancing, martial arts, climbing, everyday activities.

Limitation: lower quality than MoCap due to estimation noise. HoloMotion-1 uses a training strategy that's robust to this noise.

Source 2: Curated MoCap

AMASS and LAFAN1 — high-quality motion capture datasets with studio-grade precision. These provide accurate supervision for foundational motions, complementing the breadth of video data.

Source 3: In-House Motion Data

Horizon Robotics' proprietary data focuses on deployment-critical motions: teleoperation sequences, high-dynamic demonstrations, locomotion patterns. This bridges the gap to real-world robot usage.

Retargeting: SMPL → G1 Kinematics

All motion data arrives in SMPL format (a parametric human body model). Converting it to G1's robot skeleton requires HoloRetarget — a retargeting algorithm that runs at:

• 3,000+ FPS on RTX 4090 for offline training data generation
• 300+ FPS on-robot for real-time teleoperation

Retargeting handles differences in body proportions (a 1.7m human vs. 1.27m G1), DoF count, and joint range constraints.

Training in IsaacLab

The policy trains entirely in IsaacLab — NVIDIA's GPU-accelerated robotics simulation framework built on Isaac Sim/PhysX. IsaacLab enables:

• Thousands of parallel environments on a single GPU
• Realistic contact dynamics critical for locomotion
• Domain randomization for sim-to-real robustness

Training procedure:

1. State initialization: robot starts at a random frame within a motion clip
2. Reference motion: policy receives target joint positions from the reference clip
3. Action: policy outputs joint position targets; PD controller converts to torques
4. Reward: tracking error + smoothness + stability terms
5. Domain randomization: randomize mass, friction, motor delay to close the sim-to-real gap

The sequence-level training strategy trains the policy on long motion sequences rather than individual steps — this is what makes KV-cache inference so natural: the policy is already "thinking" in temporal sequences during training.

MuJoCo Evaluation: Numbers That Matter

While training uses IsaacLab, evaluation runs in MuJoCo for consistent benchmarking across methods. The primary metric is MPKPE (Mean Per-Keypoint Position Error) in millimeters — how far each body keypoint deviates from the reference motion.

HoloMotion-1 reduces error by 45% vs. SONIC while achieving 2-3× faster inference. This is rare in ML — usually accuracy and speed trade off. The MoE + KV-cache combination breaks that tradeoff.

Evaluation spans multiple out-of-domain benchmarks:

• AMASS (out-of-domain): diverse motions from test splits unseen during training
• HumanML3D: motions from multiple sources and capture methods
• LAFAN1: high-quality studio MoCap sequences

HoloMotion-1 leads on all benchmarks.

Zero-Shot Transfer to Real G1

The most impressive result: a policy trained entirely in simulation, deployed directly onto a Unitree G1 humanoid (29 DoF) — no fine-tuning, no real-world data collection, no domain adaptation tricks.

On real hardware, the policy runs at 200+ FPS — fast enough to react to unexpected disturbances and maintain balance through perturbations. The G1 successfully executes:

• Martial arts: high kicks, spinning techniques, combined arm-leg sequences
• Dance routines: full-body choreography synchronized across all joints
• Crawling: smooth transitions between standing and floor-level postures
• Sit-to-stand: natural transitions that stress-test hip and knee coordination

All using one policy — no mode switching, no loading different models for different behaviors.

Installation and Getting Started

System Requirements

# Hardware (training)
GPU: NVIDIA RTX 3090+ (minimum), A100 (recommended)
RAM: 32GB+
Storage: 100GB+ for datasets

# Software
Ubuntu 20.04 or 22.04
CUDA 12.1+
Python 3.10+
IsaacLab (for training)
MuJoCo 3.x (for evaluation)

Clone and Install

git clone https://github.com/HorizonRobotics/HoloMotion.git
cd HoloMotion

# Create conda environment
conda create -n holomotion python=3.10
conda activate holomotion

# Install IsaacLab first (see docs.omniverse.nvidia.com)
# Then install HoloMotion:
pip install -e .

Download Pre-Trained Models (v1.2)

HoloMotion v1.2 ships two ready-to-deploy models:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="HorizonRobotics/HoloMotion_assets",
    repo_type="dataset",
    local_dir="./pretrained"
)

Two models are available:

• Motion tracking model: follow a reference motion clip (MoCap or video-reconstructed)
• Velocity tracking model: command-based locomotion (forward speed, direction, turning rate)

Run Inference in MuJoCo

# Motion tracking (requires a reference motion clip)
python scripts/eval_mujoco.py \
    --model-path pretrained/holomotion_v1.2_motion_tracking.pt \
    --motion-file data/sample_motions/dance_clip.npz \
    --robot g1

# Velocity tracking (no reference needed — command-based)
python scripts/eval_mujoco.py \
    --model-path pretrained/holomotion_v1.2_velocity_tracking.pt \
    --vx 0.5 --vy 0.0 --omega 0.3 \
    --robot g1

Deploy to Real G1

# Ensure G1 is connected via Ethernet (default IP: 192.168.123.161)
python scripts/deploy_g1.py \
    --model-path pretrained/holomotion_v1.2_motion_tracking.pt \
    --motion-file data/deploy_motions/walk_forward.npz

Safety note: Always have a trained operator nearby with access to the emergency stop. Verify battery level >50% before running dynamic motions. Start with slow, low-amplitude reference clips before attempting high-dynamic demos.

Version Roadmap

One of HoloMotion's strengths is its transparent, publicly-shared development roadmap:

The roadmap reveals the long-term vision: a single foundation model that controls any humanoid robot, on any terrain, given any command input — analogous to how a single LLM can handle diverse language tasks.

Why This Matters for Robotics Engineers

HoloMotion-1 represents a paradigm shift in robot motion control:

Before: Write separate controllers for each task. Spend months tuning. Get narrow, brittle behavior that doesn't generalize.

With HoloMotion-1: One policy handles thousands of motions. Learns from human video. Deploys zero-shot. Per-task cost drops dramatically.

This mirrors what GPT-3 did to NLP: instead of training task-specific models for summarization, translation, Q&A, you fine-tune or prompt a foundation model. HoloMotion-1 is building toward that model for robotics.

Three ways to use it today:

1. You have a Unitree G1/H1: Download v1.2 pre-trained models and run velocity tracking immediately. No custom training needed.

2. You have motion data (MoCap or video): Use HoloRetarget to convert it to G1 skeleton format, then fine-tune on top of HoloMotion-1's pre-trained weights.

3. You're doing research: The MoE + KV-cache architecture is a strong template. Study how sequence-level training interacts with KV-cache inference — there are open questions here about optimal sequence length and caching strategies.

For practical whole-body control implementation on G1, see also cuRobo whole-body motion planning for G1. For a comparison with SONIC's architecture, see GROOT-N1 WBC architecture breakdown.

Paper Reference

HoloMotion-1 Technical Report — Maiyue Chen, Kaihui Wang, Bo Zhang, Xihan Ma, Zhiyuan Yang, Yi Ren, Qijun Huang, Zihao Zhu, Yucheng Wang, Zhizhong Su — Horizon Robotics, 2026

GitHub: HorizonRobotics/HoloMotion

Related Posts

• SONIC × GEAR: High-Quality Humanoid Whole-Body Control
• DyPeS-VLA: Cross-Embodiment with Dynamics Prior and MoE
• GROOT-N1 WBC Part 1: Architecture Overview