Psi-Zero (Ψ₀): Loco-Manipulation VLA Pipeline with 80 Demos

When 80 Demos Beats 10× More Data

Summer 2026 brought a surprising paper to the robotics community: USC Physical Superintelligence (PSI) Lab published Ψ₀ (Psi-Zero) at Robotics: Science and Systems (RSS) 2026 — winning the Best Paper Award at the 3D-LLM/VLA Workshop at CVPR 2026.

What stood out wasn't the model size or special hardware. It was the number: with only 80 demonstrations per task, Ψ₀ beats GR00T N1.6 (NVIDIA's state-of-the-art humanoid foundation model) by over 40% success rate, while using more than 10 times less training data.

In an era where AI robotics races toward massive datasets — more data usually wins — this seems impossible. So how does Ψ₀ pull it off?

This guide walks through the complete pipeline: from architectural design rationale, installation, teleoperation data collection, 3-stage training, to deployment on real hardware.

What Is Loco-Manipulation and Why Is It Hard?

Locomotion (movement) and manipulation (object interaction) are typically studied separately. A robot can learn to walk well, or learn to grasp objects well — but doing both simultaneously in a long sequential chain is an entirely different challenge.

A typical example in Ψ₀: the "Pick bottle and pour into cup" task requires the robot to sequentially:

1. Walk to the bottle's location
2. Squat down to reach it
3. Grasp the bottle, rotate off the cap
4. Walk to the cup
5. Pour the liquid

Errors accumulate across 5 steps. GR00T N1.6 with hundreds of demos only achieves 30-40% on these compound tasks. Ψ₀ achieves ~65% with just 80 demos.

The key insight: Ψ₀ doesn't win by having more data — it wins by having the right kind of data and learning in the right order. This is the "scaling the right data" philosophy vs. "scaling data."

Triple-System Architecture: Brain, Muscle, Reflex

Ψ₀ uses a 3-tier architecture called System-0 / System-1 / System-2, inspired by Daniel Kahneman's dual-process theory of cognition:

System-2: Vision-Language Backbone (Slow Thinking)

• Model: Qwen3-VL-2B-Instruct (fine-tuned)
• Role: Understands language instructions and egocentric head camera images
• Mechanism: Autoregressive next-token prediction on human egocentric videos

System-2 is the "high-level brain." It doesn't issue specific joint commands — instead it creates rich visual-action representations from hundreds of hours of human manipulation video.

System-1: Action Expert (Fast Thinking)

• Model: Multimodal Diffusion Transformer (MM-DiT), ~500M parameters
• Role: Predicts whole-body action chunks (29-36 DoF)
• Key feature: Flow-matching (like Stable Diffusion 3), not autoregressive

System-1 is the "muscle" — receives embeddings from System-2 and outputs exact joint angles in real time.

Why MM-DiT beats Naive DiT? MM-DiT uses dual-modulation with joint attention between language/vision tokens and action tokens — allowing the two modalities to deeply communicate rather than being naively concatenated.

System-0: RL Tracking Controller (Execution Reflex)

• Model: AMO (Autonomous Motion Optimizer) — RL policy
• Role: Lower-body stabilization for locomotion
• Mechanism: Receives high-level velocity commands from System-1, handles actual walking/balancing

System-0 is the "spinal reflex" — it handles locomotion so System-1 can focus entirely on manipulation. This explicit division of labor is what most VLA papers miss.

For a deeper architectural analysis, see detailed Ψ₀ architecture breakdown.

Why Decoupled Training Beats Joint Training

Many current VLAs co-train on mixed human + robot data simultaneously. Ψ₀ rejects this approach — here's why:

Humans and robots have fundamentally different kinematics: humans have 48 DoF task-space actions while the G1 only has 29 DoF. Co-training on this mixed data confuses the model between two distributions and it doesn't learn either well.

Ψ₀'s solution — learn separately, in order:

1. Pre-train System-2 on human videos → semantic understanding
2. Post-train System-1 on robot data → robot-specific control
3. Fine-tune the combined system on task-specific robot demos

Result: 80 demos per task is sufficient to acquire new skills — because System-2 already has a strong semantic foundation from hundreds of hours of human video.

Installation

Hardware requirements:

• Python 3.10
• NVIDIA GPU (RTX 3090/4090/5090, or A100 for full training)
• CUDA sm_120+ for training; inference works on RTX 3090

Step 1: Clone and Setup

# Install uv — much faster package manager than pip
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone repo
git clone [email protected]:physical-superintelligence-lab/Psi0.git
cd Psi0

# Create virtual environment with Python 3.10
uv venv .venv-psi --python 3.10
source .venv-psi/bin/activate

# Install dependencies — skip LFS to avoid downloading model weights immediately
GIT_LFS_SKIP_SMUDGE=1 uv sync --group serve --group viz --group psi

# Flash Attention — must use exactly this version
uv pip install flash_attn==2.7.4.post1 --no-build-isolation

Step 2: Verify Installation

python -c "import psi; print(psi.__version__)"

If it prints a version number without errors → environment is ready.

Data Collection: Teleoperation Setup

To fine-tune on a new task, you need to collect 80 demonstrations via teleoperation. USC PSI Lab uses the following setup:

Required hardware:

• Robot: Unitree G1 + Dex3-1 dexterous hands (7 DoF/hand)
• Headset: PICO 4 Ultra VR headset (tracks wrist position/orientation)
• Gloves: MANUS Quantum Metagloves (captures finger joint angles)
• Camera: Intel RealSense D435i head-mounted on robot (egocentric view)

Collection workflow:

1. Operator wears headset + gloves
2. Wrist tracker → IK solver → upper-body joint angles
3. Locomotion (walking, squatting) controlled separately via System-0 RL policy
4. Record: RGB frames + joint angles (36 DoF) + timestamps

Data format: LeRobot-compatible — use LeRobot tools to visualize and process trajectories. See LeRobot G1 whole-body guide to set up LeRobot toolchain.

Each task requires approximately 80 high-quality trajectories. Practical collection rate: about 1-2 hours for 80 demos, depending on task complexity.

No Unitree G1 hardware? Start with the SIMPLE simulator (USC PSI Lab's custom simulation environment) — use the finetune-simple-psi0.sh script below.

3-Stage Training Pipeline

This is the heart of Ψ₀. The three stages cannot be reordered — each stage builds on the foundation of the previous one.

Stage 1: Pre-Training System-2 on Human Video

Dataset:

• EgoDex: ~829 hours of egocentric human video, 48 DoF task-space actions
• Humanoid Everyday (HE): ~31 hours of G1/H1 robot video

Tokenizer: FAST (Frequency-based Action Sequence Tokenizer)

• Compresses 48 DoF → 20 tokens, L1 loss = 1.95×10⁻⁴
• Objective: autoregressive next-action token prediction — like GPT but predicting movement tokens instead of words

Training commands:

# Pre-train on EgoDex (200K steps)
bash scripts/train/psi0/pretrain-egodex-psi0-fast.sh

# Pre-train on Humanoid Everyday (30K steps)
bash scripts/train/psi0/pretrain-he-psi0-fast.sh

Compute: 64 A100 GPUs, 10 days, 230K steps.

Don't have 64 A100s? Use the pre-trained checkpoint from HuggingFace: USC-PSI-Lab/psi-model. Start from Stage 2 or Stage 3 directly.

Stage 2: Post-Training System-1 (Action Expert)

Once System-2 has strong representations, train System-1 to learn robot-specific control.

Dataset: Humanoid Everyday — 3M frames, 36 DoF joint space (Unitree G1)

Architecture details:

• MM-DiT with dual-modulation (not standard DiT)
• Joint attention between VL tokens and action tokens
• Flow-matching objective instead of DDPM noise prediction — faster and more stable

# Post-train action expert on humanoid data
bash scripts/train/psi0/posttrain-he-psi0.sh

Compute: 32 A100 GPUs, 30 hours, 30K steps (batch size 2048).

Stage 3: Fine-Tuning on Task Data (80 Demos)

This is the stage you actually run yourself with the 80 demos you've collected.

# Replace $task with your task name
export task=Pick_bottle_and_turn_and_pour_into_cup

# Fine-tune on real robot data
bash scripts/train/psi0/finetune-real-psi0.sh $task

# Or fine-tune on simulation data
bash scripts/train/psi0/finetune-simple-psi0.sh $task

Hyperparameters:

• 40K training steps
• Batch size: 128
• Learning rate: 1e-4 with cosine scheduler

Time estimate: On an RTX 4090, approximately 3-4 hours per task. Model typically converges around 20K steps.

Real-Time Chunking (RTC): Fixing Motion Jitter

A practical problem when deploying VLAs: inference latency causes jitter in robot motion. When the model needs 100ms to predict the next action chunk, the robot jerks to a stop every 100ms.

Ψ₀ solves this with Real-Time Action Chunking (RTC): during training, mask the first d∈[1,6] action tokens from the loss calculation. This teaches the model to "skip past" the startup phase when it needs to execute an action chunk immediately.

Practical effect: noticeably smoother robot motion with no jitter between action chunks.

Inference and Deployment

# Serve model with RTC mode
bash ./scripts/deploy/serve_psi0-rtc.sh

# Or with explicit parameters:
uv run --active --group psi --group serve serve_psi0 \
  --host 0.0.0.0 \
  --port 22085 \
  --run-dir=$run_dir \
  --ckpt-step=$ckpt_step \
  --action-exec-horizon=24 \
  --rtc

Deploy the client on the robot:

bash ./real/scripts/deploy_psi0-rtc.sh

Key parameters:

• --action-exec-horizon=24: number of action steps to execute before querying the model again
• --rtc: enables Real-Time Chunking mode
• --port 22085: default PSI inference server port

Inference hardware: RTX 3090 or 4090 is sufficient. A100 is not required for deployment.

For network deployment (robot communicating with a remote inference server), network latency below 10ms typically works well with horizon=24.

Results: The Numbers

Ψ₀ was evaluated on 8 real-world long-horizon loco-manipulation tasks, each comprising 3-5 sequential sub-tasks (grasping, pouring, rotating, walking, squatting, carrying, pushing, pulling):

Ablation study on task "Pick bottle and pour":

The ablation shows every stage is essential — remove any one and performance drops dramatically.

Insights: What Experts Notice That Beginners Miss

1. "Scaling the right data" matters more than "scaling data"

Ψ₀ uses 10× less data but wins because of data quality and proper curriculum ordering. The lesson for everyone: before collecting more data, check whether existing data has the right distribution first.

2. Decoupled learning beats joint training when domain gap is large

Humans and robots have fundamentally different kinematics. Co-training forces the model to handle two distributions simultaneously. Separating learning stages allows each phase to specialize at the right level of abstraction.

3. System-0 (RL locomotion) is the unsung architectural hero

Most VLA papers assume locomotion stability and don't model it explicitly. Ψ₀ makes System-0 a first-class architectural component — this lets System-1 focus entirely on manipulation without needing to "think about" balance.

4. 80 demos is a sweet spot, not a magic number

80 high-quality, diverse demos (varied viewpoints, lighting conditions, starting positions) beats 500 monotonous demos. Fine-tuning works in Ψ₀ because the pre-trained foundation is strong — not because 80 is a magic number.

5. Comparing with GR00T N1.6 on data efficiency

GR00T N1.6 with Cosmos Reason requires hundreds to thousands of demos and still underperforms Ψ₀. This doesn't mean GR00T is bad — it means Ψ₀'s decoupled architecture is a genuine advance in data efficiency.

Common Pitfalls

1. Flash Attention version mismatch: Must use flash_attn==2.7.4.post1. Other versions cause hard-to-debug CUDA kernel errors.
2. CUDA compute capability: Needs sm_120+ for full training. RTX 3090 is sm_86 — fine for inference, may be limited for some training kernels.
3. LFS pointer trap: Use GIT_LFS_SKIP_SMUDGE=1 when cloning to avoid downloading model weights unnecessarily — fetch from HuggingFace when needed.
4. Insufficient demo diversity: If all 80 demos are recorded in the same lighting/viewpoint, the model will overfit. Use at least 3 lighting conditions and 2 different starting positions.
5. Motion jitter: If the robot jerks, try reducing --action-exec-horizon from 24 to 16.
6. Task naming: Task names must use underscores not spaces (e.g., Pick_bottle_and_pour not Pick bottle and pour).

Resources

• GitHub: physical-superintelligence-lab/Psi0 — Apache 2.0
• Paper: arXiv 2603.12263 — RSS 2026
• Model weights: huggingface.co/USC-PSI-Lab/psi-model
• Dataset: huggingface.co/datasets/USC-PSI-Lab/psi-data
• Project page: psi-lab.ai/Psi0

Related Posts

• Psi-Zero Series 1: Overview of the Open Humanoid Foundation Model
• OpenWBC: VR Teleop + Whole-Body VLA Guide for Unitree G1
• LeRobot + Pi0 Fast: Whole-Body VLA for Humanoid G1