UCAG-P: One VLA Policy for Every Robot (98.3% LIBERO)

Imagine training an AI that can control a single-arm robot, a bimanual humanoid, and even a human hand — all from one single checkpoint, with no per-embodiment fine-tuning required. That's the problem UCAG-P solves.

On August 26, 2026, the Xiaomi Embodied Intelligence Team and University of Macau released "One Policy, Many Embodiments: Unified Camera-Centric Action Geometry Pre-training for Heterogeneous Embodied Manipulation" — a VLA framework achieving 98.3% on LIBERO and 89.2% on RoboTwin Hard using a single unified checkpoint without any benchmark-specific fine-tuning.

The Problem: Why Is Cross-Embodiment Hard?

Before diving into UCAG-P, it's worth understanding why training a single policy for multiple robot types is fundamentally difficult.

Each robot type has a completely different action space:

• Single-arm robots: actions are 6–7 joint angles or Cartesian velocity values
• Bimanual humanoids: two arms with 12–14 degrees of freedom each, plus hands
• Dexterous hands: tens of degrees of freedom across all fingers

When you try to merge data from all these embodiments, the fundamental question is: which action space serves as the common language? If you use a robot arm's joint angles, a human hand doesn't have those joints. If you use Cartesian coordinates, you still need a different mapping for each embodiment — and these mappings are far from trivial.

Most prior approaches solve this by:

1. Training separate models for each embodiment → wastes resources, no transfer learning
2. Using a fake "common" action space (e.g., residual actions) → loses embodiment-specific information
3. Training only on robot data → ignores the massive corpus of human hand video demonstrations

UCAG-P proposes a fundamentally different answer: instead of finding a common action space, find a common geometric space that the camera can observe.

Core Idea: If the Camera Can See It, Use It

Regardless of whether it's a robot arm, a humanoid, or a human hand, a wrist-mounted or head-mounted camera will always observe the motion of the wrist tip and fingertip. This is exactly what UCAG-P exploits.

Instead of predicting joint angles or Cartesian velocity, UCAG-P predicts the motion of two anchor points in camera space:

• p₀: The wrist point or end-effector tip
• p₁: The gripper center or grasp center

These two points are always observable from the camera, whether it's a robot arm, a humanoid, or a human hand. A separate translator module then converts this geometric motion into embodiment-specific control commands.

This is a simple but powerful insight: camera-observable geometry serves as the bridge between embodiments.

UCAG-P Architecture: Three Components

UCAG-P consists of three main components:

1. Vision-Language Backbone (Qwen3-VL-4B-Instruct)

UCAG-P uses Qwen3-VL-4B-Instruct as its backbone to encode multi-view RGB observations and language instructions. A key addition is learnable action-query tokens that extract compact features for action prediction, keeping the backbone from attending to the full visual context on every step.

The backbone processes:

• Multi-view RGB images from wrist and head cameras
• Language instructions as text tokens
• Outputs: hidden states + action-query features

2. Shared Camera-Centric Motion Head

This is the heart of UCAG-P — a shared motion head that predicts geometric action chunks for all embodiments.

The action vector has 30 dimensions per timestep:

• Left manipulator motion (10D): 3D displacement of p₀, 3D displacement of p₁, in-plane rotation, gripper state
• Right manipulator motion (10D): same structure
• Camera motion (10D): ego-motion of the camera

All displacements are measured relative to the first frame in the camera coordinate system — this ensures consistent representation across embodiments.

The motion head is trained with geometric supervision whenever camera-centric labels are available. Crucially, both robot data and human hand data can provide these labels — robots via forward kinematics, human hands via hand keypoint detection.

3. Geometry-Conditioned Action Translator

This is the "translation" module from geometric motion to embodiment-specific control commands. The translator receives:

• Predicted camera-frame motion (from the motion head)
• Camera-to-base transforms (from calibration)
• Local Jacobians (robot-specific kinematics)
• Geometry tokens: vectors encoding the embodiment structure

Geometry tokens tell the translator whether it's working with a robot arm, a humanoid, or a human hand. For robot arms, the translator applies Jacobian-based inverse kinematics to convert Cartesian motion to joint velocity. For humanoids, it processes both arms simultaneously.

Three-Stage Training Pipeline

UCAG-P doesn't train end-to-end in one shot. It follows a carefully ordered three-stage pipeline:

Stage 1: Camera-Centric Specialization

Train the VLM backbone + motion head on all data with geometric supervision. The goal: teach the model to predict camera-centric anchor motion accurately, regardless of embodiment.

At this stage, the translator is not involved — the model only learns "observe scene + instruction → predict p₀, p₁ motion in camera space."

Stage 2: Geometry-Conditioned Translation

Train the translator using ground-truth camera trajectories and executable command labels. This stage isolates the learning of the mapping from geometric motion to embodiment-specific commands.

The translator learns: "given this geometric motion and this robot's Jacobian → these joint velocities."

Stage 3: Joint Optimization

Jointly train the motion head + translator on mixed robot-human data. This mirrors inference-time conditions — the motion head predicts, the translator converts, and both are updated together.

The three-stage design matters: training everything from scratch simultaneously would allow gradients from the translator to corrupt the geometric representation being learned by the motion head.

Training Data: 6,373 Hours from 11 Datasets

The scale of UCAG-P's training data is impressive:

The standout number is 36.7% from human hand data — not robot data at all. This is the key advantage of the camera-centric approach: egocentric hand videos from EgoVerse, EgoDex, and VITRA can be incorporated directly without teleoperation or complex annotation.

In the UCAG-P framework, human hands are treated as just another embodiment within the same geometric space. Human wrist keypoint → p₀, thumb-index midpoint → p₁. Their motion in camera frame is used identically to robot data.

Benchmark Results: State of the Art Across All Tests

Single checkpoint, no benchmark-specific fine-tuning:

LIBERO (98.3% average):

Other benchmarks:

The most notable number is 82.0% zero-shot on LIBERO-Plus — a benchmark testing generalization with distractor objects and modified backgrounds. UCAG-P was never trained on LIBERO-Plus yet achieves 82%, demonstrating that camera-centric representation learns genuine geometric understanding, not just memorization.

On RoboTwin Hard (dual-arm manipulation with randomized distractor objects), UCAG-P achieves 89.2% — significantly higher than generalist baselines using native action spaces.

Real-World Results on Piper Robots

The authors also evaluate on physical Piper robots across three tasks:

Drawer opening at 90% is particularly impressive — this task requires precise contact and force management, which is harder than picking up free objects. This validates that the geometric representation is accurate enough to handle contact-rich manipulation.

Cross-Embodiment Transfer

One of the most important tests: train on ALOHA, test zero-shot on ARX robot.

ALOHA and ARX are two bimanual robots with different kinematics — different workspace, different joint limits, different mechanical design. UCAG-P achieves 35% zero-shot success on ARX after training only on ALOHA data.

35% might sound low, but this is zero-shot — no ARX data in training whatsoever. A baseline using ALOHA's native joint angles would fail completely since the joints are incompatible. This demonstrates that camera-centric representation genuinely learns transferable manipulation geometry.

Human-to-robot transfer also works: UCAG-P can learn from human hand videos and transfer to a robot arm at 2× the data efficiency compared to using only robot demonstrations.

Limitations and Future Directions

The authors honestly acknowledge several limitations:

1. Camera calibration dependency: The geometry translator requires accurate camera-to-base transforms. If calibration is off, the translation is off too. This is a significant practical concern for deployment.

2. Hand keypoint detection quality: For human data, accurate wrist and finger keypoint detection is required. Dark videos or bad viewing angles cause keypoint failures → incorrect labels.

3. Extreme morphology mismatch: While the geometry translator helps, robots with very different kinematics (e.g., delta robots vs SCARA) remain challenging. 35% zero-shot is good but not yet production-ready.

4. Code not yet released: At the time of writing, code, model weights, and training scripts are not yet available. The authors have committed to releasing them at GitHub.

Comparison with Other Cross-Embodiment Approaches

UCAG-P isn't the first cross-embodiment approach. A quick comparison:

UCAG-P's biggest advantage is leveraging human hand data — a massive corpus that almost every other cross-embodiment approach ignores. With 2,339 hours of EgoDex, EgoVerse, and VITRA, the model learns manipulation geometry from everyday egocentric videos without any teleoperation.

Preparing to Use UCAG-P

Code isn't released yet, but here's what to prepare:

Minimum hardware:

• GPU with 24GB+ VRAM for fine-tuning (RTX 3090/4090 or A100)
• Wrist camera at minimum 640×480 resolution
• Calibration setup to obtain accurate camera-to-base transforms

Data pipeline (4 steps):

1. Episode indexing: Index all episodes by embodiment type
2. Visual/language standardization: Normalize image formats, tokenize instructions
3. Geometric alignment: Compute camera-centric anchor trajectories (p₀, p₁)
4. Temporal windowing: Sliding window with masking for unavailable labels

Check out this guide on fine-tuning VLAs on LIBERO to get familiar with the training infrastructure before UCAG-P code drops.

Conclusion

UCAG-P represents an important paradigm shift in robot learning: instead of trying to align action spaces (which are hardware-specific), align through geometric observation (which the camera always provides).

The results — 98.3% LIBERO, 89.2% RoboTwin Hard, 82.0% zero-shot LIBERO-Plus from a single checkpoint — validate that camera-centric geometric representation is the right inductive bias for generalist robot policies.

But the most exciting insight isn't the benchmark numbers. It's the design philosophy: a human hand is just another embodiment. Once you erase that boundary, you unlock the ability to use millions of hours of ordinary egocentric video to train robots. That's a data leverage no other current cross-embodiment method has.

Paper: arXiv 2608.26058 — Project page: public-bots.github.io/UCAG-P

Related Posts

• DyPeS: Cross-Embodiment VLA with Dynamics Prior and MoE
• HEX-VLA: Cross-Embodiment Policy for Humanoid Whole-Body Control
• Fine-tuning VLA on LIBERO with Embodied-R1.5