Series

Vima Humanoid Manip

The "Vima Humanoid Manip" series has 5 parts — read them in order from part 1.

vimacross-attentiontransformerPart 1

VIMA Architecture: Cross-Attention Transformer Explained

Decode VIMA's cross-attention architecture: how T5 encodes multimodal prompts and XAttn GPT generates motor commands for 17 robot manipulation tasks.

6/16/202610 min read

Tutorial

VimaBench: 17 Tasks and 4-Level Generalization Protocol

vimavimabenchbenchmarkPart 2

manipulation

VimaBench: 17 Tasks and 4-Level Generalization Protocol

Install VimaBench, run the 200M checkpoint demo, and analyze all 4 eval partitions — from placement to novel task generalization. Which level matters most for humanoid deployment?

6/16/202615 min read

Deep Dive

Object Tokenizer: From Raw Pixels to Object Tokens with Mask R-CNN + ViT

vimaobject-tokenizermask-rcnnPart 3

manipulation

Object Tokenizer: From Raw Pixels to Object Tokens with Mask R-CNN + ViT

A deep dive into VIMA's object tokenizer: Mask R-CNN objects, ViT crop features, bbox MLP position signals, and T5 prompt assembly.

6/17/202615 min read

Deep Dive

vimadatasetimitation-learningPart 4

manipulation

Dataset 650K: Collecting Large-Scale Multi-Task Manipulation Data for VIMA

Deep dive into VIMA's 650K trajectory dataset: procedural generation in PyBullet, pkl file format, HuggingFace download, and why just 1% of the data beats all baselines.

6/17/202612 min read

Deep Dive

vimahumanoidunitree-g1Part 5

humanoid

Humanoid Adaptation: Extending VIMA to High-DoF Humanoid Robots

VIMA series capstone: adapt from tabletop 6-DoF to Unitree G1 23-DoF — freeze backbone, swap action head, bridge data gap with teacher-student and LoRA fine-tuning.

6/17/202619 min read