VnRoboVnRobo
AboutPricingBlogContact
🇻🇳VISign InStart learning
🇻🇳VI
VnRobo logoVnRobo logo

Learn Physical AI and experiment in your browser. Connect your fleet when you have real robots.

Product

  • Features
  • Pricing
  • Knowledge Base
  • Services

Company

  • About Us
  • Blog
  • Contact

Legal

  • Privacy Policy
  • Terms of Service

© 2026 VnRobo. All rights reserved.

Made with♥in Vietnam
VnRoboVnRobo
AboutPricingBlogContact
🇻🇳VISign InStart learning
🇻🇳VI
  1. Home
  2. Blog
  3. LeRobot Ecosystem: Complete Guide 2026
wholebody-vlaai-perceptionmanipulationtutorial

LeRobot Ecosystem: Complete Guide 2026

Overview of Hugging Face LeRobot — models, datasets, hardware support, and how to get started with $100.

Nguyen Anh TuanMarch 22, 20265 min readUpdated: Nov 14, 2026
LeRobot Ecosystem: Complete Guide 2026

What is LeRobot?

LeRobot is Hugging Face's open-source framework for robot learning — from data collection, model training, to real-world deployment. If you know Hugging Face Transformers for NLP, think of LeRobot as Transformers for robotics.

Launched in 2024, LeRobot quickly became the standard framework for imitation learning in the robotics community. Version 0.5.0 (2026) brings major improvements: humanoid support, 10x faster training, more models and hardware.

This complete guide covers LeRobot's architecture, supported models, hardware, and how to start with just $100.

LeRobot Architecture

code
┌────────────────┐
│   Data Layer   │ ← Collect demonstrations (teleoperation, human videos)
├────────────────┤
│   Model Layer  │ ← Pre-trained policies (ACT, DiffusionPolicy, BERT-style)
├────────────────┤
│  Training Loop │ ← Imitation learning on collected data
├────────────────┤
│ Deployment     │ ← Run policy on real robot
└────────────────┘

Key Features

1. Pre-trained Models

  • ACT (Action Chunking with Transformers): Sequences of actions
  • DiffusionPolicy: Action distribution (handles multi-modal actions)
  • VLA Models: Vision-Language-Action for language-conditioned tasks
  • BERT-style Encoders: For state representation

2. Supported Robots

Hardware out-of-the-box support:

  • Low-cost: GoBilda manipulator, TurtleBot 4 ($100-500)
  • Mid-range: Franka Panda, UR5 ($10K-50K)
  • High-end: ALOHA bimanual, Unitree H1 ($100K+)
  • Humanoids: Support added in v0.5.0

3. Dataset Format

Standardized RLDS (Reversible Language Data Serialization) format:

code
dataset/
├── episode_0/
│   ├── observations/
│   │   ├── image.npy          # Camera RGB (256x256x3)
│   │   ├── depth.npy          # Depth (256x256)
│   │   └── proprio.npy        # Joint angles [q1,...,q7]
│   ├── actions/
│   │   └── action.npy         # 7D action [dx, dy, dz, drx, dry, drz, gripper]
│   └── language_instruction.txt  # "Pick up the red cube"
├── episode_1/
...

Getting Started: Build a $100 Robot

Shopping List:

  • GoBilda manipulator: $80
  • Logitech USB camera: $20
  • Orange Pi 5: $60
  • USB-to-serial adapter: $5
  • Total: ~$165

Step 1: Collect Data via Teleoperation

python
from lerobot.data_collection import DataCollector

# Initialize
collector = DataCollector(
    robot_type="gobilda",
    resolution=(256, 256),
    fps=30
)

# Teleoperate: operator controls robot, policy learns
for episode in range(50):  # Collect 50 demonstrations
    collector.start_episode()
    
    while not collector.episode_done():
        # Read operator input (joystick/haptic device)
        operator_cmd = get_operator_input()
        
        # Execute on robot
        collector.step(operator_cmd)
        
        # Auto-save observations + actions
    
    collector.save_episode()

# Dataset saved in RLDS format

Step 2: Train Policy

python
from lerobot.models import ACTPolicy
from lerobot.training import train

# Load dataset
dataset = load_dataset("./data/my_robot_demos")

# Create policy
policy = ACTPolicy(
    hidden_size=256,
    num_layers=4,
    chunk_size=16  # Predict 16 future actions
)

# Train
train(
    policy=policy,
    dataset=dataset,
    batch_size=32,
    num_epochs=20,
    learning_rate=1e-4
)

# Save
policy.save("./models/my_policy")

Step 3: Deploy to Robot

python
from lerobot.inference import PushTPolicy

# Load trained policy
policy = PushTPolicy.from_pretrained("./models/my_policy")

# Run on robot
robot = GoBildaRobot()
obs = robot.get_observation()

while True:
    # Policy predicts next action
    action = policy(obs)
    
    # Execute
    robot.step(action)
    
    # Get new observation
    obs = robot.get_observation()

Advanced: Fine-tune Pre-trained Models

Instead of training from scratch, fine-tune Hugging Face's pre-trained model:

python
from lerobot.models import AutoPolicy

# Load pre-trained model (trained on 100K+ demonstrations)
policy = AutoPolicy.from_pretrained("lerobot/act-robot-v0.5")

# Fine-tune on your data (only 50 demonstrations!)
policy.train_mode()
for epoch in range(5):
    for batch in dataloader:
        loss = policy.compute_loss(batch)
        loss.backward()
        optimizer.step()

# Deploy
policy.eval()
action = policy(obs)

Ecosystem Tools

HubLab: Collaborative Training

Share datasets and models with community:

python
dataset.push_to_hub("username/my-robot-dataset")
policy.push_to_hub("username/my-robot-policy")

Benchmarking

Compare your policy against baselines:

python
from lerobot.benchmarks import benchmark

score = benchmark(
    policy=my_policy,
    dataset=eval_dataset,
    task="pick-and-place"
)
print(f"Success rate: {score:.1%}")

Performance

LeRobot policies achieve:

  • Pick-and-place: 85% success (with 100 demos)
  • Insertion tasks: 72% success
  • Bimanual coordination: 68% success

With domain randomization + sim-to-real, success rates improve to 90%+.

Limitations

  1. Requires good data: Bad demonstrations → bad policy
  2. Reward is implicit: No explicit reward signal (unlike RL)
  3. Not real-time: 10-50 ms latency (suitable for non-reactive tasks)

When to Use LeRobot

✓ Task has clear demonstrations (operator can show) ✓ Need quick prototyping (fast training) ✓ Limited compute (runs on Jetson) ✓ Want community support + pre-trained models

✗ Task requires real-time reaction (use classical control) ✗ Very different from any demo (use RL) ✗ Need formal safety guarantees

Next Steps

  1. Start with pre-trained ACT model
  2. Collect 50-100 demonstrations on your robot
  3. Fine-tune for 5-10 epochs
  4. Deploy and iterate

LeRobot democratizes robot learning — you don't need massive datasets or 4 years of PhD research anymore.


Resources

  • GitHub: github.com/huggingface/lerobot
  • Docs: huggingface.co/lerobot
  • Paper: LeRobot: Open-Source Robot Learning
  • Model Zoo: huggingface.co/lerobot (100+ models)

Tool recommendations

VLA train/deploy stack

Train on cloud/workstation, then deploy optimized models to Jetson or the robot computer.

Cloud GPU for VLA / policy training Use for imitation learning, diffusion policies, RL, and robotics model fine-tuning. View cloud GPU → NVIDIA Jetson Orin NX / Orin Nano Edge deployment hardware for perception, logging, and optimized inference. View Jetson → Hugging Face / robotics dataset hosting Host datasets, checkpoints, and model cards for cleaner LeRobot/VLA workflows. View platform →

Related posts

  • Action Chunking Transformers (ACT): Architecture in Detail -- Deep dive into the ACT policy
  • Diffusion Policy: A Revolution in Robot Manipulation -- Diffusion-based robot learning
  • VLA Models: RT-2 → Octo → OpenVLA → π0 -- Foundation models for robots
  • Embodied AI 2026: Overview and Trends -- The big picture of AI for robots
NT

Nguyễn Anh Tuấn

Robotics & AI Engineer. Building VnRobo — sharing knowledge about robot learning, VLA models, and automation.

Explore VnRobo

Fleet MonitoringROS 2 IntegrationAMR Solutions
← Previous
Multi-Object Manipulation: Sorting, Stacking & Rearranging
Next →
Bimanual Manipulation: Teaching Robots to Use Both Arms

Related Posts

NEWResearch
SLA: Continual Pre-training π0.5 with 3D Spatial Supervision (+24%)
vlapi0manipulation
wholebody-vla

SLA: Continual Pre-training π0.5 with 3D Spatial Supervision (+24%)

SLA aligns intermediate visual embeddings with a frozen 3D perception model during continual pre-training, improving real-world manipulation success by 24 percentage points without changing the inference architecture.

11/13/202610 min read
NT
NEWTutorial
NEO-VLA: Encoder-Free VLA Achieves 95.3% on LIBERO
vlamanipulationflow-matching
wholebody-vla

NEO-VLA: Encoder-Free VLA Achieves 95.3% on LIBERO

NEO-VLA removes the pretrained vision encoder entirely, using a lightweight patch embedder and a multi-layer flow-matching action expert to achieve 95.3% on LIBERO with full fine-tuning — and 90.9% with a frozen backbone.

11/11/20269 min read
NT
NEWTutorial
Run NEO-VLA on LIBERO: An Encoder-Free VLA Guide
vlaneo-vlalibero
wholebody-vla

Run NEO-VLA on LIBERO: An Encoder-Free VLA Guide

Understand NEO-VLA, install VLAKit, serve its released LIBERO checkpoint, and interpret training recipes and benchmark results correctly.

11/10/202614 min read
NT
VnRobo logoVnRobo logo

Learn Physical AI and experiment in your browser. Connect your fleet when you have real robots.

Product

  • Features
  • Pricing
  • Knowledge Base
  • Services

Company

  • About Us
  • Blog
  • Contact

Legal

  • Privacy Policy
  • Terms of Service

© 2026 VnRobo. All rights reserved.

Made with♥in Vietnam