VLASH: Real-Time VLAs via Async Inference (11.8× Faster)

The Core Problem: Your Robot is Always Looking at the Past

Imagine playing ping-pong with 500ms of lag. You see the ball, but your arm responds half a second later. By then, the ball has long since bounced away.

This is exactly the problem plaguing Vision-Language-Action models (VLAs) in real-world robot deployment. A state-of-the-art model like π₀.₅ needs roughly 500–560ms to compute each action chunk. During that time, the robot either freezes (waiting for the result) or keeps executing stale actions — either way, it cannot react to events happening now.

VLASH (MIT Han Lab, arXiv 2512.01031) solves this with an elegantly simple idea: instead of predicting actions based on the robot's current state, estimate where the robot will be when the actions actually execute, and predict from that future state.

The result: up to 11.8× lower reaction latency on H100, robots that can play ping-pong against humans at 55% success rate (all baselines: 0%), whack-a-mole scores 9× higher than synchronous inference — all without changing a single layer of the VLA architecture.

Paper: VLASH — arXiv 2512.01031, MIT / Tsinghua / UC Berkeley / NVIDIA / Caltech / UCSD

Background: Why Async Inference Is Hard

Before diving into VLASH, it helps to understand why naive solutions fail.

Synchronous inference (the standard approach):

t=0:     Capture frame → send to VLA model
t=0→t+Δ: Robot FREEZES, waiting for inference
t+Δ:     Receive action chunk → start executing

With Δ ≈ 500ms, the robot stalls for half a second every time it needs to "think." Interactive, dynamic tasks are simply impossible.

Naive asynchronous inference (the obvious fix):

t=0:     Capture frame → send to VLA model
t=0→t+Δ: Robot KEEPS MOVING (executes previous chunk)
t+Δ:     Model finishes — but computed actions for state s₁ (at t=0)
          Robot is now at state s₃ (at t+Δ)!

The robot keeps moving — good. But the model predicted actions for a robot at position A, while the robot is actually at position B. This temporal misalignment causes jerky, discontinuous motion and drops accuracy by up to 30.5% on dynamic benchmarks.

VLASH achieves the best of both worlds: robot keeps moving and the model predicts from the correct state.

How VLASH Works: Future-State Rollout

The core insight of VLASH is beautifully simple:

"I just commanded the robot to execute action chunk [a₁, a₂, a₃, ...]. So Δ seconds from now, I already know where the robot will be — I can compute it by accumulating those actions onto the current state."

Formally:

Known:    current state s₁ (at t=0)
Known:    action chunk being executed: [a₁, a₂] (issued at previous step)
Estimate: s₃ = s₁ + a₁ + a₂  ← where robot will be at t=Δ
Input to model: (observation_t=0, s₃)
Output: action chunk to execute starting at t=Δ → temporally aligned!

This future-state rollout eliminates the prediction-execution mismatch without adding any inference overhead. The robot state is just updated before being passed to the model — no inpainting, no architectural changes, no extra neural network layers.

For models with text-based state encodings like π₀.₅, VLASH updates the state tokens in the prompt. An optional lightweight projection layer can improve smoothness further, but it's entirely optional.

Temporal-Offset Fine-Tuning

Inference-time state rollout alone isn't enough — the model needs to be trained to understand future states. Without training, feeding s₃ to a model that learned to expect s₁ will confuse it.

VLASH adds temporal-offset fine-tuning: during training, instead of only using pairs (observation_t, state_t) → action_t, it augments with:

(observation_t, state_{t+δ}) → action_{t+δ}

where δ is a random offset sampled from {0, 1, ..., Δ_max}.

This teaches the model: "when given a future state, predict the corresponding future actions." No new data required — VLASH generates these pairs automatically from existing robot trajectories.

Shared Observation Training: 3.26× Faster Fine-Tuning

A naïve implementation would run a separate forward pass for each offset δ, multiplying training cost by Δ_max. VLASH uses block-sparse attention masking to pack all offsets into a single forward pass:

One sequence = [shared_observation | branch_δ=0 | branch_δ=1 | ... | branch_δ=max]

All branches attend to one shared observation (no re-encoding), but cannot cross-attend to each other (each branch independently predicts its own actions).

Result: 3.26× training speedup per step with no accuracy penalty.

Action Quantization: An Extra 1.5–2× Physical Speedup

Beyond solving temporal misalignment, VLASH adds a separate action quantization mechanism to make the robot move faster.

The idea: instead of executing 24 fine-grained actions (e.g., 1mm each), group q consecutive actions into one coarser macro-action (q × 1mm in the same slot):

Fine-grained: [a₀, a₁, a₂, a₃, a₄, a₅]  — 6 slow steps
Macro (q=2):  [â₀, â₁, â₂]               — 3 faster steps (same total motion)

The robot physically executes the same trajectory but in roughly half the time. Combined with VLASH's asynchronous inference, you get both lower latency and higher throughput.

Real-robot experiments show q=2 achieves 2× task completion speedup with only ~2% accuracy drop on pick & place. For precision tasks like stacking, q=1.5 is a better trade-off.

Installation

Requirements: Python 3.10, CUDA GPU (RTX 4090+ recommended; RTX 5090 for >30Hz with π₀.₅).

# Create conda environment
conda create -n "vlash" python=3.10
conda activate vlash

# Install ffmpeg (required for video processing)
conda install ffmpeg=7.1.1 -c conda-forge

# Clone and install VLASH
git clone https://github.com/mit-han-lab/vlash.git
cd vlash
pip install -e .

# Install PyTorch stack
pip install -U torch torchvision torchcodec

Note: Use conda-forge for ffmpeg — the version from apt-get will cause torchcodec compatibility issues.

VLASH natively supports LeRobot datasets v2.1 and v3.0, so if you already have a LeRobot-format dataset, you can start fine-tuning immediately.

Supported VLA Models

VLASH has been tested with:

• π₀.₅ (Physical Intelligence) — primary model in all benchmarks
• π₀ — works with optional state projection layer
• GR00T N1.6 (NVIDIA) — tested on Galaxea R1 Lite
• SmolVLA-450M (HuggingFace) — lightweight option for smaller GPUs

No architecture changes needed — VLASH wraps any VLA at the training and inference level.

Training

Fine-tuning uses LoRA, so GPU memory requirements are manageable. The training config specifies the model, max offset, and dataset:

# examples/train/pi05/async.yaml
model:
  name: pi05
  checkpoint: lerobot/pi05   # or a local fine-tuned checkpoint

training:
  max_offset: 4              # Δ_max — max inference delay in control steps
  shared_obs: true           # 3.26× training speedup
  batch_size: 16
  num_steps: 30000

dataset:
  path: /path/to/lerobot_dataset

Run fine-tuning:

vlash train examples/train/pi05/async.yaml

The key parameter to set correctly is max_offset. Measure the actual inference delay your hardware introduces in control steps (at your control frequency), then set max_offset ≥ that value. If max_offset is too small, the model won't be trained for delays it will encounter in deployment.

Inference

Standard async inference (no action quantization):

vlash run examples/inference/async.yaml

With 2× action quantization:

vlash run examples/inference/async.yaml --action_quant_ratio=2

Inference config:

# examples/inference/async.yaml
model:
  checkpoint: /path/to/vlash_finetuned_checkpoint

inference:
  async_mode: true
  action_quant_ratio: 1.0    # 1.0 = no quantization, 2.0 = 2× speedup

robot:
  type: so101                 # or galaxea_r1, unitree_g1, etc.
  control_freq: 30            # Hz

On an RTX 5090, π₀.₅ with VLASH runs at >30Hz — sufficient for fluid, real-time manipulation control.

Benchmark Results

Reaction Latency — The Headline Numbers

The core claim: VLASH reduces maximum reaction latency by 11.8× on H100.

Why the synchronous number is so high: synchronous reaction latency = chunk_execution_time + inference_time. With a K=24 step chunk at 30Hz, that's 800ms + 23ms ≈ 823ms. VLASH eliminates the execution wait — reaction latency drops to just 2× inference_time.

LIBERO Simulation Benchmarks

VLASH at 1-step delay exceeds synchronous performance on π₀.₅ — the model makes slightly better predictions when it knows the execution-time state.

Kinetix (12 Dynamic Tasks)

At 4-step inference delay (a challenging realistic scenario):

• VLASH: 81.7% average success rate
• Naive async: 51.2%
• Improvement: +30.5 percentage points

Dynamic Interactive Tasks (Real Robot, Galaxea R1 Lite)

This is where VLASH's advantage becomes undeniable. Tasks that require sub-100ms reactions:

VLASH is the first VLA framework to successfully play ping-pong rallies against a human operator.

Real-Robot Manipulation (LeRobot SO-101)

With action quantization (q=2):

A 2× throughput increase for 2% accuracy trade-off is an excellent deal in most industrial deployment scenarios.

Comparison with Other Methods

RTC (Real-Time Chunking) — used in Pi0-FAST on Unitree G1 — applies inpainting at inference time to correct for stale states. This adds computation, slightly widening the latency gap. Read our Pi0-FAST deployment guide here.

A2C2 requires adding correction heads to the VLA architecture, preventing drop-in usage with existing checkpoints. VLASH works with any checkpoint out of the box.

Integration with Existing Workflows

If you're already using LeRobot for data collection and training, VLASH slots in cleanly:

1. Collect demos via LeRobot teleoperation    (unchanged)
2. Push dataset to HuggingFace Hub            (unchanged)
3. Fine-tune with: vlash train async.yaml     (NEW — adds VLASH fine-tuning)
4. Deploy with:    vlash run async.yaml       (REPLACES lerobot inference)

No hardware changes, no new data collection, no dataset re-formatting. VLASH operates entirely at the training augmentation and inference scheduling layer.

For details on how VLA models have evolved from RT-2 to π₀, or for background on reactive action chunking approaches like DREAM-Chunk, see the related posts below.

Pitfalls to Watch

1. Set max_offset too small: If your hardware delay (in control steps) exceeds max_offset, the model was never trained for that delay range — accuracy drops. Measure your actual inference delay first.

2. State rollout diverges during contact: When the robot is grasping an object, the forward-simulated state may drift from reality (contact forces aren't modeled). The paper acknowledges this as an open problem. For precision grasping phases, consider reducing action_quant_ratio to 1.0.

3. ffmpeg version matters: VLASH requires ffmpeg 7.1.1 specifically, via conda-forge. The system package (apt-get) ships an incompatible version that silently breaks torchcodec.

4. Shared observation requires compatible model architecture: The block-sparse attention optimization requires the VLA to accept observation and state as separate input sequences. If your model uses fused token sequences, you may need to fall back to sequential offset encoding (still correct, just 3× slower to train).

When to Use VLASH

VLASH is the right choice when:

• Your task requires <100ms reaction time (dynamic objects, human-robot interaction)
• You have existing π₀.₅, GR00T, or SmolVLA checkpoints and want real-time deployment without retraining from scratch
• Your GPU can't reach 30Hz with synchronous inference — VLASH gives you async speed with synchronous accuracy
• You want to maximize robot throughput on assembly or sorting tasks (action quantization)

Think carefully when:

• The task is fully static (placing objects, no moving targets) — latency gains are smaller
• Robot state cannot be reliably estimated from action accumulation (highly compliant robots, underactuated systems) — the future-state estimate may drift
• You need QLoRA support for <8GB VRAM GPUs — this is noted as pending in the repo

Conclusion

VLASH represents one of the most practically impactful advances in VLA deployment of 2025. It doesn't introduce a new architecture or require millions of extra demonstrations. Instead, it fixes a fundamental mismatch that was always there: VLAs are trained to predict from the present, but actions execute in the future.

By bridging that gap with future-state rollout and temporal-offset fine-tuning, VLASH transforms π₀.₅ from a model that needs half-second buffers into a real-time controller running at 30Hz+. The ability to play ping-pong at 55% success rate — a task where every other VLA baseline scores 0% — makes this point viscerally concrete.

Apache 2.0 license, LeRobot-compatible, no architecture changes required. If you're deploying a VLA on a real robot in 2026, VLASH should be your default inference strategy.

Resources:

• GitHub: mit-han-lab/vlash
• Paper: arXiv 2512.01031
• Project page: z-lab.ai/projects/vlash

Related Posts

• VLA Models: From RT-2 to Octo to OpenVLA to π₀ — the full evolution story
• LeRobot v0.5: Pi0-FAST + G1 Whole-Body Control — deploy VLA real-time on humanoid
• DREAM-Chunk: Reactive Action Chunking via Latent World Model for VLA Robots