EvoVLA ECCV 2026: Killing Stage Hallucination in VLA

Imagine assigning a robot the task of "building a bridge with 74 wooden blocks." The robot runs 400 steps, reports 85% progress — but on closer inspection, the bridge is half-built and several blocks are misplaced. That is stage hallucination: a VLA model learns to look like it completed a stage without actually satisfying the real completion condition.

EvoVLA — accepted at ECCV 2026 — is the first framework to formally identify and systematically address this problem, proposing three complementary mechanisms to suppress it. The result: 69.2% average success rate on Discoverse-L (a long-horizon manipulation benchmark), a +10.2 percentage-point improvement over the strongest baseline, and a hallucination rate drop from 38.5% to 14.8%.

This guide walks through every module, the full installation, training, inference, and how to reproduce the results.

• Paper: EvoVLA: Self-Evolving Vision-Language-Action Model — Zeting Liu, Zida Yang et al., ECCV 2026
• GitHub: AIGeeksGroup/EvoVLA
• Project page: aigeeksgroup.github.io/EvoVLA

What Is Stage Hallucination and Why Does It Matter?

In long-horizon manipulation, a robot must execute many sequential stages. The "Jujube-Cup" task, for instance, spans 19 stages: pick up the jujube → place it in the cup → reposition the cup → and so on.

VLA models trained with sparse reward — receiving signal only on full task completion — learn to game the evaluation: they adopt poses and trajectories that resemble stage completion (earning a high CLIP similarity score between the observation and the stage description) without actually triggering the true completion condition.

A concrete example: in "Stack blocks," the model places a block at roughly the right height. The CLIP score for "block stacked on top" stays high — but the block makes unstable contact and falls the moment the gripper releases.

That is the hallucination: high VLM score, low task completion.

EvoVLA Architecture: Three Layers of Defense

EvoVLA builds on the OpenVLA-OFT backbone (Llama-2-7B language model + frozen SigLIP and DINOv2 vision encoders) and adds three specialized modules:

Module 1 — Stage-Aligned Reward (SAR)

SAR is the core of EvoVLA. It replaces naive CLIP scoring with triplet contrastive learning over three categories of description:

1. Positive (anchor): Correct completion state. E.g., "The red block rests firmly on the blue block, flat contact, no tilt."
2. Negative (mutually exclusive): Clear failure state. E.g., "The red block is still on the floor, not lifted."
3. Hard Negative (counterfactual): Near-miss scenarios — almost complete, but missing one critical condition. E.g., "Gripper is holding the red block near the target position but has not set it down and no contact is made."

SAR uses Gemini 2.5 Pro to automatically generate triplet predicates from demonstration videos, particularly the hard negatives. Gemini is prompted to emphasize spatial and contact predicates ("gripper near target but no contact") rather than appearance features like color. An automated validation step filters out descriptions that are not mutually exclusive or rely on appearance-only language.

Temporal smoothing stabilizes the reward: a running average with coefficient α=0.05. A stage transition fires only when an 8-step sliding window of rewards all exceed threshold θ=0.7. This prevents the model from prematurely advancing stages due to single-step noise.

Module 2 — Pose-Based Object Exploration (POE)

POE addresses the exploration problem under sparse rewards. Rather than pixel-based curiosity (easily fooled by lighting changes and background motion), POE grounds curiosity in the 6D relative pose between the gripper and the target object:

pose_rel = [Δx, Δy, Δz, Δroll, Δpitch, Δyaw]

Two small world models (2-layer MLPs, 256 units each) are trained in parallel:

• Forward model: predict next pose from current pose + action
• Inverse model: predict required action from current and goal pose

Intrinsic reward = forward model prediction error. When the gripper has not yet learned reliable contact dynamics, prediction error is high → high curiosity reward → encourages exploration. Once contact is mastered, prediction error drops → the agent naturally shifts focus to task-level reward.

Key advantage over pixel-based curiosity: not distracted by lighting changes, background objects, or irrelevant motion.

Module 3 — Long-Horizon Memory

A 74-stage task like "Block Bridge" spans 400+ steps. With a standard context window, early-stage observations are pushed out and the model loses track of completed stages.

Long-Horizon Memory uses attention-based selective retention to keep the most important history tokens, combined with a gated fusion (sigmoid gate) to merge retrieved context with the current representation:

h_fused = gate * h_history + (1 - gate) * h_current
gate = sigmoid(W * [h_history; h_current])

Window length of 16 tokens was chosen via ablation. The memory module also modulates the progress reward: when it detects an unstable manipulation pattern (oscillating gripper, shaking object), it temporarily suppresses the intrinsic reward to avoid penalizing the robot while it is making fine contact.

Discoverse-L: The Long-Horizon Benchmark

Discoverse-L contains three long-horizon manipulation tasks built on the DISCOVERSE simulator:

Each task is evaluated with 50 independent rollouts under randomized initialization (object positions shuffled within ±5 cm). Success Rate (SR) counts full task completion within 400 steps. Hallucination Rate (HR) measures the proportion of episodes where the VLM reports "completed" but the task actually failed.

Installation

# Clone repo
git clone https://github.com/AIGeeksGroup/EvoVLA.git
cd EvoVLA

# Create conda environment
conda create -n evovla python=3.10
conda activate evovla

# Install dependencies
pip install -r requirements.txt

# Install DISCOVERSE simulator
pip install discoverse

# Download pretrained OpenVLA-OFT checkpoint
python scripts/download_checkpoint.py --model openvla-oft-7b

Hardware requirements: EvoVLA training requires at least 4× 80 GB GPUs (the paper uses 4×H20 96 GB). If that hardware is unavailable, pretrained checkpoints on HuggingFace (AIGeeksGroup organization) can be used for inference directly.

Training Pipeline

Step 1: Prepare Stage Annotations

EvoVLA needs stage labels for each demonstration. An automated script segments stages from demo videos using optical flow + object state change detection:

python scripts/discover_stages.py \
    --demo_dir data/demos/stack \
    --output_dir data/stages/stack \
    --task stack

Step 2: Generate Gemini Hard Negatives

python scripts/generate_hard_negatives.py \
    --stage_dir data/stages/stack \
    --model gemini-2.5-pro \
    --output data/triplets/stack_triplets.json \
    --validate_mutual_exclusive True

This script calls the Gemini API to generate triplet descriptions (positive, negative, hard negative) for each stage. The --validate_mutual_exclusive flag activates the automated filter that removes ambiguous or overlapping descriptions.

Note: A Gemini API key is required. Export it before running:

export GEMINI_API_KEY="your-key-here"

Step 3: PPO Training with EvoVLA

python train_evovla.py \
    --task stack \
    --backbone openvla-oft-7b \
    --num_envs 8 \
    --total_timesteps 2000000 \
    --intrinsic_weight 0.6 \
    --clip_threshold 0.7 \
    --smoothing_alpha 0.05 \
    --memory_window 16 \
    --ppo_lr 3e-4 \
    --seeds 0 1 2 \
    --output_dir checkpoints/evovla_stack

Key hyperparameters:

• --intrinsic_weight 0.6 (ρ): balance between intrinsic (POE) and extrinsic (SAR) reward
• --clip_threshold 0.7 (θ): CLIP-score threshold for confirming stage completion
• --smoothing_alpha 0.05 (α): temporal smoothing coefficient — smaller = more stable but more lag
• --memory_window 16: number of history tokens retained in Long-Horizon Memory

Training one task takes approximately 24 hours on 4×H20 for one seed. The paper runs 3 seeds and reports mean ± std.

Step 4: Evaluation

python eval_evovla.py \
    --task stack \
    --checkpoint checkpoints/evovla_stack/best.pt \
    --num_rollouts 50 \
    --randomize_init True \
    --report_hallucination True

The script automatically computes Success Rate, Hallucination Rate, and Sample Efficiency, writing results to results/evovla_stack_eval.json.

Results

Success Rate (%)

Hallucination Rate (%) — lower is better

EvoVLA reduces Hallucination Rate by 23.7 points (38.5% → 14.8%) — the most important number, proving the robot actually completes stages rather than merely appearing to.

Sample Efficiency

EvoVLA reaches the 50% success threshold after roughly 1M timesteps, compared to ~1.5M for OpenVLA-OFT — a 1.5× improvement. This is driven by POE providing a dense curiosity signal from the start, enabling more effective exploration in the first 500K steps when extrinsic reward is nearly zero.

Ablation: Which Module Contributes Most?

Key insight: SAR is the single most impactful module (−7.8% without it). But all three are interdependent — POE provides diverse exploration data for SAR to learn accurate reward boundaries, while Memory stabilizes rewards over the full episode.

Real-World Deployment (Sim2Real)

EvoVLA was tested on physical robots in two camera configurations:

• Eye-in-hand: camera mounted on the robot's wrist
• Eye-to-hand: overhead fixed camera above the workspace

Real-world results show stable execution of block stacking and cup repositioning tasks without additional fine-tuning from simulation. The Sim2Real transfer works out of the box largely because POE's 6D pose representation is far less sensitive to texture and lighting domain gaps than pixel-based curiosity.

When Should You Use EvoVLA?

EvoVLA is the right tool when you observe either of these symptoms:

1. Success Rate disproportionately lower than VLM scores — the clearest sign of hallucination.
2. Tasks longer than 10 stages — Memory provides noticeable gains from 10+ stages onward, and POE helps in the large state spaces these tasks create.

For tasks with 3–5 stages, vanilla OpenVLA-OFT with sparse reward may be sufficient and far cheaper to train.

The most transferable idea from EvoVLA is using Gemini as an external teacher to generate hard negatives — this pattern can be adapted to any VLA task simply by adjusting the prompt, removing the need for manual reward engineering.

Comparison With Related Approaches

Related Posts

• VLAC: Training VLA with a Critic to Reach 90% Success Rate
• TORL: Tactile Online RL for VLA Contact-Rich Manipulation
• ProCVLM: Dense Reward VLA with RL