TORL-VLA: Fine-tune VLA with Tactile Sensing and Online RL for Contact-Rich Manipulation

Imagine a VLA robot that performs flawless pick-and-place in the lab, but the moment it needs to handle a raw egg or engage a mechanical latch — it fails repeatedly. That is the core problem TORL-VLA (arXiv 2606.09337, June 2026) addresses head-on.

Current VLA models are visually fluent but tactilely blind. They see through cameras and act through language tokens, but have no sense of the force at their fingertips. When contact conditions deviate from the training distribution — a surface is slightly more slippery, a holder is 1mm narrower — the policy has no feedback signal to adapt. The result: excessive grasping force that crushes fragile objects, or infinite retry loops that waste time and damage parts.

TORL-VLA solves this with two complementary mechanisms:

1. Wrench-Aware VLA: tactile sensors on the fingertips, integrated into the VLA via a novel late-fusion architecture
2. Online RL with Intervention-Censored Critic: a lightweight actor-critic that refines the VLA's actions in real-time during deployment, solving the credit assignment problem that plagues human-supervised RL

This guide walks through the paper's key ideas, the architecture in depth, hardware requirements, training procedure, and deployment results.

Why VLA Needs Tactile Feedback

Think about picking up a raw egg. Your eyes see the egg, but your fingers are what regulate grip force. With eyes closed, you can still safely grip an egg using touch alone. With numb fingers, even perfect vision doesn't prevent you from crushing it.

Current VLAs are in the "numb fingers" state — they see, but cannot feel.

Three specific failure modes this creates:

1. Contact distribution shift: Demonstrations are collected in ideal conditions. During deployment, objects slip slightly, friction varies, clearances differ by fractions of a millimeter. Without tactile feedback, the policy has no mechanism to adjust.
2. No completion signal: Inserting an object into a holder requires knowing whether it has "seated" properly. Camera resolution is insufficient to judge 0.5mm gaps, but the tactile force profile changes unambiguously at successful engagement.
3. Inefficient retry behavior: On failure, the policy repeats the same motions because there is no signal saying "this approach is wrong — change strategy."

TORL-VLA addresses all three with the Wrench-Aware reference model and the Online RL adaptation module.

Architecture: Two Stages, Three Modules

TORL-VLA operates as a two-stage pipeline:

Stage 1 — Wrench-Aware VLA (Reference Model)

An offline model pre-trained on 539 demonstration trajectories. It takes as input:

• 3 camera feeds: global RealSense D435i, wrist RealSense D405, gripper fisheye
• Language instruction: natural language task description
• Tactile wrench history: 12D wrench vector (6D × 2 fingertips) with J=10 samples over 2 seconds

The model outputs two things simultaneously:

• Reference action chunk: predicted action sequence (like a standard VLA)
• Future wrench sequence: predicted near-future contact forces

Tactile Integration Architecture

The key design choice is how to fuse tactile into the VLA. Early injection (prepending tactile tokens to the input) disrupts pretrained visual-semantic features. TORL-VLA instead uses late fusion after visual-language encoding:

Tactile sensors → 12D wrench × J samples → MLP encoder → Tactile token Pt
Visual-Language encoder → z_vis
Late fusion: Self-attention([Pt; z_vis]) → fused representation

Additional architectural components:

• Soft-routing Mixture-of-Experts (MoE): dynamic expert routing based on wrench context, selecting the appropriate expert for each contact phase
• Zero-initialized wrench bypass: a residual shortcut so fine-grained contact information bypasses the main attention stack and is preserved

Training loss jointly optimizes action and wrench prediction:

L_ref = ||û_action - u_action||² + λ_w ||û_wrench - u_wrench||²

Where λ_w = 0.3 — wrench prediction acts as a structured regularizer.

Stage 2 — Online RL Module (Refinement)

A lightweight actor-critic that runs alongside Stage 1 during live deployment. It consists of:

Stage Estimator: Classifies which phase of the task the robot is currently executing (pre-contact, in-contact, placement, release) and routes to the appropriate actor-critic head.

Actor-Critic Refinement:

• Actor: takes reference action from Stage 1 + current wrench + predicted wrench → outputs refined action
• Critic: evaluates refined action quality

The robot re-queries Stage 1 every K=10 steps (0.5 seconds at 20Hz), maintaining both long-horizon planning from the VLA and real-time refinement from RL.

Intervention-Censored Critic — Solving Credit Assignment

This is the paper's key theoretical contribution. In robot Online RL, human operators frequently intervene when the policy is about to make an error. A standard critic treats post-intervention successes as evidence that pre-intervention actions were good — completely wrong credit assignment.

TORL-VLA introduces the Intervention-Censored (IC) Critic:

# Normal transitions (no intervention):
y_task = r_t + γ^K * min_Q_task(s_{t+K}, a_{t+K})

# Intervention boundary (m_int = 1):
y_ic = r_t - c_int + (1 - done) * (1 - m_int) * γ^K * min_Q_ic(...)

m_int is a mask marking intervention boundaries. When m_int=1, the bootstrap target is removed and an intervention cost c_int is applied. The semantic: "this action triggered an intervention → it was bad, don't credit the success that followed."

The actor loss combines three terms:

L_π = λ_bc * L_bc  -  λ_task * E[min_Q_task]  +  λ_ic * E[ΔQ_ic]

• L_bc: behavior cloning penalty to prevent drift from the reference model
• min_Q_task: maximize task success value
• ΔQ_ic: penalize actions that tend to trigger interventions

Hardware Requirements

Robot Platform

The paper uses a Piper robotic arm by AgileX Robotics with a Pika gripper — a 6-DOF arm popular in robotics research at around $8k USD. The TORL-VLA architecture is platform-agnostic; any arm with a mountable gripper and wrist cameras can be adapted.

Tactile Sensors — Tacta Fingertip Sensors

The most critical and least familiar component. The paper uses Tacta fingertip sensors — two sensors mounted on the inner surfaces of the gripper fingers:

• Dimensions: 21×18.5×4.5mm (very compact)
• Array: 6×8 piezoelectric elements per sensor
• Output: 48 pressure values → converted to 6D wrench (Fx, Fy, Fz, Mx, My, Mz)
• Two sensors → 12D wrench vector total

Converting tactile array to wrench:

tactile_left = sensor_left.read()    # shape: (6, 8)
tactile_right = sensor_right.read()  # shape: (6, 8)

# Calibrated linear transform
wrench_left  = calibration_matrix_left  @ tactile_left.flatten()   # → 6D
wrench_right = calibration_matrix_right @ tactile_right.flatten()  # → 6D

# Concatenate
wrench_12d = np.concatenate([wrench_left, wrench_right])  # → 12D

Calibration matrices are determined via hand-eye calibration combined with FEM simulation of the gripper geometry.

Camera Configuration

Three cameras with distinct roles:

All three are synchronized at 20Hz with timestamp alignment to avoid temporal mismatch between visual and tactile data streams.

Training Procedure

Step 1: Demonstration Collection

Collect 539 full-task trajectories (complete task execution, not individual subtasks). Each trajectory records:

• 3 camera streams at 20Hz
• 12D wrench history (10 samples × 2.0s)
• Joint positions and velocities
• Language instruction
• Intervention timestamps (critical for IC Critic)

Tips for quality demonstration data:

• Execute at natural speed — don't slow down artificially
• Include failure recoveries (both human and robot recovery)
• Label every human intervention with a precise timestamp
• Distribute conditions evenly (cup orientation, latch angle, egg position)

Step 2: Train Reference Model

pip install torch torchvision transformers einops

python train_reference.py \
    --data_dir ./data/demos \
    --base_model pi0.5 \
    --enable_moe \
    --enable_wrench_bypass \
    --wrench_loss_weight 0.3 \
    --batch_size 32 \
    --lr 1e-4 \
    --epochs 100 \
    --output_dir ./checkpoints/reference

With 539 trajectories and a pi0.5 backbone, training takes approximately 6-8 hours on an A100. Expected loss curve:

Epoch 10:  L_action=0.24, L_wrench=0.18, L_total=0.29
Epoch 50:  L_action=0.09, L_wrench=0.07, L_total=0.11
Epoch 100: L_action=0.06, L_wrench=0.05, L_total=0.075

Step 3: Online RL Adaptation (On-Robot)

Online RL is not trained offline — it trains directly on the physical robot in two phases:

Warmup Phase (before live deployment):

# 300 demo chunks, 5k gradient updates
# Purpose: initialize actor close to reference policy
warmup_trainer = OnlineRLWarmup(
    demo_buffer=demo_buffer,
    n_chunks=300,
    n_gradient_steps=5000
)
warmup_trainer.train(actor, critic)

Main Interaction Phase (on the real robot):

# 120.8 minutes total robot interaction time
# Update-to-data ratio: 5 gradient steps per transition
online_trainer = OnlineRLTrainer(
    actor=actor,
    task_critic=task_critic,
    ic_critic=ic_critic,
    update_to_data_ratio=5,
    gamma=0.99,
    tau=0.005  # EMA for target network
)

for episode in robot_episodes:
    trajectory = robot.rollout(actor, reference_model)
    online_trainer.update(trajectory)

Intervention logging during collection:

def on_human_intervention(t):
    intervention_mask[t] = 1.0
    intervention_cost[t] = -1.0  # c_int penalty

Experimental Tasks

Task 1: Coffee Cup Placement

• Description: Place a coffee cup into a tight holder with limited camera visibility (cup body blocks the view)
• Challenge: Requires just enough force for the cup to "snap" into position without tipping
• Tactile signal: At successful seating, wrench profile shifts abruptly and stabilizes

Task 2: Latch Manipulation

• Description: Grip, flip, and lock a mechanical latch
• Challenge: Three distinct contact phases — grip → flip → lock — each with a different force profile
• Tactile signal: The latch "click" creates a wrench spike; Stage Estimator detects this to trigger phase transition

Task 3: Egg Handling

• Description: Grasp a raw egg and place it in a tray
• Challenge: Extremely narrow force window — too strong crushes it, too light and it slips
• Tactile signal: Optimal wrench range is approximately 0.3N–0.8N per fingertip

Full-task: Execute all three sub-tasks sequentially in a single autonomous rollout without human assistance.

Results

Comparison with Baselines (30 trials per method)

Key takeaways:

• 100% success on Cup and Egg — perfect across all 30 trials
• Full-task 28/30 vs 12/30 for π0.5 — a 133% improvement
• Faster execution: 165.45s vs 199.65s — fewer retries means less time wasted
• TORL-VLA w/o RL already outperforms baselines significantly — proving tactile integration has independent value before any RL adaptation

Ablation Insights

• MoE routing: Largest single contributor in the reference model — removal causes the steepest success rate drop
• Wrench bypass: Most important for Egg task — the bypass preserves fine-grained force signals that are "smoothed out" by deep attention layers
• IC Critic: Largest contribution on Latch task — because Latch has the most human interventions during training, making correct credit assignment most critical

Inference Pipeline

After training, the inference loop looks like this:

from torl_vla import TORLVLAAgent

agent = TORLVLAAgent(
    reference_model_path="./checkpoints/reference/best.pt",
    rl_actor_path="./checkpoints/online_rl/actor_final.pt",
    rl_critic_path="./checkpoints/online_rl/critic_final.pt",
    device="cuda"
)

while not task_done:
    obs = {
        "images": {
            "global": cam_global.capture(),
            "wrist": cam_wrist.capture(),
            "gripper": cam_fisheye.capture()
        },
        "wrench_history": tactile_buffer.get_last(n=10),  # 10×12D
        "language": "Place the coffee cup into the holder"
    }

    # Stage 1: Reference from Wrench-Aware VLA
    ref_action, pred_wrench = agent.reference_predict(obs)

    # Stage 2: RL Actor refines based on stage
    stage = agent.estimate_stage(obs)
    refined_action = agent.rl_refine(obs, ref_action, pred_wrench, stage)

    robot.execute(refined_action)
    tactile_buffer.append(tactile_sensors.read())

The K=10 replanning cadence (0.5s) balances computation cost with real-time responsiveness.

Comparison With Other Approaches

Main tradeoff: TORL-VLA requires ~120 minutes of robot interaction time for online adaptation — a real operational cost to weigh against the performance gains.

Code & Resources

As of July 2026, the official codebase from the authors is marked Coming Soon on the project page. A community reproduction exists:

• openpi-RLT: Reproduction of the RLT baseline (without IC Critic) — useful for understanding the online RL component in isolation
• Paper: TORL-VLA arXiv 2606.09337
• Project page: torl-vla.github.io

Watch the project page or the authors' GitHub (Huaihang Zheng et al., Beijing Institute of Technology) for the official code release.

Conclusion

TORL-VLA represents a meaningful step forward for contact-rich manipulation. The three core contributions:

1. Wrench-Aware VLA: Late fusion + MoE + bypass — tactile integration that preserves pretrained visual features
2. Online RL with IC Critic: real-robot adaptation that correctly attributes credit in the presence of human intervention
3. Stage-specific refinement: intelligent routing by contact phase

The 28/30 full-task result (vs 12/30 for the baseline) demonstrates that both components contribute independently and complement each other.

If you are building a manipulation system for contact-rich tasks — assembly, peg-in-hole, fragile object handling — this architecture is worth following closely when the official code releases.

Related Posts

• FORCE: Boost VLA Fine-tuning with Value-Calibrated RL Warm-Up
• Contact-Rich Manipulation: From Theory to Deployment
• Fine-tuning VLA on LeRobot with Embodied Data