APT-RL: KAIST HOUND Hits 6 m/s With Multi-Skill Locomotion

Why do quadruped robots still stumble when transitioning from stair-climbing to obstacle-jumping? The answer is gait transition — seamlessly switching between movement modes (gaits) remains one of locomotion robotics' hardest unsolved problems. Until APT-RL came along.

APT-RL (Action Pretrained Transformer-based Reinforcement Learning) is the latest framework from KAIST, published in Science Robotics — the field's most prestigious journal — as the July 2026 cover story. The KAIST HOUND robot running APT-RL achieved a peak speed of 6 m/s jumping down a three-step staircase, and completed a 1.1 km urban campus route through diverse complex terrain with nothing but onboard sensors.

KAIST HOUND on the Science Robotics July 2026 cover — source: skillquadsr.github.io

The Core Problem: Why Is Gait Transition So Hard?

In locomotion robotics, a "gait" is a movement mode — trot (diagonal legs move together), bound (front/back pairs), gallop. Each gait suits different speeds and terrains:

The problem: each gait has its own "grammar" — torque patterns, rhythm, balance strategy all differ completely. Training a single policy that knows all gaits and knows when to switch is extremely difficult. Traditional approaches fall into three traps:

1. Train each gait separately → Needs complex external switching logic, transitions are rarely smooth
2. Train one general policy → Performance averages out, loses each gait's specific strengths
3. Motion capture imitation learning → Expensive to collect, hard to scale across terrains

APT-RL solves this by learning the shared "grammar" of all gaits first, then learning when to use each — like learning phonetics before vocabulary, instead of memorizing each word independently.

APT-RL: Three Sequential Learning Phases

APT-RL divides learning into three sequential phases. Each builds on the previous — you cannot skip or reorder them.

Phase 1: Representation Learning — Learning the Gait "Grammar"

Goal: Find a shared latent space across all gaits, using trajectory optimization data instead of expensive motion capture.

The team ran Trajectory Optimization on a 2D Single Rigid Body Dynamics model (sagittal plane only) to generate 180,000 trajectories — 90,000 for trot and 90,000 for bound — representing 15.5 hours of motion, computed in just 8 minutes. This is the first key insight: instead of recording real robot video, they simulated simplified dynamics and used it as a "virtual teacher."

A Transformer-based Variational Autoencoder (TVAE) is then trained on this dataset:

Input: 3-frame proprioception history (69-dim per frame)
         ↓
   [Transformer Encoder — 4 layers, nhead=3]
         ↓
   Latent distribution (μ, σ) — 16-dim each
         ↓ reparameterize (KL weight λ = 0.1)
   Latent vector z — 16-dim
         ↓
   Gait-Specific Decoders:
       ├── Trot Decoder  → 12-dim joint torques
       └── Bound Decoder → 12-dim joint torques

The key insight: Both trot and bound get encoded into the same 16-dimensional latent space. This means the vector z captures "movement intent" regardless of which specific gait is active — like how the same brain encodes meaning whether you speak English or Vietnamese, with only the output decoder differing.

Phase 2: Reinforcement Learning — Learning When to Use Each Gait

With the TVAE trained and frozen, Phase 2 trains a new policy in Isaac Gym using PPO.

Policy output — 29 dimensions total:

• Latent action (16-dim): Vector z fed into pretrained decoders to generate torques
• Auxiliary action (12-dim): Fine-tuned via PD controller with k_p = 80, k_d = 2
• Gait selection logit (1-dim): Sigmoid → 0 = trot, 1 = bound, updated at 2 Hz

The actual motor torque is computed as:

τ_input = τ_dec + k_p·(q_default − q_t + 0.2·a_aux) − k_d·q̇_t

The a_scale = 0.2 coefficient limits how much the auxiliary action can override the pretrained decoder's output — preventing the RL agent from "undoing" what Phase 1 learned.

Curriculum learning across 7 terrain types of increasing difficulty: flat, rough, stairs, hurdles, stepping stones, gaps, fallen branches. Command velocity range: x ∈ [-1, 7] m/s, y ∈ [-1, 1] m/s. KL regularization in RL is set to 2.5×10⁻⁶ to keep latent actions within the distribution TVAE was trained on — if the RL policy explores latent values outside TVAE's training distribution, the decoder produces garbage torques.

Phase 3: Perceptual Distillation — Learning to See in Order to Run

The RL policy in Phase 2 uses privileged information — a ground-truth terrain map that only exists in the simulator. On a real robot, this is unavailable. Deploying to hardware requires a student policy using only onboard sensors.

Student policy sensor inputs:

• Intel RealSense D435 depth camera (range 0.3–3.0m) → CNN encoder → 32-dim features
• 2D LiDAR at 40 Hz, 45 bins, range 0.6–5.0m, resolution 0.1m/bin
• Proprioception: joint positions + velocities + IMU (rated to 10+ g shock)
• GRU encoder synthesizing all inputs into a perception embedding

Training uses DAgger (Dataset Aggregation) — the student mimics the teacher policy but receives real sensor observations, progressively rolling out its own policy rather than always copying the teacher.

A small but critical detail: the KAIST team designed custom 3D-printed vibration absorbers for the sensors. Without them, depth images blur severely at 4–6 m/s, making the CNN features unreliable.

Results: Hard Numbers on a Real Robot

Peak speeds measured:

• 4.25 m/s climbing a 60-cm step — Froude number = 3.85
• 6.0 m/s jumping down a 3-step staircase (0.3m height, 0.58m depth) — Froude number = 7.69

The Froude number Fr = v² / (g·L) where L is leg length. Fr > 1 means centrifugal force exceeds gravity — the robot is genuinely airborne rather than stepping. Fr = 7.69 is an agility level never before reported for an outdoor, untethered quadruped robot.

Real-world navigation:

• Urban route: 1.1 km through KAIST campus with varied stairs, hurdles, and uneven surfaces
• Forest route: 0.34 km through woodland with fallen logs, tree roots, and uneven ground

In both routes, no GPS, no motion capture, no external localization — only depth camera + 2D LiDAR + IMU.

Comparison with baselines:

• Adaptive policy (trot + bound switching) outperforms fixed-gait baselines across all high-difficulty terrain types (levels 4–10 in curriculum)
• Largest advantage on hurdles, gaps, and stairs — exactly where precise gait-switching timing matters most

Analysis: Why APT-RL Matters

1. Pretrained skill priors genuinely scale. Using trajectory optimization as a "virtual teacher" for representation learning isn't new in concept, but APT-RL proves it scales to agility levels never seen before on real outdoor hardware.

2. No motion capture required. The entire pipeline only needs a physics simulator and trajectory optimization. No mocap, no retargeting. This makes it much easier to add new gaits or port to different robot platforms.

3. Automatic gait selection at 2 Hz. No hand-crafted heuristics for when to switch gaits — the policy learns from reward. This means the robot can discover switching rules the engineers didn't anticipate.

4. Truly onboard-only perception. No external infrastructure. This is the condition of real deployment, and HOUND still hits 6 m/s.

Limitations and Extensions

The paper acknowledges several open points:

• Only 2 gaits: trot and bound. Gallop, pronk, and more complex gaits aren't yet integrated — in principle, adding a new decoder should work
• 2D terrain model in Phase 1: Sagittal-plane dynamics don't fully capture side-slip or yaw dynamics
• Pure locomotion: No arm or gripper integration

Natural extensions from APT-RL:

• Loco-manipulation — add an arm while using multi-skill locomotion as the mobile base
• Humanoid bipedal with more gaits (walk, jog, sprint, jump, crouch)
• Online terrain adaptation that adjusts without retraining

Getting Started: Reproducing the Core Idea

APT-RL doesn't have a public GitHub repository at the time of writing. But you can replicate the core concept with Isaac Lab and a custom TVAE. Here's a minimal skeleton:

import torch
import torch.nn as nn

class TVAE(nn.Module):
    """
    Transformer VAE learning a shared latent space across gaits.
    state_dim=69 (3 frames * 23-dim proprioception)
    latent_dim=16
    """
    def __init__(self, state_dim=69, latent_dim=16):
        super().__init__()
        encoder_layer = nn.TransformerEncoderLayer(
            d_model=state_dim, nhead=3, batch_first=True
        )
        self.encoder = nn.TransformerEncoder(encoder_layer, num_layers=4)
        self.mu_head = nn.Linear(state_dim, latent_dim)
        self.logvar_head = nn.Linear(state_dim, latent_dim)

        # Separate decoder per gait
        self.trot_decoder = nn.Sequential(
            nn.Linear(latent_dim, 64), nn.ELU(),
            nn.Linear(64, 12)   # 12 joint torques
        )
        self.bound_decoder = nn.Sequential(
            nn.Linear(latent_dim, 64), nn.ELU(),
            nn.Linear(64, 12)
        )

    def encode(self, x):
        # x: [batch, n_frames, state_dim]
        h = self.encoder(x)
        return self.mu_head(h[:, -1]), self.logvar_head(h[:, -1])

    def reparameterize(self, mu, logvar):
        return mu + torch.randn_like(logvar.exp().sqrt()) * logvar.exp().sqrt()

    def forward(self, x, gait='trot'):
        mu, logvar = self.encode(x)
        z = self.reparameterize(mu, logvar)
        decoder = self.trot_decoder if gait == 'trot' else self.bound_decoder
        return decoder(z), mu, logvar

def tvae_loss(tau_hat, tau_true, mu, logvar, lambda_kl=0.1):
    recon = nn.functional.mse_loss(tau_hat, tau_true)
    kl = -0.5 * torch.mean(1 + logvar - mu.pow(2) - logvar.exp())
    return recon + lambda_kl * kl

To run a locomotion baseline in Isaac Lab (not full APT-RL, but a solid starting point):

# Install Isaac Lab (requires Isaac Sim 4.x)
pip install isaaclab

# Train quadruped velocity tracking baseline
python scripts/reinforcement_learning/rsl_rl/train.py \
    --task=Isaac-Velocity-Rough-Anymal-C-v0 \
    --num_envs 4096

From there, replace the standard action output with (latent_action, aux_action, gait_logit) as described in the paper.

Key Takeaway

APT-RL is a textbook example of "divide and conquer" in robot learning: instead of asking a single RL agent to simultaneously learn gait mechanics, terrain reasoning, sensor fusion, and switching logic, the problem is split into three independently learnable phases.

Notably, all three phases use familiar tools — trajectory optimization, VAE, PPO, DAgger. Nothing exotic. The innovation is how they're assembled and the order they're applied in.

If you're building a locomotion controller for a quadruped or humanoid, APT-RL is a blueprint worth studying carefully — especially the idea of using trajectory optimization to bootstrap representation learning without any motion capture data.

Related Posts

• RL Locomotion: Reinforcement Learning Foundations for Robot Motion
• Quadruped Robots: How Four-Legged Robots Learn to Walk and Run
• Locomotion Parkour: Obstacle Traversal With Deep RL
• Sim-to-Real Transfer: Bridging the Gap Between Simulator and Reality