BiGym 2.0: G1 VLA with GR00T-WBC and LeRobot

A robot can hold a plate while standing still and drop it as soon as it steps sideways. It can learn to open a drawer with its arm and fail when it needs to lower its body to reach the handle. These situations explain why a fixed robot-arm benchmark cannot fully measure humanoid manipulation.

BiGym 2.0 turns this problem into a downloadable MuJoCo environment. A Unitree G1 performs household tasks while a whole-body controller handles balance and walking. This tutorial takes you from installation to an ACT checkpoint, then explains how the LeRobot demonstrations support VLA research. You do not need a physical robot or a VR headset to begin.

The primary references are the SWIRL Lab BiGym2 repository, the original paper, and the repository documentation. Commands below were checked against public source code on October 14, 2026. This is not a claim that training was executed or that the reported research scores were reproduced on a particular machine.

1. Research idea: keep the execution substrate consistent

BiGym 2.0 inherits household scenes and tasks from BiGym but uses a G1 with a locomotion controller. Its central design choice is to keep GR00T-WBC frozen and use the same controller during demonstration collection, training, and evaluation. This reduces a common source of mismatch: collecting demonstrations with one execution system and evaluating learned policies with another.

The high-level policy chooses what the arms should do and how the body should move. GR00T-WBC executes body commands while maintaining balance and walking. A manipulation policy therefore still needs to coordinate body motion with grasping, even though it does not directly predict every leg joint target.

Technical GIF from the project README. Watch the legs to see the difference between the earlier execution system and the G1 controller.

The release provides 20 tasks with 60 successful human VR demonstrations per task: 1,200 episodes in total. Task groups cover reaching, tabletop manipulation, dishwasher operations, and kitchen-counter work. move_plate is a useful starting point because approach, grasp, transport, and release are easy to observe. stack_blocks is substantially harder because transport and precise placement must work across several stages.

For the underlying data concepts, see LeRobot: a framework for learning policies from demonstrations. Here, LeRobot initially means a structured dataset format. A dataset using that format does not automatically make every policy trained on it a vision-language-action model.

2. Architecture: a high-level policy above WBC

Read the system as three connected parts: observations, high-level decisions, and physical execution.

Head RGB + right-wrist RGB + left-wrist RGB
                    + proprioception
                    + task text for a VLA
                              |
                              v
                   High-level policy
                  /         |         \
          Arm targets   Gripper     Body commands
                                    vx, vy, height, wz
                                    optional torso pitch
                                        |
                                        v
                             Frozen GR00T-WBC
                               Balance + Walk
                                        |
                                        v
                           Leg / waist joint targets
                                        |
                                        v
                         MuJoCo simulation + contacts
                                        |
                                        v
                              Next observation

According to the backend documentation, BiGym2 runs NVIDIA's Balance and Walk policy pair through ONNX Runtime. Their weights are not fine-tuned alongside the manipulation policy in the benchmark. GR00T-WBC should also be distinguished from a GR00T VLA model: the controller executes body commands, while the high-level model can be ACT, Diffusion Policy, or a VLA.

For move_plate, the action has 20 elements: four body commands, 14 arm joint targets, and two gripper commands. Tasks with a torso-pitch command use 21 elements. Their proprioceptive vectors have 50 or 56 dimensions, respectively. Query the environment layout rather than copying the dimensions of one task into another task's adapter.

The regular environment API normalizes actions to [-1, 1] and maps each slot to its physical range. An all-zero vector does not mean hold the current pose. It sends many slots to the middle of their range, including arm joints and height. Start with demonstrations and the example policy instead of assuming zero actions preserve everything. The official configuration lists the exact ranges.

3. Installation: start with the core environment

The repository requires Python 3.10 or newer. LeRobot export and re-rendering require Python 3.12 or newer. If you are creating a new environment and want to follow the re-rendering section, Python 3.12 is a practical choice. Core functionality runs on Linux and macOS; VR collection has separate Linux requirements.

Run these commands in the terminal of your own experiment machine:

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/swirl-uk/BiGym2.git
cd BiGym2
uv sync --python 3.12

uv sync installs dependencies from the repository configuration, including the environment, demonstration loader, evaluator, and controller. You do not need the VR extra to read demonstrations or run the ACT example. On headless Linux, configure rendering as described in the installation guide:

export MUJOCO_GL=egl
uv run python examples/loco_random_agent.py

On macOS, omit the EGL export. The random-agent example checks that the environment can reset and step. Low rewards are expected because random actions are unlikely to perform the intended task.

Do not treat the GPU used in the authors' throughput measurements as a mandatory minimum requirement. Actual costs depend on rendering, environment count, model size, and image decoding. Your first objective is to initialize one task. Measure throughput and memory on your machine before deciding whether you need more hardware.

4. Reset and inspect the first observation

You can run the following Python in an interpreter opened with uv run python, or use it in an experiment notebook:

from bigym.loco import make

env = make("move_plate")
timestep = env.reset(seed=620000)
print(env.action_spec())
print(timestep.rgb_obs.shape)
print(timestep.low_dim_obs.shape)
print(env.wholebody_action_layout())
print(env.low_dim_component_slices())
env.close()

For the official move_plate configuration, RGB has shape (3, 3, 84, 84) and proprioception has shape (50,). Camera order is head, right_wrist, left_wrist. The first two dimensions both equal three, but mean different things: cameras and color channels. Swapping cameras or confusing CHW with HWC can corrupt the input without necessarily producing an obvious shape error.

Reset includes 200 warmup control steps, equivalent to four seconds of simulated time at 50 Hz. The controller needs this period to settle the robot before the policy takes over. Removing warmup to save time changes the initial conditions and prevents a fair comparison with the official benchmark.

Each high-level control step lasts 0.02 seconds, while the scene uses 1 ms physics integration. This does not guarantee that your machine can run inference in real time. You must still measure latency and action delivery if you build a system that follows wall-clock time. Getting started documents the API and observation layout.

5. Download and understand the LeRobot demonstrations

Download one task before fetching the entire dataset:

uv run bigym-download --list
uv run bigym-download --task move_plate
uv run bigym-view --task move_plate

The official dataset is SWIRL-Lab/bigym-g1-native60. Each task occupies a LeRobot v3 folder, with training images stored as lossless PNG frames. bigym-view opens a browser viewer so you can inspect episodes and simulator state instead of working only with numerical tables.

Official task image. In the demonstrations, track when the gripper closes, when the body changes position, and when the plate is released.

Load trajectories through the Gymnasium interface:

from bigym.loco import make_gym

env = make_gym("move_plate")
demos = env.get_demos(60, only_successful=True, decode_images=False)
demo = demos[0]
print(demo["action"].shape)
print(demo["obs"]["state"].shape)
print(demo["obs"]["rgb"][0].shape)
env.close()

The loader exposes the transition relationship obs[t] --action[t]--> obs[t+1]. With decode_images=False, frames remain compressed and are decoded when accessed. This matters when RAM is limited. The documentation estimates roughly 15 GB for all 60 decoded demonstrations of a long task such as stack_blocks, compared with roughly 5 GB when keeping the images compressed. Those figures cover dataset memory, before model parameters and training batches.

Normalization is another important trap. Stored actions use the ranges recorded in batch metadata; get_demos() converts them to the environment's action ranges. If you read parquet directly and send dataset-normalized outputs into an environment using different statistics, the robot receives incorrect commands even when tensor dimensions match. Save preprocessing and inverse transforms alongside the checkpoint. Read the demonstration documentation before writing your own converter.

6. Train ACT and produce an inspectable checkpoint

ACT predicts a sequence of future actions called an action chunk. The example uses a CNN for camera images, a state branch, and a transformer decoder to generate that sequence. Training uses an L1 loss; inference uses temporal ensembling to combine overlapping predictions.

This is a simplified ACT example. It omits the original ACT's CVAE and pretrained ResNet backbone. Successfully running it therefore does not reproduce the paper's ACT results. It is a useful way to understand the pipeline and identify data errors before moving to a larger model. The implementation and command-line flags are in examples/train_act.py.

First try a short run that exercises the complete path:

uv run --with torch --with imageio-ffmpeg python examples/train_act.py \
  --task move_plate \
  --steps 1000 \
  --batch 16 \
  --workers 0 \
  --eval-episodes 5 \
  --record 1 \
  --out act_runs/move_plate_smoke

This smoke-run configuration is a suggestion for this tutorial, not the paper's training recipe. One thousand gradient steps and five evaluation episodes check data loading, the forward pass, checkpoint saving, and video output. They do not provide enough evidence to judge benchmark performance.

Once that path works, use the example defaults with a fresh output folder:

uv run --with torch --with imageio-ffmpeg python examples/train_act.py \
  --task move_plate \
  --record 3 \
  --out act_runs/move_plate_example

The script defaults to 20,000 gradient steps, batch size 64, chunk length 20, and evaluation on 100 episodes. It chooses CUDA when PyTorch can see a compatible GPU. If memory is insufficient or data-loader workers fail, lower the batch size and use --workers 0. Record your changes so later runs remain interpretable.

Outputs include policy.pt, MP4 videos, and episode state files for the viewer. A fresh output folder must be empty, which helps prevent mixing checkpoints from different experiments. A falling training loss indicates closer agreement with demonstration actions. Success rate measures whether those predictions complete an entire task through physical interaction.

7. Inference: load the policy and inspect execution

Evaluate the checkpoint again without retraining:

uv run --with torch --with imageio-ffmpeg python examples/train_act.py \
  --load act_runs/move_plate_example/policy.pt \
  --eval-episodes 100 \
  --record 3

uv run bigym-view --demo-dir act_runs/move_plate_example

At inference time, the model receives RGB and state, predicts an action chunk, and the wrapper produces an action for the current timestep. Temporal ensembling smooths transitions between chunks. The wrapper also needs to clear its history when a new episode starts; the example implements a reset() method for that purpose.

Watch the recordings slowly and ask specific questions. Does the robot approach the correct side of the plate? Do the arms collide? Does the gripper close before contact? Does the body move before the object is stable? These observations help separate perception failures, incorrect normalization, and sequencing problems. A visually convincing success video does not replace evaluation across all seeds.

For a custom model, the evaluator accepts a callable mapping timesteps to actions:

from bigym.loco.eval import evaluate

# my_policy must return actions in the environment's expected layout.
result = evaluate(
    my_policy,
    task_name="move_plate",
    method="my-policy",
)
print(result["success_rate"])
print(result["protocol_violations"])

Here, my_policy is something you must supply, not a built-in model class. If it maintains recurrent state or an action queue, implement reset() so the evaluator clears that state between episodes.

8. Move toward VLA training and 224×224 images

A VLA adds task language and pretrained knowledge to action prediction. It still needs correct camera order, action layout, normalization, and execution timing. Before spending GPU time on fine-tuning, verify that a smaller policy reads the dataset correctly and controls the intended environment.

BiGym2 stores simulator state for each frame, allowing you to re-render cameras at a different resolution instead of merely enlarging the existing 84×84 images:

uv sync --extra lerobot
uv run bigym-rerender-lerobot --task move_plate --size 224 224

This requires Python 3.12 or newer and writes a copy to bigym-g1-native60-224/move_plate by default. An environment loading that copy must use the matching camera shape. Changing the visual input is an experimental choice that should be disclosed with the results.

There is a reproduction detail to watch: the paper describes π0.5 receiving the 84×84 views with model input processing at 224×224. Native re-rendering at 224×224 supplies different visual detail from resizing those original inputs. Do not treat the two experiments as identical. Current repository documentation also mentions a derived 224×224 view; check the paper version, dataset metadata, and actual trainer configuration when reproducing results.

The repository does not offer a flag that turns train_act.py into a π0.5 trainer. You need the corresponding model trainer and an observation/action adapter. Write down the data contract first: camera names and dimensions, language prompt, proprioceptive dimensions, action representation, normalization statistics, and how many actions are executed before replanning. Resolve this contract before tuning a learning rate.

For background on connecting this workflow to teleoperation, see collecting G1 data for GR00T. It explains why cameras and action semantics must remain consistent from recording through deployment.

9. Results: inspect strengths and unresolved tasks

The following success rates come from Table II of the BiGym 2.0 paper. Values are percentages reported by the authors, not measurements from the tutorial commands.

π0.5 has the highest mean in the main comparison but performs worse than ACT on pick_box. No policy leads every task. Beyond those nine tasks, stack_blocks remains difficult: ACT achieves 0% and CQN-AS approximately 1%. π0.5 was not evaluated on that task.

Aggregation differs too. ACT uses three training runs, averaging each run's last five checkpoints with 100 episodes per checkpoint. π0.5 uses one training run per task, pooling five final checkpoints with 50 episodes each. The columns therefore do not provide equivalent evidence about stability across training runs.

An example of contact-rich manipulation: body height and posture directly affect how the arms reach the fixture.

10. Evaluation and mistakes that distort conclusions

The repository protocol uses 100 episodes with seeds 620000 + i. Success requires both task completion and no fall during the episode. The task predicate must hold for one second. An object that briefly touches the target and bounces away is not a completed placement.

A fall is latched when pelvis tilt exceeds 53 degrees or pelvis height drops below 0.30 m. Falling does not necessarily end the episode immediately, so final reward alone is insufficient for scoring. Use the evaluator and report the fell share when analyzing rollouts.

Before comparing models, check four things:

1. Inputs: correct camera order, dtype, and shape; no accidental object simulator state in a visual policy's inputs.
2. Actions: matching checkpoint/environment normalization; no missing or shifted slot on pitch-enabled tasks.
3. Reset: preserve warmup and controller-state restoration; clear the action queue between episodes.
4. Protocol: inspect protocol_violations, report episode count, and disclose camera or controller changes.

Do not set controller=None and report that run as the standard benchmark. That option removes the lower-body controller and changes the execution substrate. A five-episode evaluation also produces no official leaderboard record. Changing camera resolution can make a run unofficial according to the evaluator, even when it is a useful research experiment.

If you collect new demonstrations, the vr and lerobot extras have conflicting setuptools requirements. Do not install both in a single shared uv sync configuration. Separate collection and export environments following the installation guide. Reading the published demonstrations does not require either extra.

The most useful outcome of this exercise is a checkpoint tied to a specific task, explicit preprocessing, inspectable rollouts, and a success-rate report with an honest protocol description. BiGym 2.0 helps establish that chain before moving a whole-body policy to physical hardware. Simulation scores alone do not establish sim2real performance.

Related Posts

• LeRobot: a framework for learning policies from demonstrations
• Collecting G1 data for GR00T
• WholeBodyVLA: a training pipeline tutorial