unifolm-vla + Unitree G1 (Post 4): fine-tuning from Qwen2.5-VL-7B — 8-GPU and single-GPU LoRA

This is post 4 of the unifolm-vla + Unitree G1 series. The previous post produced an RLDS dataset. This post: fine-tuning the VLA model.

The problem: Unifolm-VLM-0 is not public

unifolm-vla has two components:

• VLM backbone: Qwen/Qwen2.5-VL-7B-Instruct — publicly available on HuggingFace ✅
• Action head: specialized weights for predicting robot actions
• Unifolm-VLM-0: Unitree's continued-pretrained checkpoint on robot data — not yet public ❌ (as of this writing)

The official repo references Unifolm-VLM-0 but the path is a placeholder — it can't be downloaded. The solution: fine-tune directly from Qwen2.5-VL-7B-Instruct — the VLM already understands the visual world, it just needs to learn robot control.

Trade-offs when using Qwen2.5-VL-7B-Instruct instead of Unifolm-VLM-0:

Practical conclusion: with 100-150 demos and ~120 epochs, Qwen2.5-VL-7B-Instruct reaches ~80% of Unifolm-VLM-0's performance on simple tasks. Enough to test the full pipeline and learn.

Approach A: 8-GPU Full Fine-tune (Official)

The official repo method — requires 8× NVIDIA GPUs.

GPU requirements

Modify config to use public checkpoint

Find the training config and change pretrained_model_path:

cd ~/unifolm_ws/unifolm-vla

# Find config file
find . -name "*.yaml" -path "*/config/*" | head -10

# Change the checkpoint path
# From: pretrained_model_path: "/path/to/Unifolm-VLM-0"
# To:   pretrained_model_path: "$HOME/models/Qwen2.5-VL-7B-Instruct"

# src/unifolm_vla/config/train_config.yaml

model:
  # CHANGE: use Qwen2.5-VL-7B-Instruct instead of Unifolm-VLM-0
  pretrained_model_path: "/home/user/models/Qwen2.5-VL-7B-Instruct"
  
dataset:
  rlds_data_dir: "/home/user/datasets/g1_pickplace_rlds"
  dataset_name: "g1_pickplace"
  robot_type: "g1_dex3"
  
training:
  num_epochs: 120       # increased from ~80 since starting from non-specialized checkpoint
  batch_size: 4         # per GPU
  learning_rate: 1e-4
  warmup_steps: 200     # longer warmup since model needs more adaptation
  save_every_n_epochs: 10
  
  gradient_checkpointing: true
  bf16: true

Run training

conda activate unifolm
cd ~/unifolm_ws/unifolm-vla

accelerate launch \
  --config_file src/unifolm_vla/config/deepseeds/deepspeed_zero2.yaml \
  --num_processes 8 \
  src/unifolm_vla/training/train_unifolm_vla.py

Expected terminal output:

[DeepSpeed] Using ZeRO Optimization Stage 2
Epoch 1/120 | Step 25 | Loss: 2.847 | LR: 5.0e-5
Epoch 1/120 | Step 50 | Loss: 2.341 | LR: 1.0e-4
...
Epoch 10/120 | Val Loss: 1.234 | Saved checkpoint: ./checkpoints/epoch_10/
...
Epoch 120/120 | Val Loss: 0.487 | Best checkpoint: epoch_110

Loss target: starts ~2.5-3.0, should reach ~0.4-0.6 after 120 epochs for simple tasks.

Approach B: Single-GPU QLoRA (Beginner Workaround)

If you only have 1× RTX 4090 (24GB), fine-tune Qwen2.5-VL-7B with LoRA via PEFT. Note: this is a custom script, not in the official repo. You'll need to integrate it manually with the unifolm-vla action head.

Setup

conda activate unifolm
pip install peft bitsandbytes transformers accelerate

python -c "import peft; print('peft version:', peft.__version__)"

Custom LoRA training script

Create train_lora_single_gpu.py:

"""
Custom single-GPU LoRA fine-tuning for unifolm-vla.
Trains Qwen2.5-VL-7B-Instruct with QLoRA on G1 robot data.
NOTE: This is a workaround — not official unifolm-vla API.
"""

import torch
import numpy as np
from pathlib import Path
from transformers import (
    Qwen2VLForConditionalGeneration,
    AutoProcessor,
    BitsAndBytesConfig,
)
from peft import LoraConfig, get_peft_model, TaskType
import h5py
import glob
from torch.utils.data import Dataset, DataLoader

# ── 1. Model with QLoRA ───────────────────────────────────────────────────────

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
)

model = Qwen2VLForConditionalGeneration.from_pretrained(
    "/home/user/models/Qwen2.5-VL-7B-Instruct",
    quantization_config=bnb_config,
    device_map="auto",
    torch_dtype=torch.bfloat16,
)

processor = AutoProcessor.from_pretrained(
    "/home/user/models/Qwen2.5-VL-7B-Instruct"
)

# ── 2. LoRA config ────────────────────────────────────────────────────────────

lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=16,               # LoRA rank — increase to 32 if VRAM allows
    lora_alpha=32,
    target_modules=[
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
    lora_dropout=0.05,
    bias="none",
)

model = get_peft_model(model, lora_config)
model.print_trainable_parameters()
# Expected: trainable params: ~40M / 7B (0.6%) — fits single GPU

# ── 3. Dataset ────────────────────────────────────────────────────────────────

class G1RobotDataset(Dataset):
    def __init__(self, hdf5_dir: str, instruction: str = "pick up the red cup"):
        self.files = glob.glob(f"{hdf5_dir}/train/*.hdf5")
        self.instruction = instruction
    
    def __len__(self):
        return len(self.files)
    
    def __getitem__(self, idx):
        with h5py.File(self.files[idx], 'r') as f:
            frames = f['obs/left_wrist_rgb'][:]
            actions = f['action'][:]
            instruction = f['language_instruction'][()].decode()
        
        mid_frame = frames[len(frames) // 2]   # (H, W, 3), uint8
        mid_action = actions[len(actions) // 2]  # (28,)
        
        return {
            "image": mid_frame,
            "instruction": instruction or self.instruction,
            "action": torch.tensor(mid_action, dtype=torch.float32),
        }

dataset = G1RobotDataset("/home/user/datasets/g1_pickplace_hdf5")
dataloader = DataLoader(dataset, batch_size=1, shuffle=True)

# ── 4. Training ───────────────────────────────────────────────────────────────

optimizer = torch.optim.AdamW(
    filter(lambda p: p.requires_grad, model.parameters()),
    lr=2e-4, weight_decay=0.01,
)

# Simple action prediction head — maps VLM hidden states → joint commands
action_head = torch.nn.Linear(3584, 28).to("cuda")  # 3584 = Qwen2.5-VL-7B hidden dim
head_optimizer = torch.optim.AdamW(action_head.parameters(), lr=2e-4)

num_epochs = 50

for epoch in range(num_epochs):
    total_loss = 0
    for batch in dataloader:
        messages = [{
            "role": "user",
            "content": [
                {"type": "image", "image": batch["image"][0].numpy()},
                {"type": "text", "text": f"Robot task: {batch['instruction'][0]}. Predict action."}
            ]
        }]
        
        text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
        inputs = processor(
            text=[text],
            images=[batch["image"][0].numpy()],
            return_tensors="pt", padding=True
        ).to("cuda")
        
        with torch.amp.autocast("cuda", dtype=torch.bfloat16):
            outputs = model(**inputs, output_hidden_states=True)
            hidden = outputs.hidden_states[-1][:, -1, :]
            predicted_action = action_head(hidden.float())
            loss = torch.nn.functional.mse_loss(predicted_action, batch["action"].to("cuda"))
        
        optimizer.zero_grad()
        head_optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        head_optimizer.step()
        total_loss += loss.item()
    
    if (epoch + 1) % 10 == 0:
        print(f"Epoch {epoch+1}/{num_epochs} | Loss: {total_loss/len(dataloader):.4f}")

# ── 5. Save ───────────────────────────────────────────────────────────────────

save_dir = Path("./checkpoints/lora_g1_pickplace")
model.save_pretrained(save_dir / "lora_weights")
torch.save(action_head.state_dict(), save_dir / "action_head.pt")
processor.save_pretrained(save_dir / "processor")
print(f"Saved to {save_dir}")

Run LoRA training

conda activate unifolm
python train_lora_single_gpu.py

# VRAM usage breakdown:
# Qwen2.5-VL-7B 4-bit quantized: ~6GB
# LoRA adapters: ~0.5GB
# Action head: negligible
# Activations + optimizer states: ~8GB
# TOTAL: ~14-15GB → fits comfortably in RTX 4090 (24GB)

Expected output:

trainable params: 41,943,040 || all params: 7,615,832,064 || trainable%: 0.55
Epoch 10/50 | Loss: 0.8341
Epoch 20/50 | Loss: 0.4123
Epoch 30/50 | Loss: 0.2847
Epoch 40/50 | Loss: 0.2156
Epoch 50/50 | Loss: 0.1934
Saved to ./checkpoints/lora_g1_pickplace

Comparing both approaches

Recommendation:

• Beginner with 1 GPU: use Approach B to learn the pipeline. Performance is enough to verify the end-to-end flow works.
• Research / production: Approach A, or rent cloud GPUs (Lambda Labs, RunPod, Vast.ai have 8× A100 for ~$5-10/hr).

Cloud GPU for Approach A

If you don't have 8 GPUs but need full performance:

# RunPod: 8× A100 SXM4 80GB, ~$8/hour
# Training 100 demos ≈ 3 hours = ~$24

# Sync data and code to cloud
rsync -avz $HOME/datasets/g1_pickplace_rlds/ runpod:/workspace/datasets/
rsync -avz ~/unifolm_ws/unifolm-vla/ runpod:/workspace/unifolm-vla/

# Run training on cloud
accelerate launch \
  --config_file src/unifolm_vla/config/deepseeds/deepspeed_zero2.yaml \
  --num_processes 8 \
  src/unifolm_vla/training/train_unifolm_vla.py

# Download checkpoint
rsync -avz runpod:/workspace/unifolm-vla/checkpoints/ $HOME/checkpoints/

Training loss target

Early stopping: if val loss rises for 3 consecutive evaluations → use best checkpoint before the rise.

Next: Deploy inference server + connect real G1 + run locomotion in parallel.

References

• unifolm-vla GitHub
• Qwen2.5-VL-7B-Instruct (HuggingFace)
• PEFT / LoRA documentation
• BitsAndBytes QLoRA
• DeepSpeed Zero Redundancy Optimizer

Related posts

• Post 3: Data pipeline JSON → RLDS
• Post 5: Deploy inference server + G1 locomotion
• GR00T N1 + G1: Fine-tuning with Isaac Lab data