Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success

In plain words

Robot policies can be built by taking a large vision-language model and training it to output robot motions. This paper asks a practical question: when you adapt such a model to a new robot and task, which of the many training choices matter? It starts from OpenVLA, a model that writes out its robot actions one piece at a time, like typing a sentence.

The authors find that producing a whole short sequence of future actions in one step, instead of one piece at a time, makes the robot much faster and also tends to make it more successful. Predicting the actions as continuous numbers with a simple regression loss, rather than as chopped-up tokens, helps further. They package these choices into a fine-tuning recipe called OFT and show it on a simulated benchmark and on a real two-armed robot.

TL;DR

• Fine-tuning recipe OFT: parallel decoding, action chunking (K=8 in the simulation experiments), continuous actions, and an L1 regression head §4.2.
• On LIBERO, the recipe reaches 97.1% average success with wrist camera and proprioceptive state inputs, versus 76.5% for fine-tuned OpenVLA; the paper reports 26x throughput over baseline OpenVLA tab. 1 tab. 2.
• A FiLM-augmented variant (OFT+) is used on a real bimanual ALOHA robot at 25 Hz; the paper reports it matching or exceeding pi0, RDT-1B, ACT and Diffusion Policy on each of its four tasks §6.3 tab. 10–tab. 13.
• Author-reported, from a small number of trials per task on one base model.

Problem

OpenVLA generates its 7 action dimensions as discrete tokens, one after another. The authors report 3–5 Hz for autoregressive generation, too slow for the 25–50+ Hz control they target on bimanual robots §1. Action chunking, which helps policies in prior work, becomes a K-fold latency cost in this scheme, since K×D sequential decoder passes are needed §3. Existing fine-tuning recipes for VLAs (LoRA on the autoregressive model, diffusion-based VLAs) were reported to be slow or unreliable on bimanual manipulation §1 §4.1. The paper's setting is fine-tuning on small demonstration sets (e.g. 500 LIBERO demonstrations per suite, 20–300 ALOHA demonstrations per task) rather than pretraining §4.1 §6.1.

Method

Base model. OpenVLA is a 7B-parameter policy: a fused SigLIP and DINOv2 vision encoder, a Llama-2 7B language model, and a projector into the language embedding space. The original model outputs one 7-D action (3 position, 3 orientation, 1 gripper) as discrete tokens, trained with next-token cross-entropy §3 app. A.1.

Design components studied §4.1 fig. 2:

1. Action generation: autoregressive versus parallel decoding.
2. Action representation: 256-bin discrete tokens versus continuous values from an MLP head.
3. Objective: next-token prediction versus L1 regression versus conditional diffusion (continuous actions).

OFT recipe §4.2 §5.5:

1. Parallel decoding with action chunking. Empty action slots are appended to the decoder input, one block of D slots per future step, and the attention mask becomes bidirectional so all K×D actions come out of one forward pass §4.2 app. A.2.
2. A continuous action head replaces the token output layer. The head maps the final hidden states at the action slots to normalized actions in [-1, +1] §4.2 app. A.1.
3. Training minimizes the mean L1 error between predicted and ground-truth action chunks §4.2 app. A.4.
4. Inference executes the whole predicted chunk open-loop before querying the model again §5.1 §6.1.

Additional inputs §5.4 app. A.1: each extra camera view passes through the same encoder and adds 256 patch embeddings; robot proprioceptive state is projected into one extra embedding; everything is concatenated before the decoder. This doubles the visual tokens from 256 to 512 for one extra wrist view.

FiLM for language grounding (OFT+) §4.3 fig. 8 app. A.3: the mean of the instruction embeddings is projected to a scale vector gamma and a shift vector beta. Each vision transformer block has its own projector pair. Both vectors have the width of the ViT hidden state and are applied identically to every patch token. The modulation is FiLM(F | gamma, beta) = (1 + gamma) ⊙ F + beta §4.3; the authors multiply by (1 + gamma) so that the pretrained activations are preserved at initialization app. A.3. FiLM is used only on ALOHA §4.3.

# OFT training step (parallel decoding, chunking, L1)  [§4.2] [app. A.2]
# Input:  images (B, V, 3, 224, 224), proprio (B, P), instruction
# Output: scalar loss

def training_step(images, proprio, instruction, target_actions):
    # Encode each view and project its 256 patches to the LLM width
    # [§4.2] [app. A.1]
    image_tokens = project_patches(vision_encoder(images))
    # (B, V*256, d)

    # Project the robot state to one extra token  [app. A.2]
    state_token = project_state(proprio)                 # (B, 1, d)

    # Append K*D empty action slots. They differ only in position
    # encoding and get no teacher-forced tokens  [§4.2] [app. A.2]
    action_slots = empty_embeddings(K * D)               # (B, K*D, d)

    # One pass with bidirectional attention over the whole sequence
    # [§4.2] [app. A.2]
    hidden = llm_bidirectional(
        concat(image_tokens, state_token, instruction, action_slots),
    )

    # Read the hidden states at the action slots and map each step's
    # slots to a 7-D action with the L1 head  [§4.2] [app. A.1]
    action_hidden = hidden[:, -K * D:]
    chunk_hidden = reshape(action_hidden, (B, K, D * d))
    predicted = l1_head(chunk_hidden)                    # (B, K, D)

    # Mean absolute error over the whole chunk  [§4.2] [app. A.4]
    # [code: vla-scripts/finetune.py#L386-L390]
    return mean(abs(predicted - target_actions))

# Inference loop for OFT  [§5.1] [§5.3]
# Input:  policy, robot, instruction
# Output: none (runs until the episode ends)

def run_episode(policy, robot, instruction):
    # Re-plan only after the whole chunk has been executed  [§5.1]
    while not robot.done():
        observation = robot.observe()
        # One forward pass returns K actions  [§5.3]
        chunk = policy(observation, instruction)         # (K, D)
        for action in chunk:
            # Open-loop execution of the full chunk  [§5.1] [§6.1]
            robot.execute(action)

# FiLM modulation of visual features  [§4.3] [app. A.3]
# Input:  features (N, d), instruction_embeds (T, d_lang)
# Output: modulated features (N, d)

def film(features, instruction_embeds, projector):
    # Average the instruction tokens into one vector  [§4.3]
    pooled = mean(instruction_embeds, axis=0)
    # Per-block projector gives gamma and beta, each of width d  [app. A.3]
    gamma, beta = projector(pooled)
    # Same scale and shift for every patch, FiLM-style  [§4.3]
    # [app. A.3]
    return (1 + gamma) * features + beta

Hyperparameters

Recipe values are from the paper's tables. Values the paper does not give are marked "not stated". Code defaults are cited separately.

Results

Author-reported. Setup: OpenVLA-7B fine-tuned per LIBERO suite with LoRA, evaluated over 500 trials per suite (10 tasks × 50 episodes) tab. 1 §5.1.

LIBERO, third-person image + language only (filtered dataset) tab. 1:

LIBERO, with wrist image and proprio tab. 1:

Efficiency, one A100, 224×224 image, 100 queries tab. 2:

Combined-suite policy and FiLM tab. 14 app. A.7: one policy for all four suites reaches 96.8 average; adding FiLM gives 97.0. Fine-tuning the underlying Prismatic VLM with the OFT recipe, without OpenVLA's robot pretraining, reaches 91.9 average tab. 15.

Bridge (real WidowX, 6 tasks, 10 trials each) tab. 16: OpenVLA-OFT 69.2 average versus OpenVLA 65.8, with the OpenVLA scoring criteria.

ALOHA, rubric score in percent (average over trials) tab. 10–tab. 13:

The text claims up to 15 points absolute average success over the baselines §1; the per-task averages in Figure 4 were not extracted here.

ALOHA inference tab. 3: OpenVLA 1.8 Hz / 0.543 s; OFT+ 77.9 Hz / 0.321 s; RDT-1B 84.1 Hz / 0.297 s; Diffusion Policy 267.4 Hz / 0.090 s; pi0 291.6 Hz / 0.086 s; ACT 432.8 Hz / 0.058 s. OFT+ is faster than OpenVLA and close to RDT-1B, but slower than Diffusion Policy, pi0 and ACT. The paper attributes this to their smaller architectures and, for pi0, a JAX implementation §6.4.

FiLM ablation (ALOHA language following): without FiLM, language following falls to chance (33%) on the two language-dependent tasks §6.3 fig. 5.

Paper vs code

Checked against the official repository https://github.com/moojink/openvla-oft at commit e4287e94541f459edc4feabc4e181f537cd569a8 (MIT, Moo Jin Kim, Chelsea Finn, Percy Liang), found linked from the project page. I read vla-scripts/finetune.py and prismatic/models/action_heads.py. The exact LIBERO and ALOHA run commands in experiments/ were not checked.

• Action head depth. The paper describes a 4-layer ReLU MLP head [app. A.1]. The code's head has an input projection, two residual blocks and an output projection, which is four linear layers in total [code: prismatic/models/action_heads.py#L59-L107]. Consistent, but the residual connections are not described in the paper.
• LoRA alpha. Not stated in the paper. Code sets alpha to min(rank, 16), i.e. 16 for rank 32 finetune.py.
• LR decay. Paper: 10× decay after 100K steps tab. 4. Code: MultiStepLR with gamma 0.1 at num_steps_before_decay (default 100K), stepped once per optimizer step finetune.py [code: vla-scripts/finetune.py#L1094-L1097]. Consistent.
• Defaults differ from the recipe. Code defaults are use_proprio=False, use_film=False, num_images_in_input=1, max_steps=200K, batch_size=8 per device finetune.py. The OFT recipe needs these set explicitly; the defaults are not the headline configuration.
• Stopping rule. The paper's convergence criterion (L1 < 0.01) does not appear in the loop; training stops at max_steps [code: vla-scripts/finetune.py#L1135-L1136].
• Loss. Code uses torch L1Loss on predicted and ground-truth actions finetune.py. Normalization details come from the data pipeline and were not traced.
• Diffusion head. Paper: 4-layer noise predictor app. A.2. Code's NoisePredictionModel uses two residual blocks around the same MLP shape action_heads.py. Consistent. Diffusion is not part of the recipe.

Pitfalls

• Two headline numbers. The 97.1% LIBERO average needs wrist image and proprio inputs and the filtered dataset. The third-person-only run reaches 95.3% tab. 1 app. A.4.
• Checkpoint selection on the test runs. Checkpoints were evaluated every 50K steps and the best is reported §5.1 app. A.4. The reported numbers are therefore optimistic relative to a fixed-step protocol.
• Unstated values. LoRA alpha, LoRA dropout, warmup and the random seeds are not stated in the paper. Code gives alpha and dropout defaults and no warmup by default. Seeds not stated.
• Stopping criterion. The text uses an L1 < 0.01 criterion app. A.4, but the loop in code trains to max_steps. Plan step budgets from the tables, not from the criterion.
• Open-loop chunks. The whole chunk is executed before replanning §5.1 §6.1. Chunk length K trades latency against closed-loop reactivity. The tables use only K=8 (LIBERO) and K=25 (ALOHA) tab. 2 tab. 3.
• Parallel decoding vs expressiveness. The authors note parallel decoding may be less expressive than autoregressive decoding but report no degradation in their tasks §4.2. They also list multimodal demonstrations as a limitation: L1 regression tends toward the median mode §8.
• Cost of the extra inputs. Extra views double visual tokens (256 to 512) and add roughly 0.04 s of latency in the LIBERO setting tab. 2. The ALOHA OFT+ model has 853M trainable parameters, of which 456M are FiLM projectors tab. 5.
• FiLM is ALOHA-specific. It was added for language grounding in the real robot and is not used on LIBERO §4.3 app. A.4. The authors say it is not needed for language following on BridgeData V2 §5.6.
• Small evaluation sets. ALOHA tasks use 10 to 24 trials each §6.1. Per-task differences of a few trials are within noise, and the paper gives no variance.
• Hardware-dependent speed. Latency and throughput were measured on an NVIDIA A100 in PyTorch, except pi0 in JAX tab. 3. Expect different numbers on other hardware.
• Base model dependence. Results are for OpenVLA-7B. The ablation shows the robot pretraining helps (97.1 vs 91.9) tab. 15, so a non-robot base is not a drop-in replacement.
• Diffusion comparison scope. The diffusion variant is evaluated with 50 DDIM steps at train and test time; the efficiency table reports reduced step counts with lower success tab. 2.

Lineage

• Builds on: OpenVLA, the base model; OFT changes its decoding, outputs and objective, not its pretraining §3.
• Compared against: OpenVLA with LoRA, Diffusion Policy, ACT, Octo, DiT Policy, Seer, MDT, pi0 (with and without FAST), RDT-1B §5.1 §6.2 tab. 1.
• Superseded by: not stated.