Octo: An Open-Source Generalist Robot Policy

In plain words

Teaching a robot a new skill usually means collecting data for that exact robot and task, and the result rarely works anywhere else. Octo is one network that is first trained on a large pile of robot demonstrations gathered from many robots. You give it camera views and a goal, either a written instruction or a picture of the finished scene. It answers with the motion the robot should make next.

The network is built from pieces that can be swapped. Each camera image, the instruction and each readout slot become tokens, and the same transformer reads all of them. A new camera or a different arm means adding a small new piece and keeping the rest. Then about 100 demonstrations are enough to finetune it. Its motion is produced the way image diffusion models make pictures: it starts from random noise and cleans it up over a short series of steps. The code and the released model weights are open, so the whole setup can be reproduced and reused.

TL;DR

Octo is an open transformer policy for robot manipulation, pretrained on about 800k trajectories from the Open X-Embodiment collection. Its inputs and outputs are tokens, so sensors and action spaces can change at finetuning time while the pretrained transformer is kept. Actions come from a small diffusion head over action chunks. The public code is MIT-licensed and matches most of the appendix hyperparameters, but its released batch size (512) differs from the paper (2048).

Problem

Robot policies are usually trained for one robot and one task, and they generalize narrowly §1. Earlier generalist robot policies fix the sensor set and the action format, do not finetune well to new domains, and the largest ones are not public §1. The paper aims at an open, pretrained policy that accepts different camera setups, different robots and both language and goal-image task specifications, and that can be adapted to new observations and action spaces with a small target dataset and a modest compute budget §1.

Method

Overview. Octo has three parts: input tokenizers, a transformer backbone and readout heads. The tokenizers turn the task (language or goal image) and the observation history into tokens. The backbone processes them as one sequence. Readout heads decode actions §3.1.

Tokenizers §3.1 app. D. Language is passed through a frozen T5-base encoder (111M parameters) that gives 16 language tokens. Images go through a shallow convolution stack and are cut into 16×16 patches, which gives 256 tokens for the third-person camera and 64 tokens for the wrist camera. Learned position embeddings are added before the tokens are arranged in sequence §3.1.

Backbone and attention §3.1. Tokens are arranged by time step. Observation tokens attend causally to the same and earlier time steps and to the task tokens. Missing inputs are masked out entirely, so a dataset without language still trains §3.1. Readout tokens are added per time step. A readout token attends to the task and earlier observations, and no observation or task token attends to it, so readouts only read the state §3.1 octo_module.py.

Action head §3.3 app. D. The action readout embedding is averaged and given to a diffusion head. The head is a 3-block residual MLP with hidden size 256, layer normalization and 20 diffusion steps on a cosine noise schedule. The transformer runs once per prediction. The denoising loop runs only inside the head. The paper reports that this beats MSE heads and discretized action heads in zero-shot and finetuning tests §3.3 tab. 2.

Training objective §3.3. The training objective is the standard DDPM objective. Noise is added to dataset actions and the denoiser learns to undo it. The sampler is the update below eq. 1. The paper says the coefficients come from the cosine schedule but gives no numeric values for them.

x^{k-1} = \alpha \big( x^k - \gamma\, \epsilon_\theta(x^k, e, k) + \mathcal{N}(0, \sigma^2 I) \big) eq. 1

# Training loss for the diffusion action head  [§3.3]
# Input:  embeddings (B, W, d) from one transformer pass per time step
# Input:  actions (B, W, H, A), H = chunk length, A = action dim
# Output: scalar loss

def diffusion_loss(embeddings, actions, pad_mask):
    # Flatten chunk and action dims into one vector per time step  [app. D]
    targets = flatten(actions)                       # (B, W, H * A)
    # Clip the targets to [-5, 5] (code only, see Paper vs code)
    targets = clip(targets, -5.0, 5.0)
    # Draw one random diffusion step per sample  [app. D]
    steps = uniform_int(0, num_steps, shape=(B, W))  # num_steps = 20
    # Draw Gaussian noise shaped like the targets  [§3.3]
    noise = normal(shape=targets.shape)
    # Mix targets and noise according to the cosine schedule  [§3.3]
    noisy = sqrt(alpha_hat[steps]) * targets
    noisy = noisy + sqrt(1 - alpha_hat[steps]) * noise
    # Predict the noise from the embedding, noisy input and step  [§3.3]
    predicted = denoiser(embeddings, noisy, steps)
    # Mean squared error on unpadded entries  [§3.3]
    loss = masked_mse(predicted, noise, pad_mask)
    # Scale by the action dimension (code only)
    return loss * action_dim

# Sample one action chunk at inference  [§3.3] [eq. 1] [app. D]
# Input:  embeddings from one transformer pass
# Output: action chunk in normalized units, shape (H, A)

def sample_actions(embeddings):
    # Start from pure Gaussian noise  [§3.3]
    noisy = normal(shape=(H * A,))
    # Denoise from the last step to the first  [app. D]
    for step in reversed(range(num_steps)):
        # Predict the noise for the current iterate  [§3.3]
        eps = denoiser(embeddings, noisy, step)
        # Coefficients of eq. 1; values are in the code, not the paper
        alpha = 1 / sqrt(alphas[step])
        gamma = (1 - alphas[step]) / sqrt(1 - alpha_hats[step])
        sigma = sqrt(betas[step])
        noisy = alpha * (noisy - gamma * eps)
        # Add fresh noise, but not at the last step (differs from eq. 1)
        if step > 0:
            noisy = noisy + sigma * normal(shape=noisy.shape)
        # Clip the iterate to [-5, 5] (code only)
        noisy = clip(noisy, -5.0, 5.0)
    # Return normalized actions; un-normalize with dataset statistics
    return noisy

Code locations for the two blocks above: the loss is in action_heads.py, the sampler in action_heads.py, the denoiser MLP in diffusion.py and diffusion.py, and the cosine schedule in diffusion.py.

Finetuning §3.3 §4.2. The paper updates all parameters and reports that this beats freezing parts of the pretrained model. The recipe is the same for every setup: about 100 target trajectories, 50k steps, cosine decay with linear warmup. New observation inputs get a new lightweight encoder, new action spaces get a new head, and the transformer weights are kept §3.1 §3.3.

Training data §3.2 app. C tab. 3. The mixture is 25 datasets from Open X-Embodiment §3.2. The curation removes datasets without image streams and datasets that do not use delta end-effector control. It also removes datasets that are too repetitive, too low-resolution or too niche. Remaining datasets are split into "more diverse" and "less diverse", and the first group is weighted double. Missing camera channels are zero-padded. The gripper is aligned so that +1 means open and 0 means closed §3.2. The sampling fractions are listed in tab. 3. The pretraining data totals about 800k trajectories §1.

Source: tab. 3, which the paper introduces in app. C.

Training details §3.4 app. D. Two sizes are trained: Octo-Small, with a ViT-S sized backbone, and Octo-Base, with a ViT-B sized backbone. The optimizer is AdamW with weight decay 0.1 and the inverse square root learning rate schedule with linear warmup. Gradients are clipped at 1.0. Two frames of observation history are used, and goals are relabelled with a uniformly sampled future state from the same trajectory. Augmentation is applied, and per training example either the language instruction or the goal image is zeroed out. Datasets without language always use goal images §3.4. The ViT-B model trains for 300k steps on TPU v4-128 in about 14 hours §3.4.

Model sizes tab. 5.

The paper also names Octo-Tiny at 10M parameters §4.3, but its architecture is not in the appendix table.

Code and checkpoints §3.5. The paper releases pretrained checkpoints, JAX finetuning scripts, the JAX pretraining pipeline and data loaders for Open X-Embodiment that work with JAX and PyTorch §3.5.

Hyperparameters

Values from the paper unless marked code. Where the paper is silent, the column says so.

Results

Author-reported. Setup: six finetuning setups on real robots, each with about 100 target demonstrations, same finetuning recipe in every setup §4.2 app. F.

Finetuning tab. 1. Success rate averaged over 20 trials per setup tab. 1 (but see Pitfalls for Berkeley Bimanual). * means new observation input (force-torque). † means new action space (joint position control).

Source: tab. 1 §4.2.

Zero-shot control §4.1 fig. zero_shot. On several in-distribution tasks from the pretraining data, Octo averages 29% higher success than RT-1-X (35M parameters) with language instructions. Its results on the WidowX and RT-1 Robot tasks are similar to RT-2-X (55B parameters). With goal images on WidowX, success is 25% higher than with language. Ten trials per task app. F. Bar values are only in the figure, not in the text.

Design ablations tab. 2 tab. 6. All on Octo-Small, WidowX setup, averaged over two language-conditioned and two goal-conditioned tasks, 40 trials tab. 2.

Per-task breakdown tab. 6:

Zero-shot generalization tab. 7. Octo-Small, WidowX. Each row averages two tasks, with 20 trials across them:

Model scale fig. scaling. Zero-shot success rises with model size on UR5 and WidowX tasks. Base is more robust to initial scene changes than Small and is less prone to early grasp attempts §4.3. No numbers in the text.

Paper vs code

Code: github.com/octo-models/octo at 241fb3514b7c40957a86d869fecb7c7fc353f540, MIT. The link comes from the paper's project page [octo-models.github.io]; no further search was needed. Files read: scripts/configs/config.py, octo_pretrain_config.py, finetune_config.py, octo/model/components/action_heads.py, diffusion.py, octo/model/octo_module.py, octo/model/octo_model.py, octo/utils/train_utils.py, octo/data/utils/task_augmentation.py, octo/data/oxe/oxe_dataset_mixes.py, octo/data/dataset.py, scripts/train.py, and examples/.

• Batch size. Paper: 2048 §3.4. Released pretraining config: 512 octo_pretrain_config.py. The train script asserts divisibility by device count and divides by process count, so 512 is a global batch train.py train.py. The released config does not reproduce the paper's batch.
• Training target. Paper text: the denoiser learns to "reconstruct the original action" §3.3. Code predicts the added noise and takes MSE on it, the standard DDPM form action_heads.py. Read the text as loose wording for the same objective.
• Sampler noise. Eq. 1 adds noise at every step eq. 1. Code adds it only for steps above zero action_heads.py. The final step is noise-free.
• Clipping. Code clips training targets to [-5, 5] action_heads.py and clips every sampler iterate to the same range action_heads.py. The paper says nothing about clipping.
• Loss scale. Code multiplies the loss by the action dimension, which is 7 in pretraining action_heads.py. The paper says nothing. This matters for the global-norm gradient clip at 1.0, which fires at a different point.
• Learning-rate schedule. Paper: inverse square root with warmup §3.4 app. D. Code: peak divided by sqrt((step + 10000) / 10000) after warmup train_utils.py. The timescale 10000 is code only.
• Finetuning hyperparameters. The paper gives only the pretraining weight decay of 0.1. Finetuning code uses 0.01 finetune_config.py, batch size 256 finetune_config.py, window 1 finetune_config.py and chunk 4 finetune_config.py. The paper states two frames of history for training §3.4 and a chunk of 64 with 12 executed actions for the Bimanual setup app. F. That Bimanual config is not in the repository.
• Language and goal dropout. Paper: per example, the language or the goal image is zeroed out §3.4. Code: the mask is drawn per time step task_augmentation.py. The pretraining config switches to delete-and-rephrase with a 50% rephrase rate octo_pretrain_config.py. It does not set the image-keep probability, which the base config sets to 0.5 config.py. Whether the base value carries over depends on how ConfigDict merges nested dicts. I did not verify this.
• Paraphrases. Code paraphrases instructions with a dataset from an external repository (rail-berkeley/OXE_paraphrases) octo_pretrain_config.py. The paper does not mention it.
• Augmentation values. Paper: crop and colour jitter, no values app. D. Code values are listed in the Hyperparameters table octo_pretrain_config.py.
• Readout pooling. Paper: the readout token is the action embedding. Code averages the readout tokens octo_pretrain_config.py action_heads.py. With one readout token per step this is the same.
• Data mixture. Paper: 25 datasets with the fractions in tab. 3. Code: 25 datasets with per-dataset multipliers oxe_dataset_mixes.py, then scaled by dataset size when balance_weights is set and normalized dataset.py. Fractions depend on the dataset sizes, which the repo does not contain.
• Model sizes. Paper: Small and Base match the code's ViT-S and ViT-B settings tab. 5 transformer.py. The code also defines a vit_t size (hidden 192, 12 layers, MLP 768, 3 heads) transformer.py. I did not confirm it is the paper's 10M Tiny model.
• Inference example. The paper's appendix example calls sample_actions with no unnormalization statistics app. B. Then the output stays in normalized units. The repository's examples pass model.dataset_statistics 03_eval_finetuned.py.

Pitfalls

• "52%" is percentage points, not a ratio. The paper says Octo beats the next best baseline "by 52%" §4.2. Table 1 gives 72% against 20% for scratch, so that is +52 points. The relative gain is about 260%.
• VC-1 average. The six VC-1 values in tab. 1 average 15.8%, but the table reports 15%.
• Bimanual trial count. Table 1's caption says every setup uses 20 trials tab. 1. The appendix says Berkeley Bimanual used 10 trials app. F. The Berkeley Coke trial count is not stated.
• "29% higher" is not defined. The text does not say whether the gain is absolute or relative §4.1.
• Zero-shot is not generalization to new objects or skills. Zero-shot tasks come from the pretraining data, and the evaluation still changes object positions, lighting and backgrounds §4.1. Novel skills score 5% on average tab. 7, so do not expect zero-shot control of skills missing from the pretraining data.
• Data imbalance. Only 27% of pretraining data has wrist camera input, and 56% has language annotations. The paper blames these for the weak wrist-camera and goal-versus-language gaps §5.
• Wrist cameras hurt finetuning in the paper's tests. Third-person-only finetuning often did better §5.
• Finetune time. Section 3.4 gives about 5 hours on one A5000, while section 4 says under 5 hours §3.4 §4.2.
• Un-normalize actions. The action head outputs normalized values. Pass dataset statistics to sample_actions (as in 03_eval_finetuned.py), or the output is in normalized units octo_model.py.
• Clip bounds vs. normalization. The ±5 clip action_heads.py is applied in the space the head works in, so it interacts with your normalization. The paper does not discuss it.
• Preprocessing is not in the paper. The pretraining config reads a pre-resized copy from a GCS bucket (gs://rail-orca-central2/resize_256_256) octo_pretrain_config.py. Only the 256×256 resize is in the paper.
• Data loading. A small shuffle buffer (20k) hurt zero-shot results. The paper shuffles frames across trajectories before decoding and uses a 500k buffer app. E. Subsampling is capped at 100 random steps per trajectory app. E. Both are easy to skip and both matter.
• Gripper convention. The paper chose absolute gripper actions (+1 open, 0 closed) over a relative variant, because the relative one reduced retries app. E.
• Proprioception. The paper reports that adding proprioceptive inputs made pretrained policies worse app. E. The finetuning config still defines a proprioception key finetune_config.py.
• Ablations use Octo-Small. Compute limits kept the ablations on Small §4.3, so the Base results are not ablated.
• Figure and table numbers. Table numbers follow the source order. Figure numbers are not stated because the source adjusts the figure counter around the teaser [main.tex:174]. Figures are cited by their LaTeX labels. Appendix letters follow the order of the appendices environment. None of these were checked against the compiled PDF.
• Equation numbering. eq. 1 is the only numbered equation in the main text §3.3.

Lineage

• Builds on: Open X-Embodiment, the data source §1 and the RT-1-X and RT-2-X baselines §4.1.
• Compared against: RT-1-X (35M) and RT-2-X (55B) for zero-shot control §4.1, and training from scratch with a ResNet and FiLM architecture plus VC-1 for finetuning §4.
• Uses ideas from diffusion policies, action chunking and ViT scaling §2; those papers have no card here.
• Superseded by: not stated in this source.