Action tokenization

Overview

Action tokenization maps real-valued signals, such as joint positions or end-effector poses, to a sequence of discrete symbols. A sequence model then predicts those symbols with the usual next-token objective, and a decoder maps the predicted symbols back to real values. The sources cover different points in this design. One writes numbers directly into the prompt of a frozen LLM, with no learned tokenizer 2307.04721 §5 2307.04721 §6. Another bins each action dimension into a fixed set of values and trains a policy on the resulting tokens 2310.08864 §4.2. A third trains a tokenizer for the action chunks of a vision-language-action policy 2501.09747 §5.2. The fourth learns a codebook over humanoid motion clips and generates motion as codebook indices from a text command 2412.14172 §4.

The shared requirement is that the tokens can be turned back into the signal. The FAST paper describes each of its steps as invertible [arxiv:2501.09747 §5.2]. These sources also show that the discretisation choice changes results: digit spacing changes how the prompt tokenises 2307.04721 §4, and the scale in FAST trades reconstruction accuracy against token count 2501.09747 §6.2 2501.09747 app. A.2.

Variants

Plain numbers in the prompt (no learned tokenizer)

The model is given a few input-output sequences as text and asked to continue the pattern. Every number is written as a short string of tokens. The paper uses an alphabet of digits, space-prefixed so that neighbouring digits do not merge, and also tests random alphabets of made-up tokens 2307.04721 §4 2307.04721 tab. 1. Robot values are rounded to integers from 0 to 100, or 0 to 300 for the whiteboard task, and each integer is one token 2307.04721 §5. The paper reports that a frozen text-davinci-003 solves 85 of 800 ARC problems 2307.04721 tab. 1. The card does not say whether these are exact-match scores. Card: Large Language Models as General Pattern Machines.

Per-timestep binning

Each action dimension at each timestep is mapped to one of a fixed set of bins, and the bins are concatenated. The autoregressive baseline in the FAST paper uses 256 bins per dimension 2501.09747 §3. Because each timestep adds one token per dimension, the count grows with control frequency. For one-second chunks the paper reports 35, 105, 140 and 700 tokens for four datasets at 5 Hz to 50 Hz 2501.09747 tab. 1. On two real-robot tasks, at 20 Hz and 50 Hz, this scheme made no task progress in the authors' single-dataset experiments 2501.09747 §6.2 fig. 6.

Open X-Embodiment uses the same idea for one policy trained on pooled robot data. Its RT-1-X model splits each action into 8 dimensions and bins each one into 256 uniform bins 2310.08864 §4.2. The text normalises each dataset's actions before binning 2310.08864 §4.1, but the released code clips to fixed bounds, and the normalisation statistics are not in that repository. The bin range is therefore set by an input the paper does not give. The reported results compare whole policies, not the tokenizer. RT-1-X beats each dataset's original method on 4 of 5 small-data domains, but it underfits the large-data domains and does not beat RT-1 there 2310.08864 §5.1 2310.08864 tab. 1. Card: Open X-Embodiment: Robotic Learning Datasets and RT-X Models.

Frequency-domain compression (FAST)

The action chunk is normalised per dimension with the 1st and 99th percentile mapped to -1 and 1 2501.09747 §5.2. A discrete cosine transform is applied along time to each dimension. The coefficients are multiplied by a scale of 10 and rounded, so most high-frequency entries become zero. The result is flattened with the lowest frequency first and compressed with a byte-pair encoding of 1024 entries 2501.09747 §5.2 [arxiv:2501.09747 alg. 1]. The same pipeline can be fitted once on about one million chunks from many robots, and the authors report that this universal tokenizer matches dataset-specific ones on their policy evaluations 2501.09747 §5.3 2501.09747 §6.3. Card: FAST: Efficient Action Tokenization for Vision-Language-Action Models.

Learned vector-quantized motion tokens (UH-1)

A 1-D convolutional encoder maps a clip of humanoid motion to latent vectors, and each latent is snapped to its nearest entry in a learned codebook. Each token covers four frames, so the sequence is about a quarter as long as the clip 2412.14172 §4 2412.14172 app. D.1. The paper gives a codebook of 2048 entries 2412.14172 app. D.3. A causal Transformer reads a CLIP embedding of the text command and predicts the next token, with an end token to stop 2412.14172 §4 2412.14172 app. D.2. A decoder maps the generated tokens back to a motion clip. Binning uses fixed bins and FAST a fixed transform, whereas here the codebook itself is learned, so the tokens form a vocabulary of motion primitives 2412.14172 §4. The paper reports that a larger codebook helps: FID falls from 0.539 with 512 codes to 0.463 with 2048 2412.14172 §5.4 2412.14172 fig. 9. On a HumanoidML3D variant the author-reported FID is 0.445 for UH-1 against 0.582 for MDM 2412.14172 tab. 1. The tokenizer is trained on Humanoid-X, a set of 163,800 clips that were retargeted to a robot and tracked by an RL policy before use 2412.14172 §3. Card: Learning from Massive Human Videos for Universal Humanoid Pose Control.

When to use which

All FAST results are author-reported and mostly shown in plots, so they should be read as trends rather than success rates 2501.09747 §6. The UH-1 results are FID and success rates on its own benchmark and robot, so they do not transfer to the other rows.

Implementation sketch

The first two encoders below share the same outer shape: normalise, transform, round to integers, then serialise. They differ in the transform and in the last step. The third replaces the fixed transform with a learned encoder. The decoders run the steps in reverse. The sources do not give the exact code for the parts marked as inputs.

# Variant A: per-timestep binning  [arxiv:2501.09747 §3]; same scheme in [arxiv:2310.08864 §4.2]
def bin_encode(actions, lo, hi, n_bins):
    # Clip each value to the range; lo and hi are inputs, not stated in the papers
    clipped = clamp(actions, lo, hi)
    # Map each value to an integer bin, one token per dimension per timestep
    # Truncates, so decoding to the lower bin edge biases values low (Open X card)
    bins = floor((clipped - lo) / (hi - lo) * (n_bins - 1))
    # Concatenate per timestep  [arxiv:2501.09747 §3]
    return bins.flatten()

# Variant B: DCT, rounding and BPE (FAST)  [arxiv:2501.09747 §5.2]
def fast_encode(actions, scale, bpe):
    # Normalise each dimension, 1st and 99th percentile to -1 and 1  [§5.2]
    # actions is assumed normalised already: (H steps, D dimensions)
    # DCT along time, one transform per dimension  [arxiv:2501.09747 §5.2]
    coeffs = dct(actions, axis=0)
    # Scale and round; most high-frequency entries become zero  [alg. 1]
    quantized = round(scale * coeffs)
    # Lowest frequency first, all dimensions per frequency  [§5.2]
    flat = quantized.flatten()
    # Byte-pair encoding, vocabulary 1024, fitted on training data  [§5.2]
    return bpe.encode(flat)

# Variant C: learned VQ tokens (UH-1)  [arxiv:2412.14172 §4] [eq. 10]
def vq_encode(clip, encoder, codebook):
    # Encode the clip with a 1-D convolutional encoder  [app. D.3]
    # One latent per four frames  [app. D.1]
    latents = encoder(clip)                              # (T/4, C)
    # Snap each latent to its nearest codebook entry; the index is the token  [eq. 10]
    distance = norm(latents[:, None] - codebook[None], axis=-1)   # (T/4, K)
    return argmin(distance, axis=-1)                     # (T/4,) codebook indices

Practical note: the official open-source wrapper for FAST cuts the sequence at a maximum token length, which drops trailing action tokens and decodes to a wrong chunk. The FAST paper does not state a maximum length, so this comes from the code, not the paper; see the FAST card. Decoders must handle invalid or truncated sequences.

The UH-1 release also caps generated sequences, at 51 tokens by default, which is about 204 frames at four frames per token. The paper does not state a maximum length, so this too comes from the code; see the UH-1 card. Decoding there takes the top-1 token at each step, with no sampling option.

Open X-Embodiment's released code emits 11 tokens per step where its paper describes 8, and its inference script builds the model with 512 bins where the paper says 256 2310.08864 §4.2. Check the checkpoint's output size before you train against it; see the Open X card.

Contradictions

• Open X-Embodiment's paper and code differ. The text gives 8 action dimensions and 256 bins 2310.08864 §4.2. The released inference code uses 11 tokens per step and 512 bins. The text normalises each dataset before binning 2310.08864 §4.1, while the code clips to fixed bounds.
• Normalisation range inside the in-context paper: the main text maps Marker in Cup positions to 0 to 100 2307.04721 §6, while the appendix says 0 to 200 2307.04721 app. C.1. The compiled PDF was not checked.
• UH-1's paper and code differ on the tokenizer. The paper gives a 2048-entry codebook and an embedding loss plus a commitment loss [arxiv:2412.14172 app. D.1] 2412.14172 app. D.3. The inference code defaults to 512 codes and uses an exponential-moving-average codebook with a commitment loss only. The released checkpoints are therefore not the paper's configuration; see the UH-1 card, Paper vs code.
• Open X-Embodiment's and UH-1's tokenizers have different aims. The first bins actions to a fixed grid and the second learns its vocabulary. The two cards report no comparison between them.
• The four papers report results on different tasks: ARC and sinusoid completion with a frozen LLM in one 2307.04721 §4 [arxiv:2307.04721 §5]; real-robot policy training with a vision-language backbone in FAST 2501.09747 §6.1; whole-policy evaluation over pooled robot datasets in Open X-Embodiment 2310.08864 §5; and text-to-motion FID on a humanoid benchmark in UH-1 2412.14172 tab. 1. Their numbers are not comparable. None of the four papers compares its tokenizer with the others, and the UH-1 card records no comparison with binning or FAST.

Papers

• Large Language Models as General Pattern Machines: writes numbers and random-token sequences into the prompt of a frozen LLM and tests completion on ARC, sinusoids and robot trajectories.
• Open X-Embodiment: Robotic Learning Datasets and RT-X Models: pools 60 robot datasets into one format and trains RT-1-X, which bins each action into 8 dimensions of 256 bins, and RT-2-X on the mixture.
• FAST: Efficient Action Tokenization for Vision-Language-Action Models: DCT-plus-BPE tokenizer for action chunks, with a universal variant and comparison against per-timestep binning and diffusion policies.
• Learning from Massive Human Videos for Universal Humanoid Pose Control: VQ-VAE tokenizer over humanoid motion clips and a text-conditioned Transformer that generates the tokens, trained on the Humanoid-X dataset.