Hierarchical Reasoning Model

In plain words

Most language models reason by writing out their steps in text, one token at a time. This paper builds a small network that reasons inside its hidden state instead. It has two connected recurrent blocks: one moves slowly and plans, the other works quickly on details. The fast block settles toward an answer for a while, then the slow block takes that result, updates the plan, and the fast block restarts from a new context. A learned signal decides when the model stops.

The model is trained from random weights on about a thousand puzzle examples, with no pre-training and no written reasoning. It is tested on Sudoku, mazes and an abstract-reasoning benchmark. The numbers are the authors' own.

TL;DR

HRM is a recurrent transformer-based model of about 27M parameters with a high-level and a low-level module. It is trained with a one-step gradient approximation, so it avoids backpropagation through time, and with deep supervision over several segments. A Q-head learns when to halt. The authors report strong results on ARC-AGI-1, Sudoku-Extreme and Maze-Hard, all trained on about 1000 examples without pre-training or chain-of-thought §1 fig. 1.

Problem

The authors argue that chain-of-thought (CoT) prompting is brittle, needs much data and is slow, and that a fixed-depth Transformer has limited effective depth for search-like reasoning §1. Plain recurrent networks converge too early and need backpropagation through time (BPTT), whose memory grows with the number of steps §1 §2 (Approximate gradient).

Method

The model has four learnable parts: an input network f_I, a low-level module f_L, a high-level module f_H and an output network f_O. One forward pass runs N high-level cycles of T low-level steps. The low-level state z_L is updated every step, conditioned on the high-level state, which changes only once per cycle §2 (Architectural details). Both modules are encoder-only transformer stacks with the same architecture, and they combine their inputs by element-wise addition §2 (Architectural details).

Hierarchical convergence. Within a cycle, the low-level module settles to a local equilibrium that depends on the current high-level state. The high-level update then gives the low-level module a new context and a new convergence phase. The authors claim this keeps the forward residual active over many steps, where a standard RNN's activity decays §2 (Hierarchical convergence) fig. 3 forward-residual figure, cited by section.

One-step gradient. Gradients are taken only through the last state of each module, treating earlier states as constants. This is motivated by the implicit function theorem on the fixed point of a deep equilibrium model, with the Neumann term I + J + J² + … truncated to I §2 (Approximate gradient) eq. 1 eq. 2 eq. 3. Memory is O(1) in the number of steps, versus O(T) for BPTT §2 (Approximate gradient).

Deep supervision. One training step is one segment. Each segment runs a forward pass from the previous segment's detached state, computes a loss, and takes an optimizer step. Samples that halt are replaced by fresh samples from the data loader §2 (Deep supervision) (ACT).

Adaptive halting. A Q-head reads the high-level state and outputs two logits (halt, continue). The number of segments is chosen per sample with a Q-learning rule and a randomised minimum segment count §2 (Adaptive computational time). The total loss adds a binary cross-entropy term for the Q-head to the sequence loss §2 (ACT, L_ACT).

Model variables used by the pseudocode below. Tokens are embedded and flattened. Outputs are token distributions. The sequence loss averages over tokens. Initial states z⁰ are fixed truncated-normal samples §2 (Architectural details).

The pseudocode follows the paper's own figure of the approximate-gradient forward pass §2 (pseudocode figure), and the halting rule in §2 (ACT). Where the official code differs, see Paper vs code.

# One HRM segment: N high-level cycles of T low-level steps  [§2] [eq. 1]
# Input x: (B, l) tokens; z_H, z_L: carried states (B, l, d)
# Output: new states and y_hat (B, l, V)

def hrm_segment(x, z_H, z_L, N, T):
    # Input network f_I: embed and project the tokens  [§2]
    x_tilde = input_network(x)                     # (B, l, d)

    # Phase 1, no gradient: low level settles, high level updates once
    # per cycle; the inputs of each module are added element-wise  [§2]
    with no_grad():
        for i in range(N * T - 1):
            z_L = f_L(z_L, z_H, x_tilde)
            if (i + 1) % T == 0:
                z_H = f_H(z_H, z_L)

    # Phase 2, one-step gradient: only the last L and H updates keep
    # gradient  [§2] (Approximate gradient)
    z_L = f_L(z_L, z_H, x_tilde)
    z_H = f_H(z_H, z_L)

    # Output head reads the high-level state  [§2] (Architectural details)
    y_hat = f_O(z_H)

    # The carried states are detached: no gradient crosses segments  [§2]
    return detach(z_H), detach(z_L), y_hat

Deep supervision and halting. Each segment computes a loss and steps the optimizer. A halted sample is replaced with a new sample §2 (Deep supervision).

# Deep supervision: one optimizer step per segment  [§2] (Deep supervision)
# Stops early for a sample when its halt decision fires  [§2] (ACT)

def train_segment(batch, carry, optimizer):
    # Forward one segment from the detached carried state  [§2] step 1
    z_H, z_L, y_hat = hrm_segment(batch.x, carry.z_H, carry.z_L, N, T)
    # Sequence loss plus the halting Q-loss  [§2] (ACT, L_ACT)
    loss = seq_loss(y_hat, batch.y) + q_loss(z_H, batch)
    # Gradient step on the segment loss  [§2] steps 2 and 3
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()
    # Halted samples get a fresh input and a reset state  [§2] (ACT)
    return replace_halted(z_H, z_L, batch)

# Halting decision after each segment  [§2] (ACT)
# m: segments run so far; m_min: sampled per sample; m_max: fixed limit

def should_halt(m, q_halt, q_continue, m_min, m_max):
    # Forced stop at the segment limit; the paper says "surpasses",
    # the code uses >=  [§2] (ACT)  [code: models/hrm/hrm_act_v1.py#L260]
    if m >= m_max:
        return True
    # Learned stop once the minimum count is reached  [§2] (ACT)
    return (q_halt > q_continue) and (m >= m_min)

# Sample the minimum segment count for one sample  [§2] (ACT)
# With probability eps: uniform over {2, ..., m_max}; else 1

def sample_m_min(eps, m_max):
    if random() < eps:
        return randint(2, m_max)
    return 1

# Bootstrapped Q targets for one segment  [§2] (ACT)
# correct: 1 if the prediction equals the target, else 0

def q_targets(m, correct, q_halt_next, q_continue_next, m_max):
    # Halt target: whether this segment's answer is right  [§2] (ACT)
    g_halt = correct
    # Continue target: at the limit only halting remains  [§2] (ACT)
    if m >= m_max:
        g_continue = q_halt_next
    else:
        g_continue = max(q_halt_next, q_continue_next)
    return g_halt, g_continue

Inference. Halting is applied during training only. At evaluation, each sample runs to the maximum segment count, which keeps batches aligned hrm_act_v1.py. The paper reports that raising the limit at inference improves accuracy on Sudoku without retraining §2 (Inference-time scaling).

Hyperparameters

Values marked "paper" are stated in the paper text or figures. Values marked "code" come from the official repository and are not in the paper. "Not stated" means neither the paper nor the code gives it. The paper's main text contains no hyperparameter table, and the LaTeX source contains no appendix content (the \appendix command has nothing after it).

Results

Author-reported. The training-set sizes are those in the Figure 1 panel titles fig. 1. Chain-of-thought (CoT) baselines are pretrained models; ARC CoT scores come from the official leaderboard, and Sudoku and Maze scores come from API evaluation §3.2. The paper does not state the prompts, API versions or sampling settings for the baselines.

Values are read from the labels printed on the paper's Figure 1 and match the text for the HRM ARC-AGI-1 result (40.3%) and the o3-mini-high and Claude values §1.

Other reported numbers, all author-reported:

• Direct pred, trained on the full Sudoku-Extreme set (3,831,994 examples), reaches 16.9% §3.2.
• Sudoku-Extreme has a mean difficulty of 22 backtracks per puzzle; Sudoku-Bench averages 0.45 §3.1.
• On Sudoku-Extreme-Full, the figure reports near-perfect HRM accuracy, but the paper gives no number in the text fig. 2.
• Analysis of the trained model on Sudoku-Extreme-Full gives participation ratio 30.22 for z_L and 89.95 for z_H, using 100 trajectories. The paper calls this correlational §4.

Paper vs code

The official repository is github.com/sapientinc/HRM, linked from the paper's footnote. It belongs to the authors' organisation (Sapient Intelligence). The clone is at commit ac15626f8db096a63c775b84c9dc868776a6feda, licence Apache-2.0. The repository contains only the ARC training config (config/cfg_pretrain.yaml is labelled "ARC training config"). Other tasks are run with command-line overrides in the README.

Where the code matches the paper:

• Hidden-state update schedule and the one-step gradient. The no-grad phase runs N·T − 1 low-level steps and N−1 high-level updates, then one gradient step per module hrm_act_v1.py §2 (pseudocode figure).
• Detached carried state between segments hrm_act_v1.py.
• Exploration and minimum segment rule, with ε = 0.1 and a sampled minimum in {2, …, M_max} hrm_act_v1.py §2 (ACT).
• Bootstrapped continue target, with the maximum-only branch at the limit hrm_act_v1.py §2 (ACT).
• Sudoku augmentation by digit map and band and row shuffles build_sudoku_dataset.py §3.2.
• ARC voting over two most frequent answers arc_eval.ipynb §3.2.

Where the code differs from the paper:

• Q-loss weight. Paper: L_ACT = loss + BCE(Q, G) with no weight §2 (ACT). Code: lm_loss + 0.5 * (q_halt_loss + q_continue_loss) losses.py.
• Q-head input. Paper: Q from the final H-module state §2 (ACT). Code: from position 0 of z_H, which is the puzzle-embedding token when the puzzle embedding is used hrm_act_v1.py hrm_v1.yaml.
• Halt target. Paper: 1{ŷ = y} §2 (ACT). Code: whole-sequence exact match, seq_is_correct losses.py losses.py.
• Token averaging. Paper: averaged over all tokens §2 (Architectural details). Code: mean over valid tokens per sequence, summed over the batch and divided by the global batch size losses.py losses.py pretrain.py pretrain.py.
• Optimizer. Paper names Adam-atan2 in the architecture paragraph and AdamW in the stability paragraph §2. Code uses AdamATan2 for dense parameters pretrain.py.
• Puzzle embedding. Paper mentions a learnable token only for ARC §3.2. Code learns a 512-dim embedding per puzzle identifier with sign SGD, decoupled weight decay and its own LR sparse_embedding.py cfg_pretrain.yaml cfg_pretrain.yaml.
• Activation. Paper says GLU §2 (Architectural details). Code uses SwiGLU with SiLU gating layers.py.
• Halt boundary. Paper: "surpasses M_max" for forced halting §2 (ACT), but "m ≥ M_max" in the continue target. Code uses >= for both hrm_act_v1.py.
• Embedding scale and init. Not in the paper. Embedding scaled by sqrt(hidden size), init std 1/sqrt(hidden size) hrm_act_v1.py hrm_act_v1.py.
• Maze filter. Paper: keep instances with shortest-path difficulty above 110 §3.1. The repository does not reproduce this. The builder downloads a prebuilt set from Hugging Face (sapientinc/maze-30x30-hard-1k) build_maze_dataset.py.
• LR schedule. Paper: constant with warm-up §2. Code's default is constant (min ratio 1.0), but the full Sudoku-Hard README run uses cosine decay to 0.1 cfg_pretrain.yaml README.md.

Pitfalls

• Which ARC version gives 40.3%. The introduction calls 40.3% the "ARC AGI Challenge" and cites ARC-AGI-2 work, but Figure 1 labels 40.3 under ARC-AGI-1 and 5.0 under ARC-AGI-2 §1 fig. 1. Treat 40.3 as ARC-AGI-1, the reading the figure supports. The code default data_path is the ARC-1 set (data/arc-aug-1000) cfg_pretrain.yaml; ARC-2 needs an override README.md.
• "More than twofold." The paper says the Direct-pred-to-HRM change on ARC-AGI-1 is "more than a twofold performance improvement" §3.2. Figure 1 gives 21.0 and 40.3, a ratio of about 1.92. The paper does not explain the gap.
• "Near-perfect" claims. The abstract and introduction say near-perfect accuracy on Sudoku and mazes §1. In the 1000-example setting, Figure 1 gives 55.0% for Sudoku-Extreme and 74.5% for Maze-Hard fig. 1. The "near-perfect" statement refers to the Sudoku-Extreme-Full analysis, with no number in the text fig. 2. No maze near-perfect result appears in the source.
• Halting boundary. The paper's forced stop says "surpasses M_max", while the continue target uses "m ≥ M_max". The code uses >= throughout hrm_act_v1.py. Use >= to match the code.
• Evaluation halts at the limit. Halting is applied only during training. Evaluation always runs to the maximum segment count hrm_act_v1.py. Any inference-time comparison depends on this.
• Hidden cost of ACT training. The bootstrapped target needs a second inner forward pass per training segment hrm_act_v1.py. The O(1)-memory claim §2 concerns BPTT and does not count this extra pass; the paper gives no wall-clock figure.
• Optimizer bound argument. The stability argument for Q-learning assumes AdamW, which bounds parameters by 1/λ §2 (Stability). The model uses Adam-atan2 pretrain.py. The paper does not show that the bound holds for Adam-atan2.
• Q-learning without target networks. The paper says replay buffers and target networks are absent, and relies on large parallel batches and bounded, normalised parameters §2 (Stability). The code's comment makes the same point hrm_act_v1.py.
• Q-head indexing and loss weight differ from the paper (see Paper vs code). An implementation that follows the paper literally will not reproduce the code's halting behaviour.
• Stablemax versus softmax. The paper uses stablemax for small-sample runs §2. The code defaults to stablemax; the full-scale Sudoku-Hard run uses softmax hrm_v1.yaml README.md.
• Numerics. Forward passes run in bfloat16 by default hrm_act_v1.py, while stablemax is computed in float64 losses.py. The README warns of numerical instability from late-stage overfitting on the 1000-example Sudoku set and advises early stopping near 100% training accuracy README.md.
• Run-to-run variance. The README reports variance of about ±2 points for small-sample runs README.md. The paper gives no seed or variance figures in the text.
• Maze data. The difficulty filter (shortest path > 110) is in the paper only; the repository downloads a prebuilt Hugging Face set build_maze_dataset.py. Reproducing the exact Maze-Hard split from the paper alone is not possible from this source.
• Missing hyperparameters. Learning rate, batch size, epochs, weight decay, betas and the segment limit are not in the paper text, and the LaTeX source has no appendix. The values in the table come from the code config and the README; they are the ARC run unless stated otherwise.
• Turing-completeness. Section 6 says HRM is computationally universal given enough memory and time §6. The source gives no proof, and the claim is not tested.
• One-step gradient justification. The derivation assumes convergence to a fixed point with I − J invertible and a continuously differentiable map §2 (Approximate gradient). The source contains no ablation against full BPTT.
• Baseline protocol. CoT baseline numbers come from the ARC leaderboard and from API runs for Sudoku and Maze, with no prompts or versions stated §3.2. The 0.0 entries for Sudoku and Maze reflect the authors' protocol.
• Brain analogy. The dimensionality analysis is correlational, as the authors state §4. Do not use it as evidence of mechanism.

Lineage

• Builds on: deep equilibrium models and the implicit-function view of fixed points §2 (bai2019deep, as cited); the one-step gradient approximation §2 (Geng2021OnTI, as cited); deep supervision in DEQ-style models §2 (DEQ-Flow, as cited); Q-learning stability via post-norm and weight bounds §2 (PQN2025, as cited).
• Compared against: CoT models (o3-mini-high, Claude 3.7 8K, Deepseek R1) and a "Direct pred" transformer that keeps HRM's training setup §3.2 fig. 1.