---
kind: concept
slug: learned-simulator
title: Learned simulator
summary: >-
  A learned simulator predicts the next state, or a short plan, and is rolled forward by feeding its output back in. GraphCast forecasts global weather with a mesh graph network. TrafficSim simulates multi-agent traffic from driving logs and reports 0.50% scenario collisions. A survey covers planning with one-step models but reports no results.
tags: [simulation, autoregressive-rollout, weather-forecasting, traffic-simulation, graph-neural-network, model-based-rl]
papers: ["2101.06557", "2212.12794", "2411.14499"]
related: [graph-neural-network, time-series-forecasting, robot-simulation, model-predictive-control, model-based-reinforcement-learning, motion-prediction, differentiable-simulation, agent-based-simulation, imitation-learning]
status: draft
updated_at: 2026-11-01
---

## Overview

A learned simulator is a network that maps the recent state of a system, plus
any known external inputs, to the state one time step later. A forecast of any
length is produced by calling this model repeatedly and feeding each prediction
back as the newest input [arxiv:2212.12794 app. §3.1]. Some variants change the
output. TrafficSim predicts a plan for every actor over a planning window, then
applies only the first part of that plan before it predicts again
[arxiv:2101.06557 §3.3].

The shared problem is error build-up. A model that is checked on single steps
can still drift when it is run for many steps, because its own small mistakes
become its next inputs. The papers handle this by training through rollouts.
GraphCast trains on rollouts of up to 12 steps [arxiv:2212.12794 app. §4.2].
TrafficSim runs its policy in closed loop for the whole 12 s simulation during
training [arxiv:2101.06557 §3.4]. In the TrafficSim ablation, training through
the rollout cut the collision rate from 5.92% to 1.28% at the same planning
length [arxiv:2101.06557 tab. 2].

Three cards cover this concept. The first is a weather forecaster. The second
is a traffic simulator for self-driving tests. The third is a survey of world
models, which describes the same kind of one-step model for planning in
reinforcement learning, not for forecasting [arxiv:2411.14499 §3.1].

## Variants

### Mesh graph network with residual output (GraphCast)

The grid is encoded onto a multi-mesh, a refined icosahedron whose edges from
all refinement levels are kept. Coarse edges give long-range links, fine edges
give local ones. One message-passing step encodes grid to mesh, 16 unshared
processor layers run on the mesh, and one step decodes back to the grid. The
output is a normalised change added to the current state
[arxiv:2212.12794 app. §3.3] [arxiv:2212.12794 app. §3.4]
[arxiv:2212.12794 app. §3.5] [arxiv:2212.12794 app. §3.6]. Card:
[GraphCast: Learning skillful medium-range global weather forecasting](/papers/2212.12794).

Reported setup: 0.25° grid of 721 × 1440 points, 6-hour step, up to 40 steps
(10 days), 36.7 million parameters [arxiv:2212.12794 app. §2.2]
[arxiv:2212.12794 app. §2.3] [arxiv:2212.12794 §GraphCast].

### Joint multi-actor traffic simulator with unrolled training (TrafficSim)

Here the one-step model is replaced by a policy for all actors in a scene. At
each tick the model encodes a rasterised map once, and for each actor encodes
its last 3 s of boxes with a recurrent network. A graph-based interaction
module then links the actors. A single scene-level latent sample is decoded
into a joint plan for every actor, so the actors' plans fit together
[arxiv:2101.06557 §3.1] [arxiv:2101.06557 §3.2] [arxiv:2101.06557 app. A].
Each sample gives one whole-scene plan, and several samples from the same start
give several plausible scenes [arxiv:2101.06557 §3.2]. The simulator applies
the first κ steps of the plan and then plans again [arxiv:2101.06557 §3.3]. On
the GTX 1080 Ti the authors measured 0.83 s, 0.45 s and 0.24 s for 2 Hz, 1 Hz
and 0.5 Hz inference, with collision rates of 0.50%, 0.85% and 0.96%
[arxiv:2101.06557 tab. 4].

Training runs the policy in closed loop for 12 s. The first 8 s have expert
labels and use an imitation term, a variational bound on the future given the
ground truth. Beyond that point the imitation term is dropped. A collision
penalty is applied to samples from the prior [arxiv:2101.06557 §3.4]
[arxiv:2101.06557 eq. 3]. The collision term lowered the collision rate
from 0.60% to 0.50% in the ablation [arxiv:2101.06557 tab. 2]. At simulation
time, rejection sampling or gradient steps on the latent can reduce collisions
further without retraining. In the reported runs they lowered the collision
rate to 0.33% and 0.12%, and raised the rule-violation rate from 2.77% to
about 3.0% [arxiv:2101.06557 tab. 5] [arxiv:2101.06557 app. B]. Card:
[TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](/papers/2101.06557).

Author-reported results on the ATG4D dataset with 15 samples per scenario
[arxiv:2101.06557 tab. 1]: scenario collision rate 0.50%, against 1.19% for
the rule-based IDM and 2.90% for the next learned baseline, ILVM. The minimum
ADE over samples is 0.57 m, the lowest in the table. The rule-violation rate
is 2.77%, against 0.25% for IDM [arxiv:2101.06557 tab. 1]. A planner trained on
about 15k synthetic scenes from this simulator reaches a collision rate of
10.73% on real test scenarios, against 10.56% when trained on real data
[arxiv:2101.06557 tab. 3]. These results come from one dataset and are not
independently checked.

### One-step transition model for planning (model-based RL)

The survey treats a learned world model as the transition function of a
Markov decision process whose reward function is known. It fits that function
by supervised learning on real transitions, with a squared one-step error
[arxiv:2411.14499 §3.1] [arxiv:2411.14499 eq. 1] or a probabilistic one
[arxiv:2411.14499 eq. 2]. A planner then picks the action sequence with the
highest predicted return over a horizon τ [arxiv:2411.14499 eq. 3]. The
survey names Monte Carlo sampling of action sequences, probabilistic ensembles
with trajectory sampling, Monte Carlo tree search over learned values, and
latent-space trajectory optimisation as ways to search
[arxiv:2411.14499 §3.1]. The survey gives no architecture or training schedule
for the one-step model. It lists cloud-side roles for such models as data
engines, reinforcement learning environments and policy evaluators, without
reporting results for them [arxiv:2411.14499 §5.5]. Card:
[Understanding World or Predicting Future? A Comprehensive Survey of World Models](/papers/2411.14499).

## When to use which

The variants serve different goals. GraphCast forecasts a system from a long
trajectory archive. TrafficSim simulates other agents that react to each other
and must avoid collisions. The planning variant chooses actions. Only the first
two report results, and their numbers come from different tasks and protocols,
so this table is about constraints, not about which variant is better.

| Situation | Use | Why |
|---|---|---|
| Actions must be chosen against a model of the environment, e.g. a robot or game | One-step transition model with planning over candidate action sequences | The survey frames this as supervised fitting of the transition function, then planning with it [arxiv:2411.14499 §3.1] [arxiv:2411.14499 eq. 1] [arxiv:2411.14499 eq. 3]. No results are reported for it [arxiv:2411.14499 §1–§7]. |
| A learned model should serve as a data engine, RL environment or policy evaluator | Consider the roles the survey lists | The survey names these roles [arxiv:2411.14499 §5.5] but reports no evidence for them. |
| Many agents must interact in a scene, e.g. other cars for a self-driving test | A joint multi-actor policy trained on logs with unrolled training | TrafficSim reports the lowest collision rate and minimum ADE in its table [arxiv:2101.06557 tab. 1]. Its rule-violation rate is higher than IDM's [arxiv:2101.06557 tab. 1]. |
| Several plausible scenes are needed from one starting state | Sample a scene latent several times | TrafficSim evaluates 15 samples per scenario [arxiv:2101.06557 §4.1]. The authors note that a diversity score can reward unrealistic spread [arxiv:2101.06557 §4.1]. |
| Dangerous cases are rare in the logs | Add a collision penalty during training | The authors add it because the logs under-represent these cases [arxiv:2101.06557 §3.4]. Its ablation effect is small: 0.60% to 0.50% [arxiv:2101.06557 tab. 2]. |
| A long archive of trajectories exists, e.g. decades of reanalysis | A learned simulator trained on it | The paper trains on ERA5 from 1979 and reports skill against an operational forecast [arxiv:2212.12794 app. §4.1] [arxiv:2212.12794 §Forecast verification results]. |
| Training budget is large | Plan for weeks on a multi-accelerator cluster | Reported as 32 TPU v4 for about 4 weeks [arxiv:2212.12794 §GraphCast] [arxiv:2212.12794 app. §4.7]. |
| Inference must be cheap | A learned one-step model with rollout | Reported under 60 s for 10 days on one TPU v4 [arxiv:2212.12794 app. §3.2]. |
| Inference must be cheaper still, and some quality can be traded | Plan less often in the traffic simulator | Halving the plan rate roughly halves the runtime, from 0.83 s to 0.45 s, and raises the collision rate to 0.85% [arxiv:2101.06557 tab. 4]. |
| Collisions must drop without retraining | Post-process plans at simulation time | Rejection sampling and latent gradient steps reduce collisions but raise the rule-violation rate [arxiv:2101.06557 tab. 5] [arxiv:2101.06557 app. B]. |
| Sharp fields at long lead times matter | Check the output before use | The MSE objective blurs predictions at long leads; optimal blurring of the baseline narrows the gap but does not close it [arxiv:2212.12794 §Forecast verification results] [arxiv:2212.12794 app. §7.4]. |
| Probabilistic forecasts are needed | Not covered by this paper | The authors note the model is deterministic and that no ensemble is provided [arxiv:2212.12794 §Conclusions]. |
| No reference simulator or archive to train on | Not covered by the GraphCast or TrafficSim cards | Training uses ERA5 as ground truth [arxiv:2212.12794 app. §1.2]. TrafficSim trains on ATG4D, but the paper does not say whether that dataset is public [arxiv:2101.06557 §4]. The planning variant does not state a data source [arxiv:2411.14499 §3.1]. |

Claims are author-reported. The GraphCast headline is that the model beats
HRES-fc0 on 90.3% of 1380 targets at 10 days [arxiv:2212.12794 §Forecast
verification results]. The Pangu-Weather comparison uses a different
initialisation and target protocol, so its 99.2% is not comparable to the
headline [arxiv:2212.12794 app. §6]. The survey reports no experiments of its
own; its benchmark table names evaluation suites without scores
[arxiv:2411.14499 tab. 8].

## Implementation sketch

The rollout loop is the common core. The GraphCast training objective is a
weighted, rollout-based squared error. The survey's planning loop uses the
same one-step model but scores candidate action sequences by predicted return
instead of forecasting a fixed horizon [arxiv:2411.14499 eq. 3]. The card's
pseudocode is its own rendering of each paper's description, not released code.

```pseudocode
# Roll the one-step model forward  [arxiv:2212.12794 app. §3.1]
# Input:  prev_state (t-1 step), state (t), forcings_for(step)
# Output: list of forecast states

def forecast(prev_state, state, num_steps):
    forecasts = []
    for step in range(num_steps):
        next_state = one_step(prev_state, state, forcings_for(step))
        forecasts.append(next_state)
        # Feed the prediction back in as the newest input
        prev_state, state = state, next_state
    return forecasts
```

```pseudocode
# Autoregressive weighted-MSE training loss  [arxiv:2212.12794 app. §4.2]
# Input:  batch of trajectories, rollout length (ramped up to 12)
# Output: scalar loss; gradients flow through every rollout step

def training_loss(batch, rollout_steps):
    prev_state, state = batch.initial_states
    total = 0.0
    for step in range(rollout_steps):
        pred = one_step(prev_state, state, batch.forcings[step])
        target = batch.targets[step]
        total = total + weighted_squared_error(pred, target)
        prev_state, state = state, pred
    return total / rollout_steps
```

Variants differ in the one-step model (here a mesh graph network with
residual output) and in the rollout schedule. The paper ramps the rollout
from 2 to 12 steps over 11,000 updates at a fixed learning rate of 3e-7, after
two phases with one step [arxiv:2212.12794 app. §4.5].

The traffic simulator uses the same outer loop but with three differences.
Each tick draws one scene latent shared by all actors. Only the first κ steps
of the joint plan are kept. The loss mixes an imitation term, which is
dropped after the labelled part, with a collision term on prior samples
[arxiv:2101.06557 §3.3] [arxiv:2101.06557 §3.4]. The paper's printed formulas
for the mixing weight and the collision loss do not produce the behaviour the
text describes, so an implementer must choose the intended form
[arxiv:2101.06557 app. A]. Its optimiser, learning rate and batch size are not
stated [arxiv:2101.06557 §4].

## Contradictions

The three cards report no numbers for the same task, so there is no direct
numerical disagreement between them. Inside the TrafficSim paper, the text
calls the rule-violation rate similar to IDM's [arxiv:2101.06557 §4], while the
table gives 2.77% against 0.25% for IDM [arxiv:2101.06557 tab. 1]. Use the
table values. The survey notes that the term "world model" is itself contested
[arxiv:2411.14499 §1], so read the planning variant as a different scope, not
as a rival result. The protocol differences within the GraphCast comparisons
are listed under "When to use which".

## Papers

- [GraphCast: Learning skillful medium-range global weather forecasting](/papers/2212.12794): a mesh graph network learned one-step weather model, rolled out autoregressively and trained on ERA5.
- [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](/papers/2101.06557): a joint multi-actor traffic policy with a scene latent, trained by unrolling through the simulator with an imitation and a collision loss.
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](/papers/2411.14499): a survey that frames a learned one-step transition model for planning in model-based reinforcement learning. Reports no experiments.