TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors

In plain words

Self-driving software is tested in simulators, and a simulator is only useful if the other cars in it behave like real drivers. Many simulators use fixed rules for this, so the simulated cars drive politely and predictably. Real drivers nudge past a parked car, make U-turns and negotiate an unprotected left turn, and the rule-based cars do none of that.

This paper learns the behaviour from recorded human driving instead. The model looks at the map and at the recent past of every car in the scene, then produces a short plan for all of them together, so that the plans fit with one another. Sampling the model several times gives several different but plausible traffic scenes from the same starting point, and the model can be run forward step by step to create a longer simulation.

Learning from recorded drives has a known weakness: a model trained only to copy what it saw drifts away from the data once its own small mistakes pile up. The authors therefore let the model drive in its own simulation during training and compare the result with the recording. They also add a penalty for collisions, because the recorded data under-represents dangerous situations, and they reduce the weight of the copying signal toward the end of the recorded part. The authors report that the resulting traffic is more realistic than that of the baselines they tried, and that it can also be used to generate extra training data for a driving planner.

TL;DR

• Learns a joint, multi-actor traffic policy from human driving logs. An implicit latent variable model draws several socially consistent plans for all actors at once §3.2.
• Trained by unrolling the policy in closed loop and backpropagating through the differentiable simulation, with an imitation loss plus a differentiable collision loss §3.4.
• On ATG4D, scenario collision rate 0.50% (next best 2.90%) and minSADE 0.57 m; traffic rule violation is worse than IDM (2.77% vs 0.25%) tab. 1.
• No official code was found. The main loss weighting, collision loss and diversity metric formulas are printed in ways that do not work as written (see Pitfalls).

Problem

Rule-based car-following models such as IDM keep actors collision-free and on the road. They cannot produce irregular manoeuvres such as U-turns or nudging, and they cannot infer the flow of traffic from partly observed signals §1 §2 app. B. Learned motion forecasters can produce varied, interacting behaviour, but when they are rolled out for a long horizon they drift from the data and cannot recover from their own errors (compounding error) §2. Plain behaviour cloning has the same problem, and the other common fixes each carry a cost: online expert queries, hand-designed perturbations, uncertainty penalties, or adversarial training that is hard to tune §2. The paper assumes that labelled trajectories exist for the simulated period, which is 8 s of the 12 s simulated, and that a rasterised HD map with lane topology and traffic-light states is available §4 app. A.

Method

Overview. At each simulation tick the model takes the map and the actors' recent history and outputs a joint plan for every actor over a fixed planning window. Only the first part of the plan is applied, and the process repeats §3.3 alg. 2. The model has four parts: a map backbone that runs once per map, a local observation module that runs each tick, a scene-level latent variable with a graph-based interaction module, and a decoder that turns the latent and the context into actor plans §3.1 §3.2 app. A.

Step 1. Global map features §3.1 app. A. The map is rasterised with 13 channels for roads, lanes and intersections, plus extra channels that mark lanes controlled by a traffic light app. A. A multi-scale convolutional backbone, adapted from PIXOR, runs once per map and its result is cached. Its four blocks use kernel size 3 and 8, 16, 32 and 64 channels. Each block is downsampled by max pooling with stride 2. The outputs are resized to a common 0.8 m resolution, concatenated, and passed through a header of two convolutions with 64 channels app. A.

Step 2. Local context per actor §3.1 app. A alg. 2.

• Map part: an actor-centred region of the cached map features is cut out with rotated RoI Align. The region extends 70 m ahead, 10 m behind and 20 m to each side. A 3-layer CNN processes it and max pooling over space gives the map vector app. A.
• Motion part: the past 3 s of the actor, seven bounding boxes 0.5 s apart, are encoded with a 4-layer GRU with 128 hidden units. Missing boxes are set to zero and a binary mask marking them is passed in alongside app. A.
• The two parts are concatenated into a 192-dimensional context vector per actor app. A.
• Headings are not regressed. They are taken as the tangent of the predicted waypoints app. A.

Step 3. Scene latent and interaction §3.2 eq. 1 alg. 1 app. A. The joint distribution of all actors' future states is modelled as an integral over one scene-level latent variable Z: P(Y | X) = ∫ P(Y | X, Z) P(Z | X) dZ eq. 1. The decoder is deterministic, Y = f(X, Z). All stochasticity is put into Z, so one latent sample gives one whole-scene plan. Z is split into one piece per actor, z_n, which is tied to actor n. The prior, the posterior and the decoder are all built on a scene interaction module (SIM) alg. 1:

# Scene Interaction Module (SIM)  [alg. 1] [app. A]
# Input:  node hidden states (N, d); box coordinates (N, 2); K (default 1)
# Output: one output vector per node

def scene_interaction(hidden, coordinates, num_rounds):
    # Build a fully connected graph over actors  [§3.2] [app. A]
    edges = all_ordered_pairs(num_actors)
    # Express each pair's coordinates in one actor's frame  [alg. 1]
    relative = pairwise_transform(coordinates, edges)
    for round in range(num_rounds):
        # Message on each edge from its two endpoints  [app. A]
        messages = edge_mlp(hidden[src], hidden[dst], relative)
        # Max pool messages over each node's neighbours  [app. A]
        aggregated = max_over_neighbours(messages, edges)
        # Update hidden state with a GRU cell  [app. A]
        hidden = gru_cell(hidden, aggregated)
    # Read out one vector per node with an MLP  [app. A]
    return readout_mlp(hidden)

Step 4. One simulation tick §3.3 eq. 2 alg. 2 app. A. The model computes the context X for every actor. A prior network, which is a SIM over X, gives the mean and scale of a diagonal Gaussian over Z. A sample of Z is drawn, and for each actor an MLP combines its context with its latent piece. The decoder then produces a plan of T_plan steps for all actors. The simulator applies the first κ steps of the plan and then repeats. The probability of a simulated trajectory factorises over ticks as in eq. 2, where each factor is the plan's first κ steps given the context. A larger κ means fewer model calls and lower simulation quality §4.2 "Multi-Step Update". Any κ up to T_plan can be chosen at simulation time without retraining §4.2.

# Simulate traffic for T ticks  [alg. 2] [§3.3]
# Input:  map raster; history of actor boxes (3 s); kappa; T_plan
# Output: actor boxes for each simulated tick

def simulate(map_raster, history, num_ticks, kappa):
    # Run the map backbone once per map and cache the result  [§3.1]
    map_features = map_backbone(map_raster)
    states = history
    for tick in range(num_ticks):
        # Build the context for every actor  [§3.1] [app. A]
        context = encode_context(map_features, states)
        # Prior over the scene latent  [§3.2] [app. A]
        z_mean, z_scale = prior_network(context)
        # Draw one scene latent sample  [alg. 2]
        z = sample_gaussian(z_mean, z_scale)
        # Decode the joint plan for all actors  [§3.2] [alg. 2]
        plans = decoder(context, z)
        # Apply only the first kappa steps of the plan  [§3.3]
        states = append(states, plans[:kappa])
    return states

Step 5. Training by unrolling §3.4. The policy is run in closed loop for the full horizon (T = 12 s), and the loss is computed at each simulated step. The gradient flows through the sampled actions by the reparameterisation trick, so a decision's effect on later states is learnt §3.4. In the labelled part (t up to T_label = 8 s) the decoder is fed posterior samples, which are conditioned on the ground-truth future. After that point it is fed prior samples §3.4.

Step 6. Imitation loss §3.4. This is the CVAE evidence lower bound at each labelled step: a Huber reconstruction term summed over all actors and over the next P future steps, plus a KL term between the posterior and the prior, reweighted by β as in β-VAE §3.4. The Huber δ, the value of β and the value of P are not stated §3.4.

Step 7. Common-sense (collision) loss §3.4 app. A. Each vehicle is approximated by five circles. For each pair of actors, the distance between the closest pair of circles is penalised, with a loss of 1 at full overlap and 0 when there is no overlap. The per-pair loss is summed over the first P future steps, and the sum over ordered pairs is scaled by 1/N² §3.4. This loss is applied to prior samples only. It is multiplied by 0.01 to match the scale of the imitation loss app. A.

Step 8. Time-adaptive mixing §3.4 eq. 3 app. A. The total loss is L = Σ_t [λ(t) · L_imitation(t) + (1 − λ(t)) · L_collision(t)] eq. 3. The text says the weight on imitation must fall to zero at T_label app. A, so that after the labelled part only the collision term remains. The printed formula for λ(t) is min((T_label − t)/T_label, 0) app. A and does not do that; see Pitfalls. The authors say the method is not sensitive to the shape of the decreasing function app. A.

# Loss for one unrolled training run  [§3.4] [eq. 3] [app. A]
# Input:  ground-truth future for t <= T_label; simulated rollout
# Output: scalar loss

def training_loss(rollout, ground_truth, label_horizon):
    total = 0.0
    for t, step in enumerate(rollout.steps):
        # Imitation term only exists where labels exist  [§3.4]
        if t <= label_horizon:
            imitation = elbo_loss(step.posterior_plans, ground_truth[t])
        else:
            imitation = 0.0
        # Collision penalty is computed on prior samples  [§3.4]
        collision = collision_loss(step.prior_plans)
        # Time-adaptive weight; formula as printed in [app. A], see Pitfalls
        weight = lambda_of(t, label_horizon)
        total = total + weight * imitation + (1 - weight) * collision
    return total

Step 9. Simulation-time constraints §4 "Incorporating Constraints" app. B. Two optional post-processing steps reduce collisions without retraining. (a) Rejection sampling: resample plans that collide, up to 10 times. If still too few collision-free plans, return the samples with the lowest collision loss. The collision loss is evaluated over the first 5 timesteps (2.5 s). (b) Gradient optimisation: keep the model frozen and take 5 gradient steps, with learning rate 1e-2, on the latent Z to lower the same collision loss over the first 5 timesteps. Optimising in latent space keeps the plans in the model's distribution app. B.

Hyperparameters

Values are as stated in the paper; "not stated" means the paper and its supplement do not give the value.

Baseline settings app. B.

Results

Author-reported. Setup: ATG4D, S = 15 samples per scenario, T = 12 s, T_label = 8 s tab. 1 §4. Baselines are described in app. B. Table numbers follow the order of the LaTeX source (see Pitfalls).

Main comparison tab. 1. Lower is better on every column except MASD, where higher means more diverse.

The ground-truth human TRV is 1.26% §4. TrafficSim's TRV is not the lowest; the text calls it "similar to IDM" §4, and IDM's value is 0.25%.

Ablation tab. 2. M0, M1: open-loop training. M2: unrolled training, T_plan = 1. M3: unrolled, T_plan = 10. M*: full model, which adds the common-sense loss.

M2 has lower minSFDE and minSADE than M*, but higher SCR and TRV. The full model M* is the one reported in the main table tab. 1.

Data augmentation tab. 3 §4.2 app. B. An imitation planner trained on about 15k synthetic examples (all actors' trajectories from simulations, 5 s scenes) is compared with the same number of real examples. Evaluation is on the real ground-truth scenarios. The planner takes a rasterised map and the actor box history and regresses 10 waypoints over 5 s app. B.

Inference frequency tab. 4 §4.2. Runtime measured on one GTX 1080 Ti §4.2.

Simulation-time constraints tab. 5 §4.2.

The authors also show an interactive tool that lets a designer change traffic lights and add, edit or remove actors during simulation. This is shown only in the supplementary video; no numbers are given §4.2.

Paper vs code

No official code.

Searched: the paper, abstract and supplement contain no code link, and no project page is linked. GitHub repository search for "trafficsim" returned only unrelated third-party projects. A search for "atg4d" returned an unrelated project. A search for "TrafficSim Suo Casas Urtasun" returned nothing. A search inside the uber-research organisation for "TrafficSim" returned nothing. No code was cloned and no code, code_commit or code_license field is set.

Pitfalls

• Time-adaptive weight formula. The text says the imitation weight must fall to zero at T_label and that λ(t) is a decreasing function §3.4 app. A. The printed formula is λ(t) = min((T_label − t)/T_label, 0) app. A. As printed, λ(t) is 0 for every t up to T_label and negative after it, so the weights in eq. 3 would not give the described schedule. The likely intent is a value that falls from 1 at t = 0 to 0 at T_label, for example max(…, 0). This is an inference from the text; implement the stated intent and record your choice.
• Collision loss. The printed loss is max(1, Σ L_pair) §3.4 app. A. The inner sum is non-negative, so this max is at least 1 and does not depend on the actors' positions. The description ("0 if no overlap, 1 if full overlap") points to min(1, ·) instead. The printed form cannot be used as written.
• Reconstruction sum and P. The reconstruction term sums over τ from t+1 to t+P §3.4, but P is never given a value. The planning horizon T_plan = 10 timesteps tab. 2 is the likely meaning; this is not stated.
• Huber δ, β and latent size. None are stated §3.4. The optimiser, learning rate, batch size and number of training steps are also not stated anywhere in the paper or supplement. A reimplementation cannot reproduce the reported numbers from the text alone.
• Ambiguous "mean" and "distance" metrics. The text names ADE and FDE as distance errors §4.1, but the formulas in app. B use squared norms ‖·‖². The table reports metres tab. 1. Decide whether to use squared or plain distances; the choice changes the values.
• MASD formula. The text says it is the average distance between the two most distinct, rule-compliant samples, computed for each actor §4.1 app. B. The printed formula takes the max over sample pairs outside the sum over actors and time, with squared norms. The max is applied at scenario level in the printed formula, not per actor. The rule-compliance filter is only described in words.
• Algorithm 2 loop. The loop variable advances by one per iteration, but each iteration writes κ steps alg. 2. The step size of the loop is not stated; assume the loop advances by κ.
• Gaussian scale. The algorithm writes N(Z_μ, Z_σ · I) alg. 2. Whether Z_σ is a standard deviation or a variance is not stated.
• "Up to 4x speedup". The text says 4x §4.2. The runtimes in tab. 4 are 0.83 s and 0.24 s at the two ends, about 3.5x. The mapping of rows to κ is not given in the text; it is inferred from the 0.5 s tick.
• Rejection sampling and gradient steps reduce collisions but raise TRV. SCR falls from 0.50 to 0.33 or 0.12, while TRV rises from 2.77 to 3.01 or 3.00 tab. 5. Those trade-offs are not discussed in the text.
• Diversity can be gamed. The authors note that MASD can reward unrealistic diversity §4.1. TrafficSim's MASD (2.50 m) is below MTP's (7.15 m) and AdversarialIL's (4.89 m) tab. 1.
• Ground-truth scenes have collisions. Labelled boxes are slightly larger than vehicles, so the 0.1 IoU threshold is needed; some true scenes still count as collisions app. B. No ground-truth SCR is reported.
• Actors outside the map. Actors without an initial lane association, such as parked cars, are excluded from the rule-violation metric app. B. The number of excluded actors is not reported.
• Baselines run at different rates. IDM is simulated at 2.5 Hz app. B; TrafficSim at 2 Hz §4. Their scores are not measured at the same temporal resolution.
• Table and algorithm numbering. This card follows the order of the LaTeX source, so Table 1 is the main comparison and Algorithms 1 and 2 are SIM and the traffic simulation. The compiled PDF was not read, so check float placement before citing numbers by table number elsewhere.
• Dataset availability. ATG4D is described in detail §4, but the paper does not say whether it is public. Do not assume it can be downloaded.
• Open-loop versus unrolled training. The open-loop model M0 has SCR 5.92% against 0.50% for the full model M* tab. 2. The authors call closed-loop training with backpropagation through simulation the most important component §4.2.

Lineage

• Builds on: the implicit latent variable model for scene-consistent motion forecasting (ILVM, Casas et al., 2020) for the latent scene formulation §3.2; the spatially aware graph neural network (Casas et al., 2019) for the scene interaction module §3.2 app. A; PIXOR and IntentNet for the map backbone §3.1 app. A; rotated RoI Align (Ma et al., 2018) §3.1; the CVAE objective (Sohn et al., 2015) and β-VAE (Higgins et al., 2017) for the imitation loss §3.4; DAgger-style analysis of compounding error §2 §3.4.
• Compared against: IDM tab. 1; MTP, ESP and ILVM as motion forecasters tab. 1; DataAug (ChauffeurNet-style perturbation) and AdversarialIL (GAIL-style with a horizon curriculum) as imitation baselines app. B tab. 1.
• Superseded by: not stated in this wiki.