--- kind: concept slug: motion-prediction title: Motion prediction summary: >- Predicting where people or interacting agents move next from past motion, or a listener's face from a speaker's words. Six variants: a recurrent plan-and-body model, a transformer that outputs a block of future frames, an equivariant interaction model, a discrete-token language model, a joint latent model of traffic, and forecasts fed into a social planner. Reported numbers use different metrics. tags: [motion, sequence-prediction, equivariance, trajectory-forecasting, transformer, discrete-tokens, social-navigation, traffic-simulation, latent-variable-model] papers: ["1809.03036", "2101.08779", "2303.10876", "2308.10897", "2411.01814", "2101.06557"] related: [recurrent-neural-network, euclidean-equivariant-network, graph-neural-network, curriculum-learning, self-attention, multimodal-fusion, vector-quantization, language-model-pretraining, generative-adversarial-network, model-predictive-control, learned-simulator, imitation-learning, differentiable-simulation] status: draft updated_at: 2026-11-01 --- ## Overview Given the recent past of one body or of several agents, predict where each one goes next. For a single skeleton the input is joint angles per frame; for several agents it is positions and, often, velocities [arxiv:2303.10876 §3.1]; for a face it is expression coefficients from a 3D face model [arxiv:2308.10897 §3.1]. The hard part is the long horizon. A model trained only to predict the next frame from real data drifts once it must feed on its own outputs, because errors add up [arxiv:1809.03036 §1] [arxiv:1809.03036 §3.4]. The traffic paper meets the same compounding error by unrolling its model through a differentiable simulator during training and comparing the rollout with logged data [arxiv:2101.06557 §2] [§3.4]. A second family changes the training target instead of the feeding scheme: the model outputs a block of upcoming frames at once, which the dance paper reports as avoiding both freezing and drift [arxiv:2101.08779 §4.1]. For several interacting agents, a further problem appears: a predictor that does not respect rotation and translation must learn the same pattern again for every orientation of the scene [arxiv:2303.10876 §1]. The listener paper is the exception to "predict from past motion". It generates a listener's face from a speaker's words, with no audio or video [arxiv:2308.10897 §6.4]. It is kept here because its card lists this concept. Its metrics are not comparable with the forecasting metrics below. The social-navigation paper uses forecasts rather than producing a new forecasting model for its results. A Social GAN predicts each tracked person's next 12 positions from their last 8 [arxiv:2411.01814 §2.1]. A local planner then adds those predicted positions to its cost [arxiv:2411.01814 §2.2]. Its reported numbers measure navigation in a simulator, not forecast error. ## Variants ### Recurrent plan-and-body model (VTLN-RNN) Two GRUs. A backward-running plan RNN, started from a sample of a learned Gaussian and conditioned on the action label, emits one guide vector per future step. A forward body RNN reads the joint vector and the guide vector at each step and predicts the next pose [arxiv:1809.03036 §3.2]. Training mixes an open-loop term, with ground-truth inputs, and a closed-loop term, with the model's own output fed back. The closed-loop weight starts at zero and rises in steps [arxiv:1809.03036 §3.4] [eq. 6]. The paper's motivation for the backward plan is gradient flow: the top-level parameters then depend on the future [arxiv:1809.03036 §3.2]. Card: [A Neural Temporal Model for Human Motion Prediction](/papers/1809.03036). Reported results are on Human3.6M, test subject 5, four actions, using the authors' NPSS metric. In the 2 to 4 s window the single-layer model has the lowest NPSS for walking, eating and smoking, and the two-layer model has the lowest for discussion [arxiv:1809.03036 tab. 5]. Parameter count is about 3.4 M for the single-layer model, against 14.8 M for ERD [arxiv:1809.03036 tab. 1]. Each training batch needs two forward passes [arxiv:1809.03036 §3.4]. The derivative features the paper adds are not in the released code; see the card before implementing [arxiv:1809.03036 §3.3]. ### Transformer with future-N supervision (FACT) Motion tokens and music tokens each pass through a small transformer. Their outputs are concatenated and read by a 12-layer cross-modal transformer with no attention mask, so every token attends to every other [arxiv:2101.08779 §4.1] [§5.2.1]. The model reads a 120-frame seed (2 s at 60 FPS) and is trained to output the next N = 20 frames under a mean squared error [arxiv:2101.08779 §4.1] [§5.2.1]. Generation is auto-regressive, one frame per step, so the model runs once per generated frame [arxiv:2101.08779 §5.2.1]. The authors report that a causal transformer trained to predict only the next frame freezes after several seconds, and that a full-attention model trained on one future frame drifts [arxiv:2101.08779 §4.1]. Their ablation has FID_k 111.69 for causal attention with shift-by-one supervision, 207.74 for full attention with one future frame, and 35.10 and 35.35 for full attention with 10 and 20 future frames [arxiv:2101.08779 tab. 3]. The paper chose N = 20 on FID_m and BeatAlign, since future-10 has the lower FID_k [arxiv:2101.08779 tab. 3]. Music and motion are fused early, which the paper reports as better on FID_k than late fusion (35.35 against 45.76) [arxiv:2101.08779 tab. 4]. The task is dance, with music as a second input, and the reported results are on AIST++ only [arxiv:2101.08779 tab. 2]. Card: [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](/papers/2101.08779). ### Equivariant interaction model (EqMotion) Each agent carries two features. A geometric feature is a set of coordinates in 3D that rotates and shifts with the scene. A pattern feature is a vector built from invariant quantities, namely speed and turning angle [arxiv:2303.10876 §4.1]. A reasoning step predicts, for each ordered pair of agents, a categorical vector from invariant inputs only [arxiv:2303.10876 §4.2]. Each layer moves the coordinates by a category-weighted sum of differences to the other agents and then applies an equivariant non-linearity [arxiv:2303.10876 §4.3]. The paper proves that the layers as written are equivariant, with the pattern features and categories invariant [arxiv:2303.10876 §4.6]. Card: [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](/papers/2303.10876). Reported results cover particles, molecules, the H3.6M skeleton and ETH-UCY pedestrians. On particle interaction recognition, equivariance gives 100% consistency over 20 random Euclidean transforms, against 93.7% for NRI [arxiv:2303.10876 tab. 1] [§5.1]. On the H3.6M skeleton, average MPJPE is 55.0 mm at 400 ms and 106.9 mm at 1000 ms [arxiv:2303.10876 tab. 2] [tab. 3]. On ETH-UCY, the deterministic average is 0.49 m ADE and 1.03 m FDE [arxiv:2303.10876 tab. 5]. The paper also reports a best-of-20 setting for pedestrians; that row is an oracle selection and must not be compared with single predictions [arxiv:2303.10876 §5.4]. All of these are author-reported. The per-layer cost is quadratic in the number of agents, because the graph is fully connected in the experiments [arxiv:2303.10876 §4.3]. The paper mentions local neighbourhoods for large point sets but does not test them [arxiv:2303.10876 §4.3]. ### Discrete-token language model (listener gestures) A VQ-VAE maps windows of listener face motion to one of 256 codes, one per 8 frames [arxiv:2308.10897 §3.2] [app. B]. GPT-2 Medium, started from text pretraining, then predicts those codes interleaved with the speaker's timed words [arxiv:2308.10897 §3.3]. The only external input is the speaker's transcript; the paper uses no audio or video [arxiv:2308.10897 §6.4]. Inference is greedy, not sampled [arxiv:2308.10897 §3.3]. During training, motion tokens in the input are replaced at random with probability 0.5 [arxiv:2308.10897 app. B]. Card: [Can Language Models Learn to Listen?](/papers/2308.10897). On the Trevor Noah listener, the full model has the lowest FD (18.22), P-FD (19.63) and L2 Affect (6.36) of the table's methods. The unconditional model has lower L2 (0.39 against 0.43), but is worse on the other metrics [arxiv:2308.10897 tab. 1] [§5.1]. Removing GPT-2 pretraining raises FD to 22.81. The variant without interleaving has FD 19.03, close to the full model's 18.22, and the paper calls that gap not significant [arxiv:2308.10897 tab. 2] [§6.1]. In a human A/B test, listeners preferred this model over nearest-neighbour retrieval 70.1% of the time and over the unconditional model 92.8% [arxiv:2308.10897 §5.2]. The preference over the original L2L baseline was 55.7%, a small margin [arxiv:2308.10897 §5.2]. ### Joint latent model of traffic (TrafficSim) One latent variable is drawn for the whole scene. A decoder turns that sample, together with each actor's context, into a plan for every actor at once [arxiv:2101.06557 §3.2] [eq. 1]. Plans drawn from one sample therefore fit together, and repeated samples from the same start give different scenes [arxiv:2101.06557 §3.2]. The map is encoded once per map. Each actor contributes three seconds of boxes, and a 10-step plan is predicted at each tick, of which only the first κ steps are applied [arxiv:2101.06557 §3.3] [app. A] [tab. 2]. The paper compares simulation rates of 2, 1 and 0.5 Hz [arxiv:2101.06557 tab. 4]. Training unrolls the policy for 12 s in closed loop. The first 8 s also carry an imitation loss against logged driving, and prior samples carry a collision penalty [arxiv:2101.06557 §3.4]. Card: [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](/papers/2101.06557). Reported results are on ATG4D, with 15 samples per scenario. The collision rate is 0.50%, against 1.19% for IDM and 2.90% for the ILVM forecaster [arxiv:2101.06557 tab. 1]. Its minimum average displacement error is 0.57 m, against 1.33 m for ILVM [tab. 1]. Its traffic rule violation rate is 2.77%, against 0.25% for IDM [tab. 1]. In the ablation, removing the closed-loop training raises the collision rate from 0.50% to 5.92% [tab. 2]. A planner trained on 15k of its simulated scenes, compared with the same number of real examples, scores close to real data on collision rate (10.73% against 10.56%) and planning L2 (4.52 m against 4.85 m) [tab. 3]. The paper also includes an interactive tool that edits actors and traffic lights, shown only in a video with no numbers [arxiv:2101.06557 §4.2]. The optimiser, learning rate, batch size and epoch count are not stated, and the paper does not give the Huber δ, the KL weight β or the latent size [arxiv:2101.06557 §3.4] [app. A]. No official code was found; see the card before reimplementing. ### Forecasts as planner costs (MP-TEB) A Social GAN forecaster, with a pooling module that models interactions between people, is run on tracked people. It reads the last 8 positions, sampled every 0.5 s, and outputs the next 12 positions, covering 6 s [arxiv:2411.01814 §2.1]. The positions come from RGB-D detections, turned into planar positions with the box centre, depth and camera intrinsics [arxiv:2411.01814 eq. 1]. The predicted paths enter a Timed Elastic Band local planner as three extra cost terms: a term that follows a predicted human path [eq. 5], a term that tracks a predicted dynamic-obstacle position [eq. 6], and a priority-side term that is subtracted from the cost [eq. 7]. They are added to the base TEB cost with a normalisation factor δ_mp [eq. 8] [arxiv:2411.01814 §2.2]. Card: [the social-navigation planner paper](/papers/2411.01814). Reported results are from Gazebo, four scenarios, 10 runs each averaged, against DWA and TEB [arxiv:2411.01814 §3.1] [§3.2]. MP-TEB has the shortest total time in three of four scenarios and the largest minimum human-robot distance in three of four [arxiv:2411.01814 tab. 1]. The authors do not report variance [§3.2] [tab. 1]. The planner itself is not in the released repository, so the cost has to be built from the equations the card's Paper vs code section gives the details. ## When to use which | Situation | Use | Why | |---|---|---| | One body, horizon of several seconds, generation feeds on its own output | Recurrent plan-and-body | Closed-loop training targets drift directly [arxiv:1809.03036 §3.4]; the two-layer model has lower NPSS than the auto-conditioning baseline on three of four actions [arxiv:1809.03036 tab. 4] | | Generation driven by an external signal such as music, with a training set of a few hundred sequences | Transformer with future-N supervision | Predicting 20 frames gave the best FID_m and BeatAlign in the paper's ablation [arxiv:2101.08779 tab. 3]; the only dataset reported is AIST++, with 329 training choreographies [arxiv:2101.08779 §5.2.1] | | Only a speaker's transcript is available, and the output is a listener's face or head | Discrete-token language model | Best FD, P-FD and L2 Affect on the Trevor Noah listener [arxiv:2308.10897 tab. 1]; without pretraining FD is worse (22.81) [tab. 2]. The result is for one listener; on Stephen, the Unaligned variant has the lowest FD [app. C] | | Tight per-frame latency or small hardware | Joint latent model at a lower simulation rate, if some collisions can be accepted | The only timing reported is for the traffic model: one 12 s rollout takes 0.83 s at 2 Hz and 0.24 s at 0.5 Hz on one GTX 1080 Ti, and the collision rate rises from 0.50% to 0.96% [arxiv:2101.06557 tab. 4]. The transformer runs a full forward pass per generated frame [arxiv:2101.08779 §5.2.1]; the listener model runs one forward pass per motion token with 345M parameters [arxiv:2308.10897 §3.3] [app. B]; the recurrent paper reports parameter counts but no timing [arxiv:1809.03036 tab. 1] | | Simulating a whole traffic scene, where other vehicles must act like human drivers and collisions must be rare | Joint latent model of traffic | Lowest collision rate of the compared models on ATG4D, 0.50% [arxiv:2101.06557 tab. 1]. Its traffic rule violation rate is higher than IDM's (2.77% against 0.25%) [tab. 1], so a simulator that must obey rules may need the rule-based model; the training recipe is incomplete (see Contradictions) | | Several interacting agents with known positions | Equivariant interaction | Outputs follow rigid transforms of the input by construction [arxiv:2303.10876 §4.6]; the interaction categories use invariant inputs [arxiv:2303.10876 §4.2] | | Input frames may be rotated or shifted relative to training data | Equivariant interaction | Consistency under random transforms is reported as 100% for particles [arxiv:2303.10876 tab. 1]. The released H3.6M code adds terms the proof does not cover, so test this yourself | | Many agents (crowds, large graphs) | Neither, untested at scale | The equivariant model's cost grows with agent pairs [arxiv:2303.10876 §4.3]; the paper does not test its local-neighbourhood option | | Judging whether generated motion has the right rhythm, not only the right pose | NPSS from the recurrent paper | The authors' user study agrees with human choices more often than MSE does (65 of 80 against 39 of 80; author-reported, small study) [arxiv:1809.03036 §4.2] [tab. 3] | | Judging pose accuracy at fixed times | MPJPE or ADE/FDE | This is the metric the equivariant paper uses [arxiv:2303.10876 §5.3] [§5.4]. For listener faces the paper uses FD, P-FD and L2 Affect instead [arxiv:2308.10897 §4.2] | | Robot must plan its own path around people, and forecasts are one input among several | Forecast-conditioned planner | The paper reports the shortest total time in three of four simulated scenarios [arxiv:2411.01814 tab. 1]. Its TEB baseline is the planner without the social terms, but the paper does not say whether the baseline parameters match MP-TEB's [§3.1]; the study is simulation only | | Comparing with a published number | The same metric and protocol | The five papers' numbers are not comparable; see Contradictions | ## Implementation sketch Five sketches, one for each of the recurrent, transformer, equivariant, listener and planner variants, plus one tick of the traffic model. They are not one shared implementation; the recurrent objective, the transformer's block loss, the equivariant layer, the token generation loop and the traffic tick are independent. ```pseudocode # Training objective of the recurrent variant [arxiv:1809.03036 §3.4] [eq. 6] # Input: seed frames, future (T_f, D) ground truth, weight lam in [0, 1] # Output: scalar loss def recurrent_loss(seed, future, lam): # Open loop: ground-truth frames are fed in at every step [§3.4] open_pred = rollout(seed, future, feed="truth") # Closed loop: the model's own output is fed back [§3.4] closed_pred = rollout(seed, future, feed="own") # lam starts at zero and is raised in steps over training [§3.4] return mse(open_pred, future) + lam * mse(closed_pred, future) ``` ```pseudocode # Block loss of the transformer variant [arxiv:2101.08779 §4.1] # Input: seed (120, D) observed motion; music (240, A) aligned with it # target (20, D) the next 20 frames # Output: scalar loss def future_n_loss(model, seed, music, target): # One forward pass emits a block of frames, not one [§4.1] predicted = model(seed, music) # (T_out, D) # Compare the first N = 20 predicted positions with the target [§4.1] difference = target - predicted[: length(target)] return mean(square(difference)) ``` ```pseudocode # One update of the equivariant variant [arxiv:2303.10876 §4.2] [§4.3] # Input: pos (N, C, 3) geometric features, h (N, D) pattern features # cat (N, N, K) interaction categories, from invariant inputs # Output: pos (N, C, 3), h (N, D) def equivariant_layer(pos, h, cat): # Pairwise distances are invariant, so they can weight messages [§4.2] dist = pairwise_norm(pos) # Each agent moves by a category-weighted sum of its differences [§4.3] pos = pos + aggregate(pos, h, dist, cat) # Pattern features update from invariant inputs only [§4.4] h = message_pass(h, dist) return pos, h ``` ```pseudocode # Greedy generation of listener codes [arxiv:2308.10897 §3.3] # Input: speaker words with end frames; n_slots motion slots of 8 frames # Output: one motion code (0 .. 255) per slot, decoded by the VQ-VAE def generate_listener(words, word_ends, n_slots, lm): sequence = [] next_word = 0 for slot in range(n_slots): # Speaker words that end before this slot go in first [§3.3] while next_word < len(words) and word_ends[next_word] < 8 * slot: sequence.append(words[next_word]) next_word += 1 # Greedy: the most likely code, no sampling [§3.3] logits = lm(sequence)[-1] # (256,) over motion codes sequence.append(motion_token(argmax(logits))) return sequence ``` ```pseudocode # Social terms added to a TEB band cost [arxiv:2411.01814 eq. 5-8] # Input: band (N, 2) planned robot positions; human_paths (P, 12, 2) # forecast per person; obstacles: list of (position, velocity) # Output: scalar social term, added to the base TEB cost def social_cost(band, human_paths, obstacles, w1, w2, delta_mp): total = 0.0 for t in range(len(band)): # Human-path term, one person at a time [eq. 5] [§2.2] # How several people combine is not stated; this sums them for path in human_paths: total += w1 * distance(band[t], path[t]) # Obstacle term, moved by its velocity over time [eq. 6] # Time unit of t is not stated; the sign of w1 and w2 is not stated for position, velocity in obstacles: total += w2 * distance(band[t], position + velocity * t) # Priority-side term, subtracted [eq. 7] [eq. 8] total -= priority_side(band[t], obstacles) # Eq. 8 has no sum over the band; summing it here is an assumption return delta_mp * total ``` ```pseudocode # One tick of the joint latent traffic model [arxiv:2101.06557 §3.3] [§3.2] # Input: actor contexts (N, 192), cached map already encoded # Output: the next kappa steps of a plan for each of the N actors def joint_tick(context, kappa): # One latent sample for the whole scene, not one per actor [§3.2] [eq. 1] z = sample_gaussian(*prior_network(context)) # Decode a plan for all actors together, so the plans fit [§3.2] plan = decoder(context, z) # (N, T_plan, ...) per actor # Apply only the first kappa steps, then build the next context [§3.3] return plan[:, :kappa] ``` ## Contradictions - **Within the recurrent paper.** Section 5 says both GRU models beat the auto-conditioning baseline on all four actions [arxiv:1809.03036 §5]. In Table 4, that baseline has the lowest NPSS for discussion (1.651, against 1.688 and 1.777) [arxiv:1809.03036 tab. 4]. In the 2 to 4 s window, the two-layer model is ahead for discussion (0.785 against 0.811) [arxiv:1809.03036 tab. 5]. The claim holds for that window only. - **Between the papers, failure modes.** The recurrent paper describes drift when a next-frame model feeds on its own output [arxiv:1809.03036 §3.4]. The dance paper describes freezing for a causal next-frame model and drift for a full-attention one [arxiv:2101.08779 §4.1]. These are different models and tasks, so the fixes are not interchangeable. - **Between the papers, metrics.** The recurrent paper argues that per-frame error penalises plausible motion that is only a little faster, slower or shifted, and proposes NPSS [arxiv:1809.03036 §3.5]. The equivariant paper reports MPJPE and ADE/FDE [arxiv:2303.10876 §5.3] [§5.4]. The dance paper reports FID_k, FID_m, diversity and BeatAlign [arxiv:2101.08779 §5.2.2]. The listener paper reports L2, FD, P-FD and L2 Affect on face coefficients [arxiv:2308.10897 tab. 1]. The traffic paper reports collision rate, rule violation and minimum or mean SFDE and SADE on ATG4D [arxiv:2101.06557 tab. 1]. The planner paper reports path length, total time and minimum human-robot distance in a simulator [arxiv:2411.01814 tab. 1]. No two of the six report the same metric set, so the numbers cannot be set side by side. The traffic and equivariant papers both report best-of-many displacement errors, and their definitions differ [arxiv:2101.06557 §4.1] [arxiv:2303.10876 §5.4]. - **Between the papers, protocol.** The recurrent and equivariant papers both use Human3.6M with six training subjects and subject 5 for test [arxiv:1809.03036 §3.1] [arxiv:2303.10876 app. D.1.3]. The horizons, joint sets and metrics differ, so the recurrent paper's short-term MSE table is not comparable with the equivariant paper's MPJPE table [arxiv:1809.03036 tab. 6] [arxiv:2303.10876 tab. 2]. The dance paper uses its own AIST++ split [arxiv:2101.08779 §5.2.1]. The traffic paper uses ATG4D lidar tracks, a 12 s horizon with 8 s of labels, and 15 samples per scenario [arxiv:2101.06557 §4]. The listener paper uses the Learning to Listen data with its own segmentation [arxiv:2308.10897 §4.1]. - **Within the traffic paper, metrics.** The full model M* has the lowest collision rate and traffic rule violation in its ablation, but the unrolled model with one-step plans, M2, has lower minimum SFDE and SADE (1.02 m and 0.54 m, against 1.13 m and 0.57 m) and a higher collision rate (1.28% against 0.50%) [arxiv:2101.06557 tab. 2]. The paper reports M*. - **Within the traffic paper, text and table.** The text describes TrafficSim's rule violation as similar to IDM's [arxiv:2101.06557 §4]. Table 1 gives 2.77% against 0.25% [tab. 1]. Quote the table. - **Within the traffic paper, post-processing.** Rejection sampling and gradient optimisation lower the collision rate from 0.50% to 0.33% and 0.12%, but raise rule violation from 2.77% to 3.01% and 3.00%. The text does not discuss this trade-off [arxiv:2101.06557 tab. 5]. - **Within the traffic paper, loss formulas.** The text says the imitation weight falls to zero at 8 s, but the printed weight is zero up to 8 s and negative after it [arxiv:2101.06557 app. A]. The printed collision loss, max(1, ·), is at least 1 and does not depend on positions, while the description implies a value between 0 and 1 [§3.4]. Implement the stated intent and record the choice. - **Within the dance paper.** FACT's FID_m is 12.40 in the main table and 12.39 in the ablations [arxiv:2101.08779 tab. 2] [tab. 3]. Quote the table you use. - **Within the listener paper, history length.** Section 6.3 calls 8 s of history the sweet spot. The configuration labelled "Ours" in the appendix uses 3 s, as does the README [arxiv:2308.10897 §6.3] [app. C]. Pick one and cite the table you use. - **Within the listener paper, across listeners.** On the Trevor Noah listener, the full model is best on FD. On Stephen, the Unaligned variant has the lowest FD (33.47) and P-FD [arxiv:2308.10897 app. C]. The headline is for one listener. - **Headline figures in the equivariant paper.** The abstract gives four percentage gains without naming their metrics. The body gives 8.6% for H3.6M short-term and 3.8% for long-term [arxiv:2303.10876 §5.3], 34.2% and 30.1% for molecules [§5.2], and 10.4% and 7.9% for pedestrians [§5.4]. The body does not state the averaging baseline behind the abstract's 24.0% and 9.2% [arxiv:2303.10876 §1]. Treat these as unverified. - **Within the social-navigation paper, text and table.** The text says MP-TEB beats DWA and TEB on both efficiency and safety in every scenario [arxiv:2411.01814 §3.2]. Table 1 does not show this. In the corridor scenario DWA has the shorter total time (23.12 s against 24.46 s), and in the multi-person scenario TEB has the larger minimum distance (0.46 m against 0.36 m) [arxiv:2411.01814 tab. 1]. Quote the table, not the claim. - **Between the planner paper and the forecasting papers, sign and scope.** The planner paper's cost terms are added to the cost with a positive factor, which read literally pulls the robot toward the predicted human and obstacle positions [arxiv:2411.01814 eq. 5-8]. The text describes avoidance [arxiv:2411.01814 §2.2]. This is unresolved on the card and is not settled here. The forecasting papers do not address planners. ## Papers - [A Neural Temporal Model for Human Motion Prediction](/papers/1809.03036): a backward plan GRU guides a forward body GRU, trained with a closed-loop weight to resist drift; proposes NPSS. - [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](/papers/2101.08779): a full-attention transformer that predicts 20 future frames from music and a 2-second seed; also introduces the AIST++ dataset. - [EqMotion: Equivariant Multi-agent Motion Prediction with Invariant Interaction Reasoning](/papers/2303.10876): an equivariant graph-style predictor for interacting agents, with invariant interaction categories; author-reported results on four kinds of data. - [Can Language Models Learn to Listen?](/papers/2308.10897): a VQ-VAE turns listener face motion into 256 tokens, which a text-pretrained GPT-2 Medium predicts from the speaker's timed words; author-reported results on one listener. - [TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors](/papers/2101.06557): a latent-variable model that plans all vehicles jointly and is trained by unrolling through a differentiable simulator; author-reported results on ATG4D. - [Social navigation with forecasts inside a local planner](/papers/2411.01814): a Social GAN forecast of each person's next 12 positions adds three cost terms to a TEB local planner; author-reported results in Gazebo.