---
kind: concept
slug: recurrent-neural-network
title: Recurrent neural network (RNN)
summary: >-
  A recurrent network carries a hidden state along a sequence. LSTM and GRU cells recur across the cited work, from forecasting and tracking to OCR, dialogue emotion, text generation and world models. In one author-reported forecasting benchmark, LSTM won the most datasets on best-of-grid accuracy, a ranking that selects on the test split.
tags: [sequence-modelling, time-series, forecasting, recurrent-neural-network]
papers: ["1512.05463", "1512.08756", "1601.06615", "1604.03635", "1609.05473", "1704.02532", "1706.01040", "1708.03498", "1711.04226", "1712.04997", "1802.02080", "1806.03713", "1809.01999", "1809.03036", "1811.00405", "1907.08456", "2003.01200", "2004.08919", "2010.02795", "2101.12072", "2102.04942", "2103.12057", "2106.15004", "2108.00298", "2208.04933", "2312.00752", "2404.09383", "2509.24852", "2012.02181", "2103.09504", "2512.06714", "1611.09904", "1910.11470", "2303.14483", "1708.02709", "2104.13371", "2405.04517", "1603.03827", "1803.01164", "1903.10128", "1905.02947", "2101.11174", "2312.06639", "1607.04423", "1704.02581", "2104.00681", "1806.06228", "1910.10536"]
related: [convolutional-recurrent-network, long-short-term-memory, linear-attention, image-super-resolution, deformable-convolution, optical-flow, attention-mechanism, time-series-forecasting, temporal-convolutional-network, emotion-recognition-in-conversation, generative-adversarial-network, language-model-pretraining, graph-neural-network, convolutional-neural-network, backpropagation-through-time, diffusion-models, conditional-random-field, expectation-maximization, visual-object-tracking, named-entity-recognition, multi-task-learning, motion-prediction, imitation-learning, text-classification, spiking-neural-network, surrogate-gradient, learnable-delays, state-space-model, parallel-scan, adversarial-text-generation, policy-gradient, model-based-reinforcement-learning, variational-autoencoder, neuroevolution, curriculum-learning, image-super-resolution, k-means-clustering, sequence-labeling, extractive-question-answering, data-augmentation, multi-view-reconstruction, neural-implicit-surface, monocular-depth-estimation, multimodal-fusion, sentiment-analysis, self-attention, time-series-classification]
status: draft
updated_at: 2026-11-01
---

## Overview

A recurrent neural network (RNN) reads a sequence one step at a time. At each
step it combines the current input with a hidden state that summarises the
steps before. The output can be the last hidden state, or the whole sequence
of hidden states, which a later layer then reads. The cited papers use this
pattern for next-value forecasting [arxiv:2103.12057 §2] [arxiv:2103.12057 §3.2.1],
scene text recognition [arxiv:1711.04226 §4.1], dialogue
emotion tracking [arxiv:2010.02795 §3.4], satellite image time series
[arxiv:1802.02080 §3.1], motion synthesis [arxiv:2102.04942 §3.2],
station-level bike demand [arxiv:1712.04997 §3.3.2], molecule and protein
encoders [arxiv:2004.08919 app. A] and sequence-to-sequence language tasks
[arxiv:2003.01200 §3.2], multi-target pedestrian tracking
[arxiv:1604.03635 § Target Motion], object grouping in an unrolled EM
clustering [arxiv:1708.03498 §2.3], probabilistic multivariate forecasting
[arxiv:2101.12072 §3] and named entity tagging [arxiv:2404.09383 §2.3]. The
newer cards add a shared recurrent stack for rumour verification
[arxiv:1806.03713 §4.2], a backward plan network for human motion
[arxiv:1809.03036 §3.2], GRU encoders over a lane graph [arxiv:2106.15004 app. A.2],
graph-gated imputation over sensors [arxiv:2108.00298 §4], recurrent spiking
layers with learned delays [arxiv:2509.24852 §2.1] and, in a survey, CNN
features feeding one LSTM [arxiv:1601.06615 §3.4.1]. Three more cards use
recurrence for video restoration [arxiv:2012.02181 §3.1], spatiotemporal video
prediction [arxiv:2103.09504 §4.1] and short-term water demand
[arxiv:2512.06714 §3.4].

Several of the newer cards change how the recurrence is used. SeqGAN uses an
LSTM as a text policy trained by policy gradient [arxiv:1609.05473 § The
Generative Model for Sequences]. DialogueRNN tracks several GRU states per
conversation [arxiv:1811.00405 §3.3]. The EA-LSTM feeds static basin attributes
only into its input gate [arxiv:1907.08456 §2.2]. The state-space cards S5 and
Mamba keep a recurrent form but compute it as a linear or selective scan
[arxiv:2208.04933 §2.2] [arxiv:2312.00752 §3.3]. The driving paper uses an LSTM
for temporal integration without measuring it [arxiv:1704.02532 §4.2]. The
Brain Intelligence position paper names no recurrent design
[arxiv:1706.01040 §3]. PredRNN sends its memory up the layer stack as well as
forward in time [arxiv:2103.09504 §4.1], and BasicVSR runs two recurrent sweeps
in opposite time directions that do not exchange information
[arxiv:2012.02181 §3.1].

Three more cards add recurrence in other roles. C-RNN-GAN uses LSTMs as the
generator and discriminator of a GAN over continuous music events
[arxiv:1611.09904 §2] [arxiv:1611.09904 §3]. The named-entity survey ranks
word-plus-character recurrent taggers as its strongest neural family
[arxiv:1910.11470 §6.4.3]. The urban graph survey lists the GRU as the most
widely used recurrent unit, and shows it both after a graph convolution and
inside one [arxiv:2303.14483 §5.2] [arxiv:2303.14483 §5.5]. The NLP survey
gives the simple RNN, LSTM and GRU equations. It reports that both gated cells
beat the simple RNN on music and speech, and that neither is clearly better
[arxiv:1708.02709 §4.2].

Two newer cards change the recurrent cell or its propagation. The xLSTM card
keeps the LSTM's gated cell but adds exponential gates, a normaliser and a
matrix memory, and reports language-model results against Transformers and
state-space models [arxiv:2405.04517 §2] [arxiv:2405.04517 §4.2]. The
BasicVSR++ card keeps bidirectional recurrent propagation, but runs it as a
second-order grid and aligns the carried states with flow-guided deformable
convolution, not a gated cell [arxiv:2104.13371 §3.1] [arxiv:2104.13371 §3.2].

The paper's recurrent family for forecasting has four cells. The Elman RNN
(ERNN) is the plain form. LSTM and GRU add gates to control what the hidden
state keeps. The echo state network (ESN) is the fourth cell in the comparison
[arxiv:2103.12057 §2]. Each cell is trained as a global model over all series
of one dataset, with the same input windows and the same output head
[arxiv:2103.12057 §3.1] [arxiv:2103.12057 §3.3].

The forecasting evidence is one author-reported benchmark on twelve public
datasets. It compares seven architectures and reports no classical statistical
baselines [arxiv:2103.12057 §3.2]. Treat its rankings as a guide to where to
start, not as a settled result. The other papers test different tasks, with
their own baselines, so their results do not rank the cells against each other.

Five more cards add recurrence in other roles. A sequential short-text
classifier encodes each text with an LSTM or a CNN, then runs a small history
layer over the texts in order [arxiv:1603.03827 §2]. RBPN carries a
high-resolution feature estimate through a loop over neighbouring video frames
[arxiv:1903.10128 §3]. The emotion-in-conversation survey summarises context
models built on recurrence, mostly DialogueRNN, and copies their results from
earlier papers [arxiv:1905.02947 §4]. The traffic GNN survey splits
spatiotemporal GNNs into RNN-based and CNN-based models, with DCRNN as a
classic example [arxiv:2101.11174 §4.2]. The deep learning survey restates the
LSTM, GRU and ConvLSTM equations and runs no experiments of its own
[arxiv:1803.01164 §V].

Two more cards use recurrence over sequences of other kinds. The multimodal
sentiment model runs one GRU per modality over the utterances of a clip, then
a GRU after each fusion stage [arxiv:1806.06228 §3.3] [arxiv:1806.06228 §3.4].
The satellite crop-type study compares an LSTM, which follows the authors'
earlier model, with a Transformer and two convolutional networks on raw
Sentinel-2 series [arxiv:1910.10536 §4.1] [arxiv:1910.10536 tab. 1].

Three newer cards put recurrence inside a larger model. A reading-comprehension
reader encodes the passage and the question with a bi-directional GRU, then
pools the passage with attention over attention [arxiv:1607.04423 §3]. A
skeleton action model runs two LSTM streams, one over joints per frame and one
over joints as an ordered sequence [arxiv:1704.02581 §4]. A monocular
reconstruction network fuses each fragment with earlier fragments through a
ConvGRU [arxiv:2104.00681 §3.2].

## Variants

### LSTM (long short-term memory)

Gated cell. In the benchmark it had the best single configuration on 4 of 12
datasets, more than any other recurrent cell [arxiv:2103.12057 §4.1] [arxiv:2103.12057 tab. 11].
The paper calls it the most accurate choice when tuned well and gives it the
best accuracy among the models with adequate training time [arxiv:2103.12057 §4.2].
Its best settings were one or two layers, with batch size 32 and min-max
normalization [arxiv:2103.12057 tab. 17] [arxiv:2103.12057 tab. 18].
Card: [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](/papers/2103.12057).

### GRU (gated recurrent unit)

Gated cell. Its best single configuration won on 2 datasets, CIF2016o12 and
M3 [arxiv:2103.12057 tab. 8] [arxiv:2103.12057 tab. 11]. The configurations
with one and two layers were both marked significantly best [arxiv:2103.12057 tab. 17].
Its best settings used min-max normalization with batch size 64 [arxiv:2103.12057 tab. 18].
The paper reports no significant difference between its best cells and LSTM's,
except that the MLP is worst [arxiv:2103.12057 §4.3]. An NLP survey reports
that neither gated cell is clearly better on music and speech tasks, and notes
that those comparisons are not on NLP tasks [arxiv:1708.02709 §4.2].
Card: [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](/papers/2103.12057).

### Exponential gating with scalar and matrix memories (xLSTM)

xLSTM keeps the LSTM's additive, gated cell state but makes the input gate, and
optionally the forget gate, exponential. A normaliser state and a stabiliser
state keep the exponentials finite, and the paper says the stabilised form gives
the same outputs and gradients [arxiv:2405.04517 §2.2] [arxiv:2405.04517 app. A.2].
The sLSTM keeps a scalar memory with memory mixing inside each head. The mLSTM
has a d × d matrix memory, updated with a covariance rule and read with a query,
and no hidden-to-hidden link, so it can be computed in parallel over the sequence
[arxiv:2405.04517 §2.3]. In the paper's 15B-token ablation, a vanilla multi-layer
LSTM reaches 2417.86 validation perplexity. Adding a residual backbone brings it
to 35.46, exponential gating to 17.70, and matrix memory to 13.48
[arxiv:2405.04517 tab. 2]. The 15B-token comparison puts xLSTM[1:0] at 13.43,
against 13.70 for Mamba and 14.25 for Llama [arxiv:2405.04517 tab. 1]. At 300B
tokens, the 1.3B xLSTM[1:0] reaches 8.89, against 9.14 for Mamba and 9.44 for
Llama [arxiv:2405.04517 tab. 3]. All numbers are author-reported.
Card: [xLSTM: Extended Long Short-Term Memory](/papers/2405.04517).

### ERNN (Elman recurrent network)

The simplest recurrent cell in the comparison, with no gates. Its best cell won
on 2 datasets. Its best settings used one or two layers, 32 or 64 units, and
returned only the last output [arxiv:2103.12057 tab. 17]. Its best single
configuration won on 2 datasets, M4 and Tourism [arxiv:2103.12057 tab. 8] [arxiv:2103.12057 tab. 11].
Card: [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](/papers/2103.12057).

### ESN (echo state network)

Its best single configuration won on ExchangeRate, dataset 3 [arxiv:2103.12057 tab. 8]
[arxiv:2103.12057 tab. 11]. Its best configurations used two or four layers and
returned the full sequence [arxiv:2103.12057 tab. 17]. The paper reports slow inference for this cell
[arxiv:2103.12057 §4.2] [arxiv:2103.12057 fig. 4]. The code also adds an extra
dense layer before the output for ESN only, which the paper does not describe
[arxiv:2103.12057 Paper vs code].
Card: [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](/papers/2103.12057).

### Bidirectional LSTM encoder with attention decoder

The input is a sequence of image features read in both directions. A
bidirectional LSTM encodes each direction, and an LSTM decoder with attention
writes one character per step, trained end to end from word labels
[arxiv:1711.04226 §4.1] [arxiv:1711.04226 §3.4]. The decoder has 256 memory
blocks; its layer count is not stated [arxiv:1711.04226 §5.2]. The authors
report the best lexicon-free result on several irregular benchmarks, but their
own tables show other methods ahead on some columns [arxiv:1711.04226 §5.3].
Card: [AON: Towards Arbitrarily-Oriented Text Recognition](/papers/1711.04226).

### LSTM generator in an adversarial setup

The LSTM is the recurrent core of a motion generator. It takes concatenated
state and target embeddings and decodes them into velocities
[arxiv:2102.04942 §3.2]. Two adversarial critics and reconstruction losses
train it [arxiv:2102.04942 §3.2]. The paper says it chose LSTM over GRU
empirically, and gives no hidden size or depth for the LSTM [arxiv:2102.04942 §5.5]
[arxiv:2102.04942 §3.2].
Card: [Robust Motion In-betweening](/papers/2102.04942).

### LSTM after a learned graph filter (station bike demand)

The LSTM is the recurrent block of GCNN_rec-DDGF. Each hour, the demand vector
of all stations is first mixed through a learned graph filter. The result then
feeds an LSTM with 24 time steps and 100 hidden units per cell
[arxiv:1712.04997 §3.3.2] [arxiv:1712.04997 tab. 2]. The paper does not write
out the LSTM gate equations [arxiv:1712.04997 §3.3.2]. The public code has no
recurrent variant [arxiv:1712.04997 Paper vs code]. On 272 Citi Bike stations
it has the lowest test RMSE of 13 models: 2.12, against 2.35 for the regular
variant without the recurrent block and 2.46 for a plain LSTM baseline
[arxiv:1712.04997 tab. 3]. These are author-reported figures from one dataset.
Card: [Predicting Station-level Hourly Demands in a Large-scale Bike-sharing Network: A Graph Convolutional Neural Network Approach](/papers/1712.04997).

### CNN with a bidirectional GRU or LSTM (drug and protein encoders)

DeepPurpose offers a bidirectional GRU or LSTM on top of a 1D CNN as a compound
encoder over SMILES strings. For proteins it offers a bidirectional RNN on top
of a CNN [arxiv:2004.08919 app. A]. The paper does not state the layer count or
hidden size; the code defaults are 2 layers and 64 units [arxiv:2004.08919 app. A].
The results table in the card does not isolate this encoder. It shows the
CNN+CNN and MPNN+AAC pairs on DAVIS, and the text says many DeepPurpose models
perform comparably [arxiv:2004.08919 §3].
Card: [DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction](/papers/2004.08919).

### GRU state trackers in a dialogue model

COSMIC runs five GRU cells over a conversation: one for context, and separate
ones for each speaker's internal state, external state and intent, plus one for
emotion [arxiv:2010.02795 §3.4]. The GRU sizes are not stated in the paper.
The released code uses 150 units for most states and 450 for the emotion state
[code: COSMIC/erc-training/train_iemocap.py#L149-L158]. On IEMOCAP it reports
65.28 weighted F1 against 64.76 for a RoBERTa DialogueRNN baseline
[arxiv:2010.02795 tab. 4]. Its ablations that drop the commonsense inputs lower
the IEMOCAP score [arxiv:2010.02795 tab. 4]. The recurrent cells are one part of
a larger system with pretrained encoders and commonsense features.
Card: [COSMIC: COmmonSense knowledge for eMotion Identification in Conversations](/papers/2010.02795).

### Convolutional GRU and LSTM

The recurrent cells here are convolutional, with kernel size 3, and read a
Sentinel-2 time series in both directions [arxiv:1802.02080 tab. 1] [arxiv:1802.02080 §5.2].
The main model is a ConvGRU with 256 cells, trained for 60 epochs with batch
size 32; a ConvLSTM was also trained, with batch size 28 [arxiv:1802.02080 §5.2].
The reported results are for the GRU encoder only [arxiv:1802.02080 tab. 2].
See [convolutional-recurrent-network](/concepts/convolutional-recurrent-network)
for the general pattern.
Card: [Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders](/papers/1802.02080).

### LSTM encoder-decoder for sequence-to-sequence

An encoder reads the input, and a decoder writes the output, usually with RNN or
LSTM cells [arxiv:2003.01200 §3.2]. Training uses teacher forcing and a
cross-entropy loss [arxiv:2003.01200 eq. 1] [arxiv:2003.01200 §3.2]. The
survey gives the LSTM objective for translation [arxiv:2003.01200 eq. 8]. It
runs no experiments of its own, and its tables copy scores from the works it
cites [arxiv:2003.01200 tab. 2].
Card: [Natural Language Processing Advancements By Deep Learning: A Survey](/papers/2003.01200).

### Sequence memory without backpropagation (HTM)

This model is recurrent only in the sense that it carries context from one
element to the next. It is not trained with gradients. Each element is a sparse
binary code over 2048 columns with 32 cells per column [arxiv:1512.05463 tab. 1].
Its learning rule is Hebbian-like and runs in a single pass over the stream
[arxiv:1512.05463 §3.3] [arxiv:1512.05463 §6.1]. The authors report NYC taxi
demand error comparable to the best LSTM, and below ARIMA, ELM and ESN
[arxiv:1512.05463 §5] [arxiv:1512.05463 fig. 10]. Under temporal noise present
throughout training, LSTM is slightly better [arxiv:1512.05463 §4.5]. On the
Reber grammar task HTM is below LSTM [arxiv:1512.05463 §6.4].
Card: [Continuous online sequence learning with an unsupervised neural network model](/papers/1512.05463).

### Feed-forward attention pooling (no recurrence)

Each step is mapped to a state on its own, and a learned attention score
averages the states over time into one vector [arxiv:1512.08756 §2.1]. The
paper's stated motivation is that recurrent networks are trained by
backpropagation through time and cannot be fully parallelised over long inputs
[arxiv:1512.08756 §1]. On two toy tasks it reaches perfect accuracy for lengths
from 50 to 10000 in its fixed-length experiment [arxiv:1512.08756 tab. 1]. One
epoch at length 10000 took 254 s, against 917 s for a single-layer vanilla RNN
of hidden size 100 on the same GPU [arxiv:1512.08756 §2.2]. The RNN is a speed
reference only. The paper reports no tuned RNN baseline on these tasks
[arxiv:1512.08756 §2.2].
Card: [Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems](/papers/1512.08756).

### RNN motion model with LSTM data association (online tracking)

Two learned blocks track an unknown number of pedestrians frame by frame. An
RNN predicts each target's next state and its existence probability. A
per-target update then mixes the prediction with its assigned detection
[arxiv:1604.03635 § Multi-Target Tracking with RNNs] [arxiv:1604.03635 § Target Motion].
An LSTM reads each target's row of distances to the detections and outputs an
assignment over the detections plus one "missing" column [arxiv:1604.03635 § Data Association with LSTMs].
The models are trained only on synthetic trajectories [arxiv:1604.03635 § Training Data].
On the MOTChallenge 2015 test set it reports 19.0% MOTA and 165.2 FPS, which
the authors say is not competitive on accuracy but faster than the methods
they list [arxiv:1604.03635 tab. 2] [arxiv:1604.03635 § Benchmark results].
On the training set, the variant with Hungarian assignment has higher MOTA than
the LSTM variant, 24.0 against 22.3 [arxiv:1604.03635 tab. 1].
Card: [Online Multi-Target Tracking Using Recurrent Neural Networks](/papers/1604.03635).

### RNN memory inside unrolled EM clustering (RNN-EM)

Neural EM unrolls the Expectation Maximization steps into a differentiable
clustering over pixels. RNN-EM replaces the gradient step that EM derives
with an unrestricted RNN. Each component's hidden state receives its
responsibility-weighted residual [arxiv:1708.03498 §2.3] [arxiv:1708.03498 fig. 2].
Because the weights are not restricted, the likelihood no longer carries a
convergence guarantee, but the recurrent form can predict the next frame of
a sequence [arxiv:1708.03498 §2.3]. The recurrent unit has 250 sigmoid
units for static shapes and 100 for flying shapes [arxiv:1708.03498 app. A.1] [arxiv:1708.03498 app. A.2].
On static shapes, RNN-EM scores AMI 0.826 ± 0.005 against 0.475 ± 0.043 for
N-EM [arxiv:1708.03498 §4.1]. On flying shapes with three objects it scores
0.970 ± 0.005 [arxiv:1708.03498 tab. 1]. These are author-reported results on
synthetic shapes and digits.
Card: [Neural Expectation Maximization](/papers/1708.03498).

### LSTM conditioner for a diffusion forecaster (TimeGrad)

An LSTM encodes the past of a multivariate series, and a conditional denoising
diffusion model samples each future step. The noise predictor is conditioned on
the RNN state from the previous step [arxiv:2101.12072 §3] [arxiv:2101.12072 eq. 10].
The RNN has two LSTM layers of hidden size 40 [arxiv:2101.12072 §4.2].
The model draws 100 sample paths, each with 100 reverse diffusion steps per
forecast step [arxiv:2101.12072 §4.2] [arxiv:2101.12072 alg. 2]. The authors
report that the number of diffusion steps can fall to about 10 with little
loss, against a best value near 100 on Electricity [arxiv:2101.12072 §4.4] [arxiv:2101.12072 fig. 3].
On CRPS_sum it has the best score on five of six datasets, and on Exchange,
the smallest, it does not [arxiv:2101.12072 tab. 2]. These are author-reported
figures. The card has no timing comparison with the other models.
The code's default RNN is an LSTM, while the example notebook uses a GRU
[code: pts/model/time_grad/time_grad_estimator.py#L47-L49]
[code: examples/Time-Grad-Electricity.ipynb#L204].
Card: [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](/papers/2101.12072).

### Char-level LSTM and BiLSTM tagger with a language vector (cross-lingual NER)

Each word is encoded by a character LSTM, and its final state is concatenated
with a word embedding. A sentence-level BiLSTM reads the words, and a
conditional random field scores tag transitions and tag outputs
[arxiv:2404.09383 §2.3] [arxiv:2404.09383 eq. 4] [arxiv:2404.09383 eq. 5].
For transfer, the character LSTM, BiLSTM, tag layers and transition weights are
shared across languages, and a learned language vector conditions the scores
[arxiv:2404.09383 §3.1] [arxiv:2404.09383 eq. 6]. The training objective adds a
weighted source-language likelihood to the target-language likelihood
[arxiv:2404.09383 eq. 7].
With 100 target sentences, the neural model beats the log-linear CRF on every
listed transfer pair, by 3.97 to 9.80 F1. Without a source language it loses on
all five targets, by 0.75 to 8.45 F1 [arxiv:2404.09383 tab. 2]. With 10,000 target
sentences and no transfer, it beats the log-linear CRF on all five targets
[arxiv:2404.09383 tab. 2].
Card: [Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields](/papers/2404.09383).

### CNN features into one LSTM (LRCN, as surveyed)

Per-frame CNN features, from RGB frames and from optical flow, feed one LSTM
layer of 256 hidden units. The frame predictions are then averaged
[arxiv:1601.06615 §3.4.1] [arxiv:1601.06615 fig. 6]. The survey is a recipe
overview and runs no experiments, so it gives no accuracy for this model
[arxiv:1601.06615 §1]. The survey's toy example is a single-hidden-layer RNN
that outputs a running sum with unit weights and ReLU [arxiv:1601.06615 §3.4].
Card: [A Taxonomy of Deep Convolutional Neural Nets for Computer Vision](/papers/1601.06615).

### Shared LSTM with stance, detection and veracity heads

One LSTM stack reads each reply branch and is shared by three tasks: stance
for every tweet, rumour detection and veracity for the whole branch. Each task
then has its own dense layers and output [arxiv:1806.03713 §4.2]
[arxiv:1806.03713 fig. 3]. The official code adds a private LSTM layer in front
of the veracity and detection heads, which the paper does not describe
[code: MTL3_CV_VeracityStanceDetection.py#L30-L69]. On PHEME with five events,
the three-task model reaches macro-F 0.396. The single-task branchLSTM reaches
0.336 and NileTMRG* 0.339 [arxiv:1806.03713 tab. 3]. On RumourEval, the
veracity-plus-stance model reaches 0.558, against 0.491 for branchLSTM and
0.539 for NileTMRG* [arxiv:1806.03713 tab. 3]. These are author-reported single
runs with no variance.
Card: [All-in-one: Multi-task Learning for Rumour Verification](/papers/1806.03713).

### Backward plan GRU guiding a forward body GRU

A backward-running GRU starts from a random draw and the action label. It emits
one guide vector per future step. A forward body GRU reads the pose and the
guide at each step and predicts the next pose [arxiv:1809.03036 §3.2]
[arxiv:1809.03036 fig. 1]. Training adds an open-loop error to a closed-loop
error, and the closed-loop weight starts at zero and steps up, to limit drift
[arxiv:1809.03036 §3.4] [arxiv:1809.03036 eq. 6]. In the long-term NPSS
table, the lowest score on each of walking, eating and smoking belongs to one of
the two proposed models [arxiv:1809.03036 tab. 4]. In a user study, NPSS
agreed with the majority vote on 65 of 80 comparisons, against 39 of 80 for MSE
[arxiv:1809.03036 tab. 3]. These are author-reported results on Human3.6M.
Card: [A Neural Temporal Model for Human Motion Prediction](/papers/1809.03036).

### GRU encoders over a lane graph

Three one-layer GRUs, each with hidden size 32, encode the target's motion, the
other agents and the lane-graph nodes [arxiv:2106.15004 app. A.2]. The
recurrence is only the encoder: graph layers, agent-to-node attention and a
route-conditioned attention decoder follow [arxiv:2106.15004 §4.1]
[arxiv:2106.15004 §4.3]. On nuScenes the model reports MinADE10 1.00 and
MissRate10,2 0.37, author-reported [arxiv:2106.15004 tab. 1]. The ablations
in tab. 2 do not remove the GRU encoders, so their effect is not measured
[arxiv:2106.15004 tab. 2].
Card: [Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals](/papers/2106.15004).

### Bidirectional recurrent graph network with graph-gated GRUs (GRIL)

Each GRU gate is a message-passing network over a sensor graph. Each time step
has two imputation stages: a first guess from the hidden state, then a second
guess from neighbouring sensors. A forward and a backward unit are combined by
an MLP [arxiv:2108.00298 §4] [arxiv:2108.00298 eq. 11]. Against BRITS, the
author-reported MAE on PEMS-BAY with point missing values is 0.67 against 1.47,
and out-of-sample on AQI it is 14.73 against 20.21 [arxiv:2108.00298 tab. 2]
[arxiv:2108.00298 tab. 1]. The percentage reductions are computed on the card,
not reported by the paper.
Card: [Filling the G_ap_s](/papers/2108.00298).

### Spiking recurrent layer with learned delays (DelRec)

A leaky integrate-and-fire recurrent layer learns a real-valued delay per
neuron (axonal) or per connection (synaptic). It is trained with surrogate
gradients [arxiv:2509.24852 §2.1]. During training each spike is spread over
neighbouring steps by a triangular kernel, whose width is annealed towards zero.
Delays are rounded to integers at inference, except on AL [arxiv:2509.24852 §4.4].
Author-reported test accuracy: 82.42% on SSC and 85.74% on AL with axonal
delays, 96.63% on PS-MNIST with synaptic delays, and 83.53% on HAR with axonal
delays [arxiv:2509.24852 tab. 1] [arxiv:2509.24852 tab. 2]. The cell is spiking and has no gates, so it is not
an LSTM or GRU.
Card: [DelRec: learning delays in recurrent spiking neural networks](/papers/2509.24852).

### LSTM policy in a sequence GAN (SeqGAN)

The generator is an LSTM that picks each token as an action. A CNN
discriminator scores each finished sequence. Monte Carlo roll-outs turn that
score into a value for each earlier token, and the policy gradient uses it
[arxiv:1609.05473 § SeqGAN via Policy Gradient] [arxiv:1609.05473 eq. 4]. The
paper says GRU cells or attention could replace the LSTM [arxiv:1609.05473 §
The Generative Model for Sequences]. Cell sizes are not stated in the paper.
The code uses embedding and hidden sizes of 32 [code: sequence_gan.py#L14-L15].
On the synthetic oracle task it reports NLL 8.736, against 9.038 for MLE
[arxiv:1609.05473 tab. 1]. These are author-reported.
Card: [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](/papers/1609.05473).

### LSTM temporal integration in a driving pipeline

Sensor fusion and an attention glimpse feed a CNN. An LSTM-style gate update
then keeps a hidden state over time, so the planner can cope with what it
cannot see [arxiv:1704.02532 §4.2]. The planner is a DQN or DDAC head
[arxiv:1704.02532 §4.3]. The test is lane keeping in TORCS, and the results are
qualitative. The paper gives no measurement for the recurrent part
[arxiv:1704.02532 §5].
Card: [Deep Reinforcement Learning framework for Autonomous Driving](/papers/1704.02532).

### Entity-aware LSTM with a static input gate (EA-LSTM)

The static attributes of each basin drive only the input gate. The weather
inputs and the previous hidden state drive the other gates, and the cell and
hidden updates are standard [arxiv:1907.08456 §2.2] [arxiv:1907.08456 eq. 7-12].
The read-out is the last hidden state, with dropout 0.4 and one layer
[arxiv:1907.08456 app. B] [arxiv:1907.08456 §2.6]. On 447 validation basins the
single model has mean NSE 0.674 and the 8-model ensemble 0.705. The paper's
LSTM with static inputs concatenated scores 0.685 on the same set, so the
EA-LSTM is not the best row in that table [arxiv:1907.08456 tab. 3].
Card: [Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets](/papers/1907.08456).

### Three GRU states per conversation (DialogueRNN)

A global GRU tracks the conversation context. One party GRU per speaker tracks
that speaker's state, and an emotion GRU reads the speaker's updated party state
to label each utterance [arxiv:1811.00405 §3.3.1] [arxiv:1811.00405 §3.3.3]
[arxiv:1811.00405 §3.3.5]. Attention over earlier global states supplies
context [arxiv:1811.00405 §3.3.3]. The bidirectional variant needs the whole
dialogue [arxiv:1811.00405 §3.4.2]. On IEMOCAP text, the bidirectional model
with attention reaches 63.40 weighted accuracy and 62.75 weighted F1. The
unidirectional model reaches 59.33 and 59.89 [arxiv:1811.00405 tab. 2].
Card: [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](/papers/1811.00405).

### LSTM mixture-density model in a world model (MDN-RNN)

An LSTM with a mixture-density output predicts the next latent vector from the
current latent, the action and its own state [arxiv:1809.01999 §2]
[arxiv:1809.01999 app. A.3]. The mixture has 5 Gaussians per latent dimension,
and the hidden size is 256 for car racing and 512 for Doom [arxiv:1809.01999
app. A.3]. A linear controller reads the latent and the LSTM hidden state
[arxiv:1809.01999 app. A.4]. Sampling divides the mixture logits by a
temperature τ, which sets how noisy the imagined environment is
[arxiv:1809.01999 §2] [code: carracing/rnn/rnn.py#L81]. A controller trained only
inside the model scores 1092 ± 556 on the real DoomTakeCover task at τ = 1.15
[arxiv:1809.01999 tab. 2] [arxiv:1809.01999 §4.2]. The car racing full model
scores 906 ± 21 [arxiv:1809.01999 tab. 1]. These are author-reported.
Card: [Recurrent World Models Facilitate Policy Evolution](/papers/1809.01999).

### Diagonal linear recurrence with a parallel scan (S5)

S5 is a multi-input, multi-output diagonal state-space layer. It runs over time
as a linear recurrence, computed with a [parallel scan](/concepts/parallel-scan)
[arxiv:2208.04933 §2.2] [arxiv:2208.04933 §3.4]. The recurrence has no gates,
so it is not an LSTM or GRU. Its reported LRA average is 87.46%, against 86.09%
for S4-LegS, and 98.58% on Path-X [arxiv:2208.04933 tab. 1]. These are
author-reported.
Card: [Simplified State Space Layers for Sequence Modeling](/papers/2208.04933).

### Selective state space with a gated-RNN reading (Mamba)

Mamba makes the step size and the input and output matrices depend on each
token [arxiv:2312.00752 §3.2]. For one state dimension, fixed A and B, and a
linear step size, the update is a gated recurrence [arxiv:2312.00752 §3.5]
[arxiv:2312.00752 eq. 5]. The authors report 4 to 5 times the inference
throughput of a Transformer of similar size [arxiv:2312.00752 §4.5]. That
throughput test uses an untrained 6.9B model [arxiv:2312.00752 app. E.5].
Card: [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](/papers/2312.00752).

### Bidirectional recurrent propagation with feature warping (BasicVSR, IconVSR)

Two recurrent branches sweep a low-resolution clip in opposite directions and
do not exchange information. Each branch warps its carried state onto the
current frame with optical flow, refines it with 30 residual blocks, and an
upsampler combines the two final states [arxiv:2012.02181 §3.1]
[arxiv:2012.02181 app. A]. Warping features rather than pixels is the authors'
main alignment choice: image warping lost 0.17 dB in their test, and removing
alignment lost 1.19 dB [arxiv:2012.02181 §3.1]. IconVSR adds a keyframe
information-refill every five frames, and lets the backward state feed the
forward branch [arxiv:2012.02181 §3.2] [arxiv:2012.02181 app. A]. On REDS4
(BI, 4×), the authors report 31.42 dB for BasicVSR with 6.3M parameters and 63 ms,
against 31.09 dB, 20.6M and 378 ms for EDVR. IconVSR reports 31.67 dB with 8.7M
parameters [arxiv:2012.02181 tab. 2]. Inference runs on whole sequences, and
memory use for long videos is not reported [arxiv:2012.02181 app. A].
Card: [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](/papers/2012.02181).

### Second-order grid propagation with flow-guided deformable alignment (BasicVSR++)

BasicVSR++ keeps BasicVSR's bidirectional recurrence and changes two parts. The
propagation runs two rounds of backward and forward sweeps, and each step uses
the two previous states of its sweep [arxiv:2104.13371 §3.1]. The alignment uses
optical flow as the base of deformable offsets, and a small network learns only
a residue on top of the flow [arxiv:2104.13371 §3.2] [arxiv:2104.13371 eq. 6].
The authors report 32.39 dB on REDS4 (BI, 4×) with 7.3M parameters and 77 ms,
against 31.42 dB for BasicVSR with 6.3M and 63 ms [arxiv:2104.13371 tab. 1]. The
smaller BasicVSR++ (S) reaches 32.24 dB with 6.4M parameters and 69 ms
[arxiv:2104.13371 tab. 2]. In the ablation, whose dataset the caption does not name, flow-guided alignment
moves the baseline from 31.48 dB to 31.94 dB, and second-order propagation adds
a further 0.14 dB [arxiv:2104.13371 tab. 3]. The runtime is for 180 × 320 input
[arxiv:2104.13371 tab. 1]. The backward branch starts at the last frame, so
outputs depend on the whole clip [arxiv:2104.13371 §3.1].
Card: [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](/papers/2104.13371).

### Spatiotemporal memory flow with decoupled memories (PredRNN)

Each unit keeps a temporal memory `C` that runs along time within its layer, and
a spatiotemporal memory `M` that runs up from the layer below. Each has its own
gates, and the output gate reads both new memories [arxiv:2103.09504 §4.2]
[arxiv:2103.09504 eq. 4]. A penalty on the absolute cosine similarity of the two
memory increments is used in training only, and the authors say it adds no
parameters at inference [arxiv:2103.09504 §4.2] [arxiv:2103.09504 eq. 5].
Reverse scheduled sampling raises the chance that the encoder sees true frames
over training. The best run uses an exponential schedule from 0.5 to 1.0
[arxiv:2103.09504 §4.4] [arxiv:2103.09504 tab. 6]. On fixed Moving MNIST, the
authors report MSE 48.4 for PredRNN-V2 against 103.3 for ConvLSTM
[arxiv:2103.09504 tab. 2]. Adding decoupling and RSS lowers PredRNN from 56.8 to
48.4, and the authors apply the two techniques to several other backbones
[arxiv:2103.09504 tab. 5]. PredRNN-V2 has 23.86 MB of parameters, against 16.60
MB for ConvLSTM [arxiv:2103.09504 tab. 2].
Card: [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](/papers/2103.09504).

### GRU with k-means class features (DCGRU)

Each reading is paired with class features from a k-means clustering of the
history. The paper picks m = 4 classes from an elbow plot [arxiv:2512.06714 §3.3]
[arxiv:2512.06714 §4.1]. Each row passes through three time-distributed dense
layers, with sizes 5, 50, 10 and 1. A 32-unit GRU reads the 96 rows, and a
one-unit GRU cell gives the output [arxiv:2512.06714 §3.4] [arxiv:2512.06714 tab. 5].
The EDCGRU variant inserts one virtual value between readings, which doubles the
window to 192 rows [arxiv:2512.06714 §3.5] [arxiv:2512.06714 §4.2]. On two
district metering areas, the one-step MAPE is 1.26% and 1.31% for DCGRU. It is
2.49% and 3.58% for the baseline BGRU, which the paper describes as the same GRU
block without class features [arxiv:2512.06714 §3.7.1] [arxiv:2512.06714 tab. 6].
GRUN, the other baseline, scores 2.06% and 2.46% [arxiv:2512.06714 tab. 6]. The
DCGRU model has 4,187 parameters, against 30,403 for GRUN
[arxiv:2512.06714 tab. 6]. All numbers come from one data set, with training and
testing on the same DMA [arxiv:2512.06714 §2].
Card: [A Novel Deep Neural Network Architecture for Real-Time Water Demand Forecasting](/papers/2512.06714).

### LSTM generator and bidirectional LSTM discriminator (C-RNN-GAN)

The generator is a two-layer unidirectional LSTM. Each step takes a random
vector concatenated with the previous output. The discriminator is a two-layer
bidirectional LSTM that scores a whole sequence as real or generated
[arxiv:1611.09904 §2] [arxiv:1611.09904 §3]. The paper sets 350 units per
layer for both networks. Training starts with six epochs of next-event
prediction, then adversarial training with feature matching, and a rule that
pauses the discriminator when its loss falls below 70% of the generator's
[arxiv:1611.09904 §3]. Results are curves only. The author reports more varied
tones than a maximum-likelihood baseline [arxiv:1611.09904 §4]
[arxiv:1611.09904 fig. 3a–b].
Card: [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](/papers/1611.09904).

### Word-plus-character recurrent tagger with affix embeddings (NER survey)

Each word vector joins a word embedding with a recurrent summary of the word's
characters. A sentence-level recurrent layer then reads the joined vectors. The
affix variant adds learned embeddings of frequent prefixes and suffixes
[arxiv:1910.11470 §6.4.3] [arxiv:1910.11470 §6.4.4]. The survey copies these
numbers from Yadav et al., with no experiments of its own. The best author-reported
CoNLL F1 is 87.26 in Spanish, 87.54 in Dutch and 79.01 in German, at 150 epochs,
and 90.86 in English, which is below two other English rows in the same table [arxiv:1910.11470 tab. 1].
The layer widths and the joining operation are not stated
[arxiv:1910.11470 §6.4.4].
Card: [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](/papers/1910.11470).

### GRU in a factorized or coupled spatio-temporal block (urban graph survey)

In the factorized design, a graph convolution runs first and a GRU then reads
its output at each time step [arxiv:2303.14483 §5.5.1]. In the coupled design,
the graph convolution replaces the linear maps inside the GRU gates, as in
DCRNN [arxiv:2303.14483 §5.5.2]. The survey gives both designs as equations and
runs no experiments, so it does not rank them [arxiv:2303.14483 §1].
Card: [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](/papers/2303.14483).

### LSTM or CNN encoder with a history window (sequential short-text classification)

Each short text is turned into one vector, by an LSTM with max pooling over
its hidden states or by a CNN with max pooling over its windows. A two-layer
head then reads the vectors of the current text and a few earlier ones
[arxiv:1603.03827 §2.1] [arxiv:1603.03827 §2.2]. Author-reported best
accuracies are 66.2% on DSTC 4 (LSTM), 84.6% on MRDA (CNN) and 73.1% on SwDA
(CNN) [arxiv:1603.03827 tab. 4]. Raising the history of the first layer helps
more than raising the second, and using history at both layers "does not help
in most cases" [arxiv:1603.03827 §4]. Card: [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](/papers/1603.03827).

### Recurrent loop with back-projection (RBPN)

For each neighbouring frame, RBPN estimates the optical flow to the target
frame and passes it through a loop that updates a high-resolution feature
estimate. The features from all neighbours are concatenated and one
convolution reconstructs the frame [arxiv:1903.10128 §3]. The recurrent state
is a feature map, not a gated cell. In the paper's ablation, swapping the
concatenation for a convolutional LSTM lowered PSNR on SPMCS-32 from 31.64 dB
to 31.46 dB [arxiv:1903.10128 tab. 3]. Author-reported 4x result: 30.10 dB on
SPMCS-11 for RBPN/6-PF, against 29.42 dB for VSR-DUF [arxiv:1903.10128 tab. 8].
Card: [Recurrent Back-Projection Network for Video Super-Resolution](/papers/1903.10128).

### GRU memory in an end-to-end mobile-manipulation policy (HarmonicMM)

The policy takes the joint embedding of two frozen-backbone camera views and
the arm proprioception, then passes it through one GRU layer with 512 units.
Linear heads read the hidden state for actions and value [arxiv:2312.06639
app. Network Architecture]. The GRU is one component of the policy, and the paper's claims are about the
whole policy. The ablations it reports remove the proprioception and the visual
backbone, not the GRU, so the recurrent layer's effect is not isolated
[arxiv:2312.06639 tab. II] [arxiv:2312.06639 tab. IV]. The author-reported
average success is 47% over four simulated tasks in unseen houses, 17.6 points
above the ReLMoGen adaptation [arxiv:2312.06639 §V-B] [arxiv:2312.06639 tab. I].
Card: [Harmonic Mobile Manipulation](/papers/2312.06639).

### Bi-GRU encoders with attention over attention (AoA Reader)

Cloze-style reading comprehension. A shared embedding feeds one bi-directional
GRU for the passage and one for the question. A passage-by-question matching
matrix is normalised twice: over passage positions for each question word, and
over question positions for each passage word. The second map is averaged over
the passage to give one weight per question word, which weights the first
maps. The result is summed per vocabulary word [arxiv:1607.04423 §3]
[arxiv:1607.04423 eq. 6-11]. The authors say this adds a few steps to the AS
Reader and no weights [arxiv:1607.04423 §3]. On the CBTest NE test split the
single model scores 72.0 and the version with 5-best re-ranking scores 74.0,
against 68.6 for the AS Reader [arxiv:1607.04423 tab. 2]. Re-ranking uses
three language-model features, one of them trained per test sample
[arxiv:1607.04423 §4]. Results are author-reported, and no official code was
found [arxiv:1607.04423 Paper vs code].
Card: [Attention-over-Attention Neural Networks for Reading Comprehension](/papers/1607.04423).

### Two-stream LSTM over skeleton joints

Skeleton-based action recognition. A temporal stream reads per-frame joint
coordinates, either as a stack of two LSTM layers or as five body-part LSTMs
feeding a whole-body LSTM [arxiv:1704.02581 §4.1]. A spatial stream reads the
joints as one ordered sequence, each step holding one joint's coordinates over a
time window, and the order is either a chain or a graph traversal
[arxiv:1704.02581 §4.2]. The two streams' class posteriors are fused, and the
paper does not write the fusion formula out [arxiv:1704.02581 §4]. Random
rotation, scaling and shear, applied only during training, add 2.7 points on
NTU cross-subject and 7.8 on cross-view [arxiv:1704.02581 tab. 1]. The spatial
stream alone scores 55.2 and 60.5 on NTU, against 67.8 and 70.5 for the
hierarchical temporal stream [arxiv:1704.02581 tab. 1]. The full model reports
71.3 and 79.5 on NTU, 94.8 on SBU and 91.7 F1 on ChaLearn [arxiv:1704.02581
tab. 3] [arxiv:1704.02581 tab. 4] [arxiv:1704.02581 tab. 5]. These are
author-reported, and the card finds no official code.
Card: [Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks](/papers/1704.02581).

### ConvGRU fusion across key-frame fragments (NeuralRecon)

Real-time 3D reconstruction from monocular video. Each fragment of nine key
frames is decoded into one sparse signed-distance volume in a single pass
[arxiv:2104.00681 §3]. A ConvGRU, written with sparse convolutions, combines the
fragment's features with a hidden state held over the global volume, so later
fragments update earlier geometry [arxiv:2104.00681 §3.2 GRU Fusion]. With five
views and the same occupancy area, the GRU reaches an F-score of 0.488, against
0.462 for a running-average linear fusion [arxiv:2104.00681 tab. 5]. The
released code adds a pointwise linear term inside each sparse convolution that
the paper's four equations do not show [arxiv:2104.00681 Paper vs code].
Card: [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](/papers/2104.00681).

### GRU context and fusion GRUs in a multimodal sentiment model (HFusion)

Each modality has its own GRU, which runs over the utterances of a clip, so
every utterance vector carries context from the clip [arxiv:1806.06228 §3.3].
After each pair of modalities is fused, a GRU with output size 500 reads the
pair sequence. A GRU with output size 550 then reads the trimodal fusion
[arxiv:1806.06228 §3.4]. The unimodal GRU width is not stated
[arxiv:1806.06228 §3.3]. A footnote says an LSTM "does not perform well" as
the context model, and gives no number [arxiv:1806.06228 §3.3]. With all three
modalities and the paper's own features on CMU-MOSI, the model reaches 77.9%
accuracy, against 77.0% for early fusion and 77.3% for the second baseline
[arxiv:1806.06228 tab. 2]. These are author-reported.
Card: [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](/papers/1806.06228).

### LSTM on raw satellite series (crop types)

The LSTM is the authors' earlier crop-type model. It reads each parcel's
series, one vector of 13 spectral bands per date [arxiv:1910.10536 §4.1]
[arxiv:1910.10536 §7.2.1]. On raw data, the 12-class accuracy is 0.90 for the
LSTM, against 0.87 for MS-ResNet and 0.83 for TempCNN. On 23 classes, the raw
kappa is 0.71 for both the LSTM and the Transformer [arxiv:1910.10536 tab. 1].
The paper calls the RNN and the Transformer slightly better than the
convolutional networks on raw data [arxiv:1910.10536 §8.1]. Preprocessing raised
every model's score, so the LSTM result is for raw data only
[arxiv:1910.10536 §8.1].
Card: [Self-attention for raw optical Satellite Time Series Classification](/papers/1910.10536).

## When to use which

The paper gives no single answer for forecasting. Its own conclusion depends on how accurate the
forecasts must be, how fast they must come, and how much tuning is affordable
[arxiv:2103.12057 §4.3]. The table below uses the constraints the cited papers
name or measure.

| Situation | Use | Why |
|---|---|---|
| Accuracy matters most and you can tune a grid | LSTM | Won the most datasets on best-of-grid WAPE [arxiv:2103.12057 tab. 11] |
| Accuracy matters and you want a stable choice across settings | Compare the mean and spread, not only the best cell | The paper ranks by mean and spread and finds CNN and LSTM the most stable [arxiv:2103.12057 §4.1] [arxiv:2103.12057 fig. 6] |
| Training time and inference time are tight | Consider a CNN before a recurrent model | CNN is the most efficient deep model in the paper's rankings [arxiv:2103.12057 §4.2] |
| Inference latency is tight | Avoid ESN | The paper reports slow inference for ESN [arxiv:2103.12057 §4.2] |
| Little tuning budget | Fix the input window at 1.25 × H and a learning rate of 0.001 | Both were the best grid values for every architecture [arxiv:2103.12057 tab. 18] |
| Long inputs where the recurrent loop is the bottleneck | Attention pooling over per-step features | Faster per epoch than a vanilla RNN on one toy task [arxiv:1512.08756 §2.2]. Tested on toy tasks only |
| Image sequences from one sensor, such as satellite series | A bidirectional ConvGRU encoder | The authors' main model, trained with 256 cells [arxiv:1802.02080 §5.2] |
| Read text from images and decode it character by character | A bidirectional LSTM encoder with an attention decoder | Best lexicon-free result on several irregular benchmarks, against the authors' own controls and copied baselines [arxiv:1711.04226 §5.3] |
| Learn from a stream in one pass, with no stored training set and no gradients | HTM sequence memory | Author-reported taxi error comparable to LSTM, but LSTM is slightly better under temporal noise [arxiv:1512.05463 §4.5] [arxiv:1512.05463 §5] |
| Adversarial motion generation with an LSTM core | LSTM over GRU | The authors chose LSTM over GRU empirically [arxiv:2102.04942 §5.5] |
| Gated recurrence, with no evidence for your task that favours LSTM or GRU | Either gated cell, over a simple RNN | On the survey's music and speech tasks both beat the simple RNN, and neither is clearly better. Its comparisons are not on NLP tasks [arxiv:1708.02709 §4.2] |
| Demand on a network of stations, where links cannot be defined by hand | A graph filter learned from data, with an LSTM block over time | Lowest test RMSE of 13 models on one author-reported dataset [arxiv:1712.04997 tab. 3]. The recurrent variant has no public code [arxiv:1712.04997 Paper vs code] |
| Encode SMILES strings or protein sequences inside a drug-target library | A CNN with a bidirectional GRU or LSTM on top, as one encoder among several | The paper offers it as an option. The card's results do not isolate it [arxiv:2004.08919 app. A] |
| Probabilistic forecasts of many correlated series, with future covariates known | An LSTM encoder with a conditional diffusion head (TimeGrad) | Best CRPS_sum on five of six datasets, author-reported [arxiv:2101.12072 tab. 2]. Sampling draws 100 paths with 100 reverse steps per forecast step [arxiv:2101.12072 §4.2], so if latency is tight, the paper's ablation that cuts steps to about 10 is the starting point [arxiv:2101.12072 §4.4] |
| Track an unknown number of pedestrians online, frame by frame, and speed matters more than accuracy | An RNN motion model with LSTM association | 165.2 FPS on the MOTChallenge 2015 test set, against 19.0% MOTA, which the authors call not competitive on accuracy [arxiv:1604.03635 tab. 2] |
| Group pixels into objects with no labels, on synthetic scenes, and predict the next frame | RNN-EM | AMI 0.826 against 0.475 for N-EM on static shapes [arxiv:1708.03498 §4.1]. RNN-EM gives up EM's convergence guarantee [arxiv:1708.03498 §2.3] |
| Low-resource NER with a related high-resource language, where 100 target sentences are labelled | A neural CRF with a shared char-LSTM and BiLSTM, plus a language vector | Beats the log-linear CRF on every listed transfer pair, by 3.97 to 9.80 F1 [arxiv:2404.09383 tab. 2] |
| Low-resource NER with no related source language | The log-linear CRF | The monolingual neural CRF loses on all five low-resource targets, by 0.75 to 8.45 F1 [arxiv:2404.09383 tab. 2] |
| Several related labels on the same stream, with labels present for only some items | A shared LSTM with one head per task, and per-item loss masks | Macro-F 0.396 against 0.336 for the single-task branchLSTM on PHEME with five events [arxiv:1806.03713 tab. 3]. The paper's loss masks missing labels, and the code down-weights them to almost zero [arxiv:1806.03713 §4.2] |
| Long human motion generated in closed loop, where errors drift | A GRU pair with a guide vector and a closed-loop loss weight that ramps up | Best long-term NPSS on walking, eating and smoking, author-reported [arxiv:1809.03036 tab. 4]. Each batch costs two forward passes [arxiv:1809.03036 §3.4] |
| Vehicle trajectories on a lane graph, with several modes to output | GRU encoders over the lane graph inside a route-based model | MinADE10 1.00 on nuScenes, author-reported [arxiv:2106.15004 tab. 1]. The GRUs are one part of the model, not a tested choice on their own |
| Utterance sequences with context, where each modality needs its own sequence model before fusion | Per-modality GRU context, then pairwise and trimodal GRU fusion (HFusion) | 77.9% accuracy with all three modalities, against 77.0% for early fusion on CMU-MOSI, author-reported [arxiv:1806.06228 tab. 2]. The code trains the pair models separately, not end to end [arxiv:1806.06228 Paper vs code] |
| Raw Sentinel-2 series, where cloud masking and gap filling are not feasible | An LSTM over the raw series, which is among the top two models on raw data | 0.90 accuracy on 12 classes, against 0.87 for MS-ResNet and 0.83 for TempCNN, author-reported [arxiv:1910.10536 tab. 1]. Preprocessing raised every model's score [arxiv:1910.10536 §8.1] |
| Sensors with gaps, a known or computable sensor graph, and an offline setting | A bidirectional GRU whose gates are graph message passing (GRIL) | PEMS-BAY point missing MAE 0.67 against 1.47 for BRITS, author-reported [arxiv:2108.00298 tab. 2]. The released code uses dense N×N supports and has no license file [arxiv:2108.00298 Paper vs code] |
| Neuromorphic hardware with programmable integer delays | A spiking recurrent layer with learned delays (DelRec) | 82.42% on SSC and 96.63% on PS-MNIST, author-reported [arxiv:2509.24852 tab. 1]. AL loses accuracy when its delays are rounded [arxiv:2509.24852 §4.4] |
| Discrete tokens scored only as a whole sequence, with no reference to compare against | An LSTM policy trained by policy gradient, with roll-outs (SeqGAN) | Synthetic oracle NLL 8.736 against 9.038 for MLE, author-reported [arxiv:1609.05473 tab. 1]. The code runs 304 partial roll-outs per 64-sequence step at T = 20 and N = 16 [code: rollout.py#L77-L103] |
| Many basins or sites share one model, and each has fixed attributes | EA-LSTM, with the attributes in the input gate | Mean NSE 0.674 for one model and 0.705 for an 8-model ensemble, author-reported. The LSTM with static inputs scores 0.685 on the same set [arxiv:1907.08456 tab. 3] |
| Utterance-level emotion labels in a conversation, where only past turns are available | Unidirectional DialogueRNN | 59.33 weighted accuracy on IEMOCAP text. The bidirectional and attention variants need the whole dialogue [arxiv:1811.00405 tab. 2] [arxiv:1811.00405 §3.4.2] |
| Learn a predictive model and a small controller, and test the controller in the real environment | MDN-RNN world model with a linear controller | 1092 ± 556 on the real Doom task at τ = 1.15, author-reported. At τ = 0.10 the real score falls to 193 [arxiv:1809.01999 tab. 2] |
| Long sequences where training should run in parallel over time, with a linear recurrence | Diagonal state-space layer with a parallel scan (S5) | LRA average 87.46% against 86.09% for S4-LegS, author-reported [arxiv:2208.04933 tab. 1] |
| Long-context language modelling where inference speed matters | Selective state space (Mamba) | 4 to 5 times the inference throughput of a similar Transformer, author-reported [arxiv:2312.00752 §4.5]. The throughput test uses untrained weights [arxiv:2312.00752 app. E.5] |
| Offline video restoration, where the whole clip is available and each frame should use both past and future frames | Bidirectional recurrent propagation with flow warping (BasicVSR) | 31.42 dB on REDS4 (BI) at 63 ms, against 31.09 dB and 378 ms for EDVR, author-reported [arxiv:2012.02181 tab. 2]. The backward sweep needs future frames, so it cannot run on a live stream |
| Future-frame video prediction where the best reported MSE matters and a 23.86 MB model is acceptable | PredRNN-V2 with decoupled memories and RSS | MSE 48.4 on fixed Moving MNIST, against 103.3 for ConvLSTM, author-reported [arxiv:2103.09504 tab. 2]. The repository has loaders for Moving MNIST, KTH and BAIR only [arxiv:2103.09504 Paper vs code] |
| Short-term forecast of one meter series, CPU only, with a small parameter budget | DCGRU with k-means class features | 4,187 parameters, against 30,403 for GRUN. One-step MAPE 1.26% and 1.31% against 2.06% and 2.46% [arxiv:2512.06714 tab. 6]. One data set only [arxiv:2512.06714 §2] |
| Same series, where sharp jumps matter more than training time | EDCGRU with one virtual value per gap | One-step MAPE 0.98% and 0.99%, with training of 850 s and 1,055 s on CPU [arxiv:2512.06714 tab. 6]. The method for making virtual values is not stated [arxiv:2512.06714 §3.5] |
| Label each short text in an ordered sequence, such as dialog acts, with the earlier texts available as context | A sequential classifier over LSTM or CNN text vectors, with a history window | 66.2% on DSTC 4 (LSTM), 84.6% on MRDA (CNN) and 73.1% on SwDA (CNN), author-reported [arxiv:1603.03827 tab. 4]. Learning rate, batch size and epochs are not stated [arxiv:1603.03827 Hyperparameters] |
| Continuous event sequences, such as note timing and loudness, where a GAN is wanted | C-RNN-GAN, with an LSTM generator and a bidirectional LSTM discriminator | Variety gains over a maximum-likelihood baseline, shown only as curves [arxiv:1611.09904 fig. 3a–b]. The paper says discrete-symbol music GANs need policy gradients [arxiv:1611.09904 §1] |
| Named entity tagging without lexicons, where a neural baseline is wanted | Word-plus-character recurrent tagger with learned affix embeddings | Best author-reported CoNLL F1 in three of four languages, copied from one cited paper [arxiv:1910.11470 tab. 1]. The affix rows use no dictionary [arxiv:1910.11470 tab. 1] |
| Language modelling where per-token decoding must stay at constant memory, and a parallel training form is affordable | xLSTM with mLSTM blocks (xLSTM[1:0]) | 13.43 validation perplexity at 15B SlimPajama tokens, against 13.70 for Mamba, author-reported [arxiv:2405.04517 tab. 1]. Training uses a parallel form that the code builds as T × T matrices per head [arxiv:2405.04517 Pitfalls] |
| Offline video super-resolution where PSNR matters most, and a flow network and two propagation rounds fit the budget | BasicVSR++, or BasicVSR++ (S) for a smaller model | 32.39 dB on REDS4 (BI) at 77 ms, and 32.24 dB at 69 ms for (S), against 31.42 dB at 63 ms for BasicVSR, author-reported [arxiv:2104.13371 tab. 1] [arxiv:2104.13371 tab. 2]. The flow network's parameters and runtime are counted in these figures [arxiv:2104.13371 §4] |
| Video super-resolution at 4x, where each neighbouring frame should be used one at a time with its optical flow to the target | RBPN, a recurrent loop with one single-image upscaler call per neighbour | 30.10 dB on SPMCS-11, against 29.42 dB for VSR-DUF, author-reported [arxiv:1903.10128 tab. 8]. The loop needs flow estimation and one upscaler call per neighbour [arxiv:1903.10128 §3] |
| Sensor graphs with time series, where the fusion design is the open choice | Factorized GCN-then-GRU, or a coupled GRU with graph gates | Both are described as equations, with no experiments in the survey [arxiv:2303.14483 §5.5] [arxiv:2303.14483 §1] |
| Mobile manipulation from two RGB views and arm proprioception, trained end to end with RL, where base and arm must move together | A single 512-unit GRU after the joint image and proprioception embedding, in a PPO policy (HarmonicMM) | 47% average success over four simulated tasks, 17.6 points above ReLMoGen, author-reported [arxiv:2312.06639 §V-B]. Real-robot trials were run in one apartment only [arxiv:2312.06639 tab. III] |
| Cloze-style reading comprehension where the answer is one word from the passage, and a bi-GRU encoder fits the budget | Attention-over-attention pooling (AoA Reader) | 72.0 on CBTest NE test for one model, and 74.0 with 5-best re-ranking, against 68.6 for AS Reader, author-reported [arxiv:1607.04423 tab. 2]. Re-ranking needs a language model trained on each test sample's document [arxiv:1607.04423 §4] |
| Skeleton action recognition where the test camera angle or body size may differ from training | Two-stream LSTM with rotation, scaling and shear augmentation | 71.3 and 79.5 on NTU cross-subject and cross-view, against 69.2 and 77.7 for trust-gate ST-LSTM, author-reported [arxiv:1704.02581 tab. 3]. The augmentation alone adds 7.8 points on cross-view [arxiv:1704.02581 tab. 1] |
| Real-time reconstruction from ordered monocular frames, where per-fragment output is enough and frames must agree with each other | ConvGRU fusion across fragments (NeuralRecon) | F-score 0.562 on the ScanNet Atlas split at 30 ms, author-reported [arxiv:2104.00681 tab. 1]. The GRU fusion beats a running average at equal views and area [arxiv:2104.00681 tab. 5] |

The evidence supports these rows only for the tasks and grids in each cited
paper. None of the papers tests very long sequences with a tuned recurrent
baseline, and the forecasting paper does not test multivariate series or online
learning.

## Implementation sketch

The recurrent forecaster in the paper is a stack of recurrent layers followed
by one dense output with one neuron per horizon step [arxiv:2103.12057 §3.2.1].
The cell type, layer count, unit count and return-sequence flag are the
hyperparameters to grid over [arxiv:2103.12057 tab. 9]. Pseudocode in our own
words:

```pseudocode
# Recurrent forecaster for one window of past values  [arxiv:2103.12057 §3.2.1]
def recurrent_forecaster(cell, layers, units, return_sequences, horizon, window):
    x = window  # shape (past_history, 1)
    for index in range(layers):
        is_last = index == layers - 1
        # Earlier layers must pass the full sequence to the next layer
        # [code: experiments/models.py#L40-L49]
        keep_sequence = return_sequences if is_last else True
        x = cell(units, return_sequences=keep_sequence)(x)
    if return_sequences:
        # Flatten the sequence of hidden states before the output
        # [code: experiments/models.py#L54-L55]
        x = flatten(x)
    # One output per horizon step  [arxiv:2103.12057 §3.2.1]
    return dense(horizon)(x)
```

Training uses Adam and a mean absolute error loss on normalized targets in the
shipped code. The paper does not name the loss [arxiv:2103.12057 §3.3]
[code: experiments/main.py#L171-L177]. Normalize each series with its own
training statistics, then undo the scaling before scoring [arxiv:2103.12057 §3.3].

## Contradictions

- **Layer count.** The text says fewer stacked layers did better within the
  recurrent models, except for ESN [arxiv:2103.12057 §4.4.2]. The selection
  table marks the two-layer LSTM as significantly best, and gives both one and
  two layers for GRU as significantly best [arxiv:2103.12057 tab. 17]. The paper
  does not reconcile the two.
- **Best cell versus mean.** The paper's headline best cells favour LSTM, while
  its mean across configurations favours CNN [arxiv:2103.12057 §4.1]. Both
  statements come from the same study, so they answer different questions.
- **LSTM or GRU.** The motion paper picks LSTM over GRU on its own empirical
  test [arxiv:2102.04942 §5.5]. The forecasting paper finds no significant
  difference between the best LSTM and GRU cells [arxiv:2103.12057 §4.3]. The
  tasks differ, and neither paper resolves the other's result. The NLP survey
  also finds neither gated cell clearly better, on music and speech
  [arxiv:1708.02709 §4.2].
- **LSTM in the sentiment paper.** Its footnote says an LSTM context model
  "does not perform well", yet the state-of-the-art baseline it reports is an
  LSTM context model [arxiv:1806.06228 §3.3] [arxiv:1806.06228 §4.2]. The
  paper gives no number for the footnote claim, so the two statements are not
  measured against each other.
- **Number of clusters.** The text of the clustering paper uses K = 3 for static
  shapes, while its appendix and code use K = 4 [arxiv:1708.03498 §4.1]
  [arxiv:1708.03498 app. A.1].
- **CNN or LSTM on sequential short-text classification.** The paper's grid
  caption says the CNN beats the LSTM on every dataset. On DSTC 4 the grid
  averages tie at 65.8, and the headline table gives the LSTM 66.2 against 65.5
  for the CNN [arxiv:1603.03827 tab. 3] [arxiv:1603.03827 tab. 4]. The headline
  runs are single runs and the grid values are averages, so the two tables do
  not compare like with like [arxiv:1603.03827 tab. 3] [arxiv:1603.03827 tab. 4].

## Pitfalls

- **Selection on the evaluation split.** The paper reports the best WAPE over
  each configuration grid, and the shipped code passes the test windows as
  validation data [arxiv:2103.12057 §4.4] [code: experiments/main.py#L190]. The
  best-cell numbers are therefore optimistic. Compare cells by mean and spread
  [arxiv:2103.12057 §4.1].
- **Single runs.** The paper says each configuration was trained several times
  [arxiv:2103.12057 §4.4], but the code runs each one once and sets no random
  seed [code: experiments/main.py#L369-L402]. Exact numbers may not reproduce.
- **Timing mixes hardware.** The paper used mixed GPUs, and the code selects a
  GPU by index. Timing comparisons between cells may mix hardware
  [arxiv:2103.12057 §4 intro] [code: experiments/main.py#L145-L152].
- **Few series.** Each series contributes one test window, so datasets with few
  series give noisy rankings [arxiv:2103.12057 §3.3] [arxiv:2103.12057 tab. 8].
- **Sizes left out.** The motion paper gives no LSTM hidden size or depth
  [arxiv:2102.04942 §3.2]. The text-recognition paper gives no encoder hidden
  size [arxiv:1711.04226 §4.1]. Take these from the code or choose them yourself,
  and say which.
- **Layers and normalisation only in code.** The dialogue code runs a
  bidirectional GRU whose output it discards, and it applies batch normalisation
  on the IEMOCAP setting only [code: COSMIC/erc-training/commonsense_model.py#L239]
  [code: COSMIC/erc-training/commonsense_model.py#L228-L231]. The paper describes
  neither [arxiv:2010.02795 §3.4]. Dropout also differs by dataset in the code
  [code: COSMIC/erc-training/train_iemocap.py#L115-L116].
- **Survey numbers.** The survey's score tables are copied from works that used
  different setups and do not rank the recurrent models
  [arxiv:2003.01200 tab. 2].
- **Gate equations with typos in the NLP survey.** Its LSTM candidate term uses
  an undefined capital X, which should be the concatenated vector
  [h_{t-1}; x_t]. Its GRU candidate uses the update gate's weight U_z, where a
  separate candidate weight is expected. Check the original GRU paper before you
  implement either [arxiv:1708.02709 §4.2].
- **Recurrent variant without code.** The bike-demand paper's best model is
  the recurrent one, and its public repository implements only the regular
  variant. Implement it from the text and Table 2. The gate equations come
  from outside the paper, which only refers to earlier work [arxiv:1712.04997 Paper vs code] [arxiv:1712.04997 §3.3.2].
- **Random RNN initial state.** DeepPurpose's CNN+RNN encoders start the RNN
  from a random normal hidden state, not zeros. Outputs then depend on the
  random number state, so fix the seed if you need repeatable predictions
  [code: DeepPurpose/encoders.py#L194-L211].
- **Silent truncation.** Those CNN+RNN encoders cut SMILES strings to 100
  characters and protein sequences to 1000 residues. The paper does not
  mention this [code: DeepPurpose/utils.py#L1091-L1112] [arxiv:2004.08919 app. A].
- **Tracking loss sign.** The paper's existence term is written as a
  log-likelihood without a minus sign. It has to be negated before it is
  minimised [arxiv:1604.03635 eq. 4].
- **Tracking network sizes.** The paper gives 300 hidden units for the motion
  RNN and 500 for the association LSTM. The shipped configs use 50 for both
  [arxiv:1604.03635 § Network size] [code: config/configBF.txt#L2-L3]
  [code: config/configDA.txt#L2-L3]. Choose the size and say which.
- **Clustering loss sign.** The sign of the inter-cluster term in the paper's
  loss (eq. 5) differs from the code, which adds it [arxiv:1708.03498 eq. 5]
  [code: nem_model.py#L253].
- **Forecast loss window.** The paper sums the TimeGrad loss over the prediction
  window only. The code computes the RNN, the target and the loss over the
  context plus the prediction [arxiv:2101.12072 §3.1]
  [code: pts/model/time_grad/time_grad_network.py#L375-L394].
- **Tagger sizes are ambiguous, and there is no code.** The sentence BiLSTM is
  given as 2 layers of 200 units each, and it is unclear whether that is per
  direction [arxiv:2404.09383 §2.3 fn. 1]. The character LSTM size is not
  stated. No official code is published, so these values must be chosen and
  reported [arxiv:2404.09383 Paper vs code].
- **Test-set selection in DelRec.** The HAR setting is picked by a sweep that
  maximises test accuracy. AL has no validation split, so its number is the best
  checkpoint over the training epochs [arxiv:2509.24852 §4.4]. Both numbers are
  optimistic.
- **DelRec delay settings.** The text gives a delay learning rate of 1e-1 for
  Mackey-Glass, but the config uses 5e-2 [arxiv:2509.24852 §4.3]
  [code: configs/perf_MG.py#L120]. Sigma also never reaches zero in training:
  the last epoch leaves about 0.006 of its initial value on SSC
  [arxiv:2509.24852 eq. 13]. Evaluation still sets it to zero
  [code: experiments/train.py#L224-L226].
- **Motion model: derivatives and loss terms.** The paper appends finite
  differences of orders 1 to 3, but the released code computes none
  [arxiv:1809.03036 §3.3] [code: motion_rnn_lm_v2.py#L166-L171]. The paper's
  eq. 5 also lacks the alternating sign of a finite difference
  [arxiv:1809.03036 eq. 5]. The code averages the open-loop loss over observed
  and future frames, while the paper uses future frames only
  [code: motion_rnn_lm_v2.py#L266-L268] [arxiv:1809.03036 §3.4].
- **Rumour model: unused learning rate and dense activations.** The code
  searches learning rates 1e-4 and 1e-3, but never passes them to the model, so
  Adam's default applies [code: parameter_search.py#L23]. The paper says the
  dense layers use ReLU, while the code's dense layers have no activation
  [arxiv:1806.03713 §4.1] [code: MTL3_CV_VeracityStanceDetection.py#L39-L64].
- **Code without a license.** The GRIN repository has no LICENSE file. Do not
  copy its code into a page. Cite it by path and implement from the paper's
  equations [arxiv:2108.00298 Paper vs code].
- **Trajectory clustering at test time.** The paper clusters 200 samples into
  K = 10 during training. It is unclear whether test-time prediction clusters
  too, and the number of rollouts per instance is not stated
  [arxiv:2106.15004 app. A.6] [arxiv:2106.15004 §4.2].
- **Roll-outs and no baseline in SeqGAN.** The reward enters the policy
  gradient with no baseline subtracted, and the paper does not discuss the
  variance this causes [code: sequence_gan.py#L146-L147]. Count the roll-out
  cost before comparing wall-clock times [code: rollout.py#L77-L103].
- **No license on the SeqGAN code.** The repository has no LICENSE file. Do not
  copy its code into a page. Cite it by path and implement from the paper
  [arxiv:1609.05473 Paper vs code].
- **Temperature is the exploit knob in the world model.** At τ = 0.10 the
  imagined game scores 2086, but the real score is 193 [arxiv:1809.01999
  tab. 2]. Choose τ by the real score. The paper's Doom controller uses
  τ = 1.15, while the code's default is 1.25 [code: doomrnn/doomrnn.py#L24].
- **Epoch counts differ in the world model.** The paper trains the VAE for one
  epoch [arxiv:1809.01999 app. A.2]. The released script runs 10
  [code: carracing/vae_train.py#L23]. Say which count you used.
- **Test-split epoch selection in DialogueRNN.** The IEMOCAP and AVEC scripts
  keep the epoch with the lowest test loss, so the reported numbers may be
  optimistic [code: DialogueRNN/train_IEMOCAP.py#L192-L196]. The script also
  applies hard-coded class weights that the paper's loss does not have
  [code: DialogueRNN/train_IEMOCAP.py#L163-L172].
- **EA-LSTM forget-gate bias.** The code sets the initial forget bias to 5 on
  part of the bias vector. The paper does not state an initial bias
  [code: papercode/ealstm.py#L75-L76].
- **Qualitative driving claims.** "Successful", "faster" and "better
  performance" in the driving paper have no measurements behind them
  [arxiv:1704.02532 §5]. Do not cite them as numerical evidence.
- **Linear recurrences train differently.** S5 and Mamba use a parallel scan
  over time, not a step-by-step loop, so the cost comparisons above do not
  carry over directly. Mamba's gated-RNN reading holds only for a special
  case: one state dimension, A = −1, B = 1 and a linear step size
  [arxiv:2312.00752 §3.5].
- **Staged sentiment model.** The paper describes an end-to-end method
  [arxiv:1806.06228 §2]. The released code trains each pair model with its own
  four-way loss, saves its activations, and trains the trimodal model on them
  as fixed inputs [code: hfusion.py#L302-L348]. The code's bimodal GRU is 600
  units followed by a 500-unit dense layer, where the paper uses a 500-unit
  GRU [code: hfusion.py#L246] [arxiv:1806.06228 §3.4].
- **Optimiser settings in the satellite LSTM.** The paper gives β2 = 0.98 for
  the RNN, MS-ResNet and TempCNN, while the code uses β2 = 0.999 for those
  three [arxiv:1910.10536 §5] [code: src/train.py#L177-L180].
- **Unreliable citations in the Brain Intelligence paper.** Its citation for a
  "dependency neural network" describes an LSTM, but the reference it gives is
  a dependency parser [arxiv:1706.01040 §2.8]. Verify citations before use.
- **Sequence ends in bidirectional video models.** The recurrences read frames
  beyond the first and last, and the paper does not say how those ends are
  filled [arxiv:2012.02181 eq. 1]. The linked repository has no model code at the
  cited commit, so no choice can be checked against it [arxiv:2012.02181 Paper vs code].
- **PredRNN decoupling weight.** The paper's objective uses weight 1 on the
  decoupling term with a sum of squared errors. The code uses 0.1 or 0.01 with a
  mean-based MSE, so the penalty is much smaller relative to reconstruction
  [arxiv:2103.09504 eq. 6] [code: core/models/predrnn_v2.py#L116].
- **PredRNN untested pipelines.** The radar and Traffic4Cast results depend on
  backbones and data loaders that are not in the repository, so they cannot be
  reproduced from it [arxiv:2103.09504 Paper vs code].
- **Class features in DCGRU are ambiguous.** The text calls them membership
  indicators, yet also calls them "values of the class". The virtual-value method
  is not given at all [arxiv:2512.06714 §3.3] [arxiv:2512.06714 §3.5].
- **Dense activations conflict within one paper.** The text says the second
  dense layer is linear, while the table says ReLU [arxiv:2512.06714 §4.1]
  [arxiv:2512.06714 tab. 5]. The card follows the table.
- **Adversarial music model: paper and code differ.** The code feeds the
  ground-truth event during pretraining, adds a second freezing threshold of
  2.0, and uses 100 units where the paper uses 350 [arxiv:1611.09904 Paper vs code].
  The paper's own pretraining wording does not say which input it means
  [arxiv:1611.09904 §3]. The code has no LICENSE file, so cite it by path only.
- **Survey text and survey table disagree.** Some gains the NER survey quotes
  in its text do not match its Table 1, such as the English gain over Chiu and
  Nichols [arxiv:1910.11470 §7] [arxiv:1910.11470 tab. 1]. Use the table values.
- **DCRNN equations versus text.** The urban graph survey says DCRNN replaces the
  linear units of an LSTM, but its equation is in GRU form
  [arxiv:2303.14483 §5.5.2] [arxiv:2303.14483 eq. 15]. Implement the equation.
- **Stabiliser and gate choices in xLSTM.** The exponential gates overflow
  without the stabiliser, so use the running-maximum subtraction [arxiv:2405.04517 §2.2].
  The released code computes mLSTM gates from q, k and v and uses a SiLU output
  gate with a skip and group norm, not the paper's linear gates and sigmoid
  output gate [arxiv:2405.04517 Paper vs code]. Pick one form and say which.
- **Two-step flow and offset residue in BasicVSR++.** The paper uses a flow from
  frame i to frame i-2 without defining it, and its offset equation has no bound
  on the residue. The released code composes the two-step flow and bounds the
  residue, and it also swaps the flow channels before adding them to the offsets
  [arxiv:2104.13371 Paper vs code]. An implementation that follows the paper alone
  will differ from the released code.

## Papers

- [An Experimental Review on Deep Learning Architectures for Time Series Forecasting](/papers/2103.12057): benchmarks LSTM, GRU, ERNN and ESN against MLP, CNN and TCN on 12 univariate datasets, with a grid of 38,000-plus models.
- [Continuous online sequence learning with an unsupervised neural network model](/papers/1512.05463): HTM sequence memory, learned in one pass without gradients, compared with LSTM, ELM, ESN and ARIMA on a taxi series.
- [Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems](/papers/1512.08756): attention pooling over per-step states as an alternative to a recurrent read, tested on two toy tasks.
- [AON: Towards Arbitrarily-Oriented Text Recognition](/papers/1711.04226): bidirectional LSTM encoder and attention LSTM decoder for scene text recognition.
- [Predicting Station-level Hourly Demands in a Large-scale Bike-sharing Network: A Graph Convolutional Neural Network Approach](/papers/1712.04997): graph CNN with a learned graph filter, and an LSTM variant, for next-hour demand at 272 stations.
- [Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders](/papers/1802.02080): bidirectional convolutional GRU and LSTM encoders for satellite image time series.
- [Natural Language Processing Advancements By Deep Learning: A Survey](/papers/2003.01200): survey of recurrent and attention models for NLP tasks, with copied benchmark tables.
- [DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction](/papers/2004.08919): library of compound and protein encoders, including a CNN with a bidirectional GRU or LSTM, for drug-target prediction.
- [COSMIC: COmmonSense knowledge for eMotion Identification in Conversations](/papers/2010.02795): GRU state trackers over a dialogue, with commonsense inputs, for emotion recognition.
- [Robust Motion In-betweening](/papers/2102.04942): LSTM generator with adversarial critics for motion transitions between keyframes.
- [Online Multi-Target Tracking Using Recurrent Neural Networks](/papers/1604.03635): RNN motion model and LSTM data association for online pedestrian tracking.
- [Neural Expectation Maximization](/papers/1708.03498): unrolled EM clustering with an RNN-EM variant, tested on shapes and digits.
- [Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting](/papers/2101.12072): TimeGrad, an LSTM-conditioned diffusion forecaster for multivariate series.
- [Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields](/papers/2404.09383): character LSTM and BiLSTM CRF tagger with a language vector for cross-lingual transfer.
- [A Taxonomy of Deep Convolutional Neural Nets for Computer Vision](/papers/1601.06615): survey that describes CNN-to-LSTM designs such as LRCN, with no experiments of its own.
- [All-in-one: Multi-task Learning for Rumour Verification](/papers/1806.03713): shared LSTM with stance, detection and veracity heads on Twitter reply branches.
- [A Neural Temporal Model for Human Motion Prediction](/papers/1809.03036): backward plan GRU and forward body GRU with a closed-loop loss ramp for long motion synthesis.
- [Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals](/papers/2106.15004): GRU encoders over a lane graph, inside a route-based trajectory model for nuScenes.
- [Filling the G_ap_s](/papers/2108.00298): GRIL, a bidirectional recurrent graph network with graph-gated GRUs for imputing sensor gaps.
- [DelRec: learning delays in recurrent spiking neural networks](/papers/2509.24852): spiking recurrent layers with learned delays, trained by surrogate gradients.
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](/papers/1609.05473): LSTM policy for discrete sequences, trained against a CNN discriminator by policy gradient.
- [Deep Reinforcement Learning framework for Autonomous Driving](/papers/1704.02532): LSTM temporal integration inside an end-to-end driving pipeline, with qualitative lane keeping only.
- [Brain Intelligence: Go Beyond Artificial Intelligence](/papers/1706.01040): position paper with no recurrent design and no results, cited for context only.
- [Recurrent World Models Facilitate Policy Evolution](/papers/1809.01999): MDN-RNN world model with a linear controller evolved by CMA-ES, on car racing and Doom.
- [Recent Trends in Deep Learning Based Natural Language Processing](/papers/1708.02709): NLP survey with a short section on the simple RNN, LSTM and GRU, and benchmark tables copied from cited work; no experiments.
- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](/papers/1811.00405): GRU states for context, each speaker and emotion, for utterance-level emotion labels.
- [Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets](/papers/1907.08456): EA-LSTM with static catchment attributes in the input gate, for rainfall-runoff over 531 basins.
- [Simplified State Space Layers for Sequence Modeling](/papers/2208.04933): S5, a diagonal state-space layer run over time as a linear recurrence with a parallel scan.
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](/papers/2312.00752): selective state-space model whose recurrence reduces to a gated RNN in one special case.
- [BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond](/papers/2012.02181): bidirectional recurrent propagation with flow-based feature warping for video super-resolution.
- [PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning](/papers/2103.09504): ST-LSTM with memory flowing up the layers and two decoupled memories, trained with reverse scheduled sampling.
- [A Novel Deep Neural Network Architecture for Real-Time Water Demand Forecasting](/papers/2512.06714): GRU with k-means class features for 15-minute water demand, with a virtual-value variant.
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](/papers/1611.09904): LSTM generator and bidirectional LSTM discriminator for continuous MIDI events, with feature matching.
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](/papers/1910.11470): survey of NER systems, with copied CoNLL and DrugNER tables for word, character and affix recurrent taggers.
- [Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey](/papers/2303.14483): survey of STGNNs for city data, with GRU-based fusion designs, and no experiments.
- [BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment](/papers/2104.13371): second-order grid propagation and flow-guided deformable alignment on top of BasicVSR.
- [xLSTM: Extended Long Short-Term Memory](/papers/2405.04517): exponentially gated LSTM with scalar and matrix memories, scaled to language modelling and compared with Transformers and state-space models.
- [Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks](/papers/1603.03827): LSTM or CNN encoder for each short text, with a history window over earlier texts, for dialog act labels.
- [The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches](/papers/1803.01164): survey of deep learning with the RNN, LSTM, GRU and ConvLSTM equations, and no experiments of its own.
- [Recurrent Back-Projection Network for Video Super-Resolution](/papers/1903.10128): recurrent loop over neighbouring frames with optical flow and back-projection, for 4x video super-resolution.
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](/papers/1905.02947): survey of emotion recognition in conversation, with recurrent context models such as DialogueRNN, and copied results.
- [Graph Neural Network for Traffic Forecasting: A Survey](/papers/2101.11174): survey of GNN methods for traffic forecasting, including RNN-based spatiotemporal GNNs such as DCRNN, with no experiments of its own.
- [Harmonic Mobile Manipulation](/papers/2312.06639): one RL policy with a 512-unit GRU that drives a Stretch robot's base and arm together from two RGB views and proprioception.
- [Attention-over-Attention Neural Networks for Reading Comprehension](/papers/1607.04423): bi-GRU reader that weights query-to-passage attention by a learned passage-to-query attention, for cloze questions on CNN and CBTest.
- [Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks](/papers/1704.02581): temporal and spatial LSTM streams over skeleton joints, fused by class posteriors, for action recognition.
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](/papers/2104.00681): sparse TSDF per key-frame fragment, fused with earlier fragments by a ConvGRU, for real-time monocular reconstruction.
- [Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling](/papers/1806.06228): per-modality GRU context and hierarchical GRU fusion for utterance-level sentiment on CMU-MOSI and IEMOCAP.
- [Self-attention for raw optical Satellite Time Series Classification](/papers/1910.10536): compares a Transformer with an LSTM and two CNNs on raw and preprocessed Sentinel-2 crop-type series.