Recurrent neural network (RNN)

Overview

A recurrent neural network (RNN) reads a sequence one step at a time. At each step it combines the current input with a hidden state that summarises the steps before. The output can be the last hidden state, or the whole sequence of hidden states, which a later layer then reads. The cited papers use this pattern for next-value forecasting 2103.12057 §2 2103.12057 §3.2.1, scene text recognition 1711.04226 §4.1, dialogue emotion tracking 2010.02795 §3.4, satellite image time series 1802.02080 §3.1, motion synthesis 2102.04942 §3.2, station-level bike demand 1712.04997 §3.3.2, molecule and protein encoders 2004.08919 app. A and sequence-to-sequence language tasks 2003.01200 §3.2, multi-target pedestrian tracking 1604.03635 § Target Motion, object grouping in an unrolled EM clustering 1708.03498 §2.3, probabilistic multivariate forecasting 2101.12072 §3 and named entity tagging 2404.09383 §2.3. The newer cards add a shared recurrent stack for rumour verification 1806.03713 §4.2, a backward plan network for human motion 1809.03036 §3.2, GRU encoders over a lane graph 2106.15004 app. A.2, graph-gated imputation over sensors 2108.00298 §4, recurrent spiking layers with learned delays 2509.24852 §2.1 and, in a survey, CNN features feeding one LSTM 1601.06615 §3.4.1. Three more cards use recurrence for video restoration 2012.02181 §3.1, spatiotemporal video prediction 2103.09504 §4.1 and short-term water demand 2512.06714 §3.4.

Several of the newer cards change how the recurrence is used. SeqGAN uses an LSTM as a text policy trained by policy gradient [arxiv:1609.05473 § The Generative Model for Sequences]. DialogueRNN tracks several GRU states per conversation 1811.00405 §3.3. The EA-LSTM feeds static basin attributes only into its input gate 1907.08456 §2.2. The state-space cards S5 and Mamba keep a recurrent form but compute it as a linear or selective scan 2208.04933 §2.2 2312.00752 §3.3. The driving paper uses an LSTM for temporal integration without measuring it 1704.02532 §4.2. The Brain Intelligence position paper names no recurrent design 1706.01040 §3. PredRNN sends its memory up the layer stack as well as forward in time 2103.09504 §4.1, and BasicVSR runs two recurrent sweeps in opposite time directions that do not exchange information 2012.02181 §3.1.

Three more cards add recurrence in other roles. C-RNN-GAN uses LSTMs as the generator and discriminator of a GAN over continuous music events 1611.09904 §2 1611.09904 §3. The named-entity survey ranks word-plus-character recurrent taggers as its strongest neural family 1910.11470 §6.4.3. The urban graph survey lists the GRU as the most widely used recurrent unit, and shows it both after a graph convolution and inside one 2303.14483 §5.2 2303.14483 §5.5. The NLP survey gives the simple RNN, LSTM and GRU equations. It reports that both gated cells beat the simple RNN on music and speech, and that neither is clearly better 1708.02709 §4.2.

Two newer cards change the recurrent cell or its propagation. The xLSTM card keeps the LSTM's gated cell but adds exponential gates, a normaliser and a matrix memory, and reports language-model results against Transformers and state-space models 2405.04517 §2 2405.04517 §4.2. The BasicVSR++ card keeps bidirectional recurrent propagation, but runs it as a second-order grid and aligns the carried states with flow-guided deformable convolution, not a gated cell 2104.13371 §3.1 2104.13371 §3.2.

The paper's recurrent family for forecasting has four cells. The Elman RNN (ERNN) is the plain form. LSTM and GRU add gates to control what the hidden state keeps. The echo state network (ESN) is the fourth cell in the comparison 2103.12057 §2. Each cell is trained as a global model over all series of one dataset, with the same input windows and the same output head 2103.12057 §3.1 2103.12057 §3.3.

The forecasting evidence is one author-reported benchmark on twelve public datasets. It compares seven architectures and reports no classical statistical baselines 2103.12057 §3.2. Treat its rankings as a guide to where to start, not as a settled result. The other papers test different tasks, with their own baselines, so their results do not rank the cells against each other.

Five more cards add recurrence in other roles. A sequential short-text classifier encodes each text with an LSTM or a CNN, then runs a small history layer over the texts in order 1603.03827 §2. RBPN carries a high-resolution feature estimate through a loop over neighbouring video frames 1903.10128 §3. The emotion-in-conversation survey summarises context models built on recurrence, mostly DialogueRNN, and copies their results from earlier papers 1905.02947 §4. The traffic GNN survey splits spatiotemporal GNNs into RNN-based and CNN-based models, with DCRNN as a classic example 2101.11174 §4.2. The deep learning survey restates the LSTM, GRU and ConvLSTM equations and runs no experiments of its own 1803.01164 §V.

Two more cards use recurrence over sequences of other kinds. The multimodal sentiment model runs one GRU per modality over the utterances of a clip, then a GRU after each fusion stage 1806.06228 §3.3 1806.06228 §3.4. The satellite crop-type study compares an LSTM, which follows the authors' earlier model, with a Transformer and two convolutional networks on raw Sentinel-2 series 1910.10536 §4.1 1910.10536 tab. 1.

Three newer cards put recurrence inside a larger model. A reading-comprehension reader encodes the passage and the question with a bi-directional GRU, then pools the passage with attention over attention 1607.04423 §3. A skeleton action model runs two LSTM streams, one over joints per frame and one over joints as an ordered sequence 1704.02581 §4. A monocular reconstruction network fuses each fragment with earlier fragments through a ConvGRU 2104.00681 §3.2.

Variants

LSTM (long short-term memory)

Gated cell. In the benchmark it had the best single configuration on 4 of 12 datasets, more than any other recurrent cell 2103.12057 §4.1 2103.12057 tab. 11. The paper calls it the most accurate choice when tuned well and gives it the best accuracy among the models with adequate training time 2103.12057 §4.2. Its best settings were one or two layers, with batch size 32 and min-max normalization 2103.12057 tab. 17 2103.12057 tab. 18. Card: An Experimental Review on Deep Learning Architectures for Time Series Forecasting.

GRU (gated recurrent unit)

Gated cell. Its best single configuration won on 2 datasets, CIF2016o12 and M3 2103.12057 tab. 8 2103.12057 tab. 11. The configurations with one and two layers were both marked significantly best 2103.12057 tab. 17. Its best settings used min-max normalization with batch size 64 2103.12057 tab. 18. The paper reports no significant difference between its best cells and LSTM's, except that the MLP is worst 2103.12057 §4.3. An NLP survey reports that neither gated cell is clearly better on music and speech tasks, and notes that those comparisons are not on NLP tasks 1708.02709 §4.2. Card: An Experimental Review on Deep Learning Architectures for Time Series Forecasting.

Exponential gating with scalar and matrix memories (xLSTM)

xLSTM keeps the LSTM's additive, gated cell state but makes the input gate, and optionally the forget gate, exponential. A normaliser state and a stabiliser state keep the exponentials finite, and the paper says the stabilised form gives the same outputs and gradients 2405.04517 §2.2 2405.04517 app. A.2. The sLSTM keeps a scalar memory with memory mixing inside each head. The mLSTM has a d × d matrix memory, updated with a covariance rule and read with a query, and no hidden-to-hidden link, so it can be computed in parallel over the sequence 2405.04517 §2.3. In the paper's 15B-token ablation, a vanilla multi-layer LSTM reaches 2417.86 validation perplexity. Adding a residual backbone brings it to 35.46, exponential gating to 17.70, and matrix memory to 13.48 2405.04517 tab. 2. The 15B-token comparison puts xLSTM[1:0] at 13.43, against 13.70 for Mamba and 14.25 for Llama 2405.04517 tab. 1. At 300B tokens, the 1.3B xLSTM[1:0] reaches 8.89, against 9.14 for Mamba and 9.44 for Llama 2405.04517 tab. 3. All numbers are author-reported. Card: xLSTM: Extended Long Short-Term Memory.

ERNN (Elman recurrent network)

The simplest recurrent cell in the comparison, with no gates. Its best cell won on 2 datasets. Its best settings used one or two layers, 32 or 64 units, and returned only the last output 2103.12057 tab. 17. Its best single configuration won on 2 datasets, M4 and Tourism 2103.12057 tab. 8 2103.12057 tab. 11. Card: An Experimental Review on Deep Learning Architectures for Time Series Forecasting.

ESN (echo state network)

Its best single configuration won on ExchangeRate, dataset 3 2103.12057 tab. 8 2103.12057 tab. 11. Its best configurations used two or four layers and returned the full sequence 2103.12057 tab. 17. The paper reports slow inference for this cell 2103.12057 §4.2 2103.12057 fig. 4. The code also adds an extra dense layer before the output for ESN only, which the paper does not describe 2103.12057 Paper vs code. Card: An Experimental Review on Deep Learning Architectures for Time Series Forecasting.

Bidirectional LSTM encoder with attention decoder

The input is a sequence of image features read in both directions. A bidirectional LSTM encodes each direction, and an LSTM decoder with attention writes one character per step, trained end to end from word labels 1711.04226 §4.1 1711.04226 §3.4. The decoder has 256 memory blocks; its layer count is not stated 1711.04226 §5.2. The authors report the best lexicon-free result on several irregular benchmarks, but their own tables show other methods ahead on some columns 1711.04226 §5.3. Card: AON: Towards Arbitrarily-Oriented Text Recognition.

LSTM generator in an adversarial setup

The LSTM is the recurrent core of a motion generator. It takes concatenated state and target embeddings and decodes them into velocities 2102.04942 §3.2. Two adversarial critics and reconstruction losses train it 2102.04942 §3.2. The paper says it chose LSTM over GRU empirically, and gives no hidden size or depth for the LSTM 2102.04942 §5.5 2102.04942 §3.2. Card: Robust Motion In-betweening.

LSTM after a learned graph filter (station bike demand)

The LSTM is the recurrent block of GCNN_rec-DDGF. Each hour, the demand vector of all stations is first mixed through a learned graph filter. The result then feeds an LSTM with 24 time steps and 100 hidden units per cell 1712.04997 §3.3.2 1712.04997 tab. 2. The paper does not write out the LSTM gate equations 1712.04997 §3.3.2. The public code has no recurrent variant 1712.04997 Paper vs code. On 272 Citi Bike stations it has the lowest test RMSE of 13 models: 2.12, against 2.35 for the regular variant without the recurrent block and 2.46 for a plain LSTM baseline 1712.04997 tab. 3. These are author-reported figures from one dataset. Card: Predicting Station-level Hourly Demands in a Large-scale Bike-sharing Network: A Graph Convolutional Neural Network Approach.

CNN with a bidirectional GRU or LSTM (drug and protein encoders)

DeepPurpose offers a bidirectional GRU or LSTM on top of a 1D CNN as a compound encoder over SMILES strings. For proteins it offers a bidirectional RNN on top of a CNN 2004.08919 app. A. The paper does not state the layer count or hidden size; the code defaults are 2 layers and 64 units 2004.08919 app. A. The results table in the card does not isolate this encoder. It shows the CNN+CNN and MPNN+AAC pairs on DAVIS, and the text says many DeepPurpose models perform comparably 2004.08919 §3. Card: DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction.

GRU state trackers in a dialogue model

COSMIC runs five GRU cells over a conversation: one for context, and separate ones for each speaker's internal state, external state and intent, plus one for emotion 2010.02795 §3.4. The GRU sizes are not stated in the paper. The released code uses 150 units for most states and 450 for the emotion state train_iemocap.py. On IEMOCAP it reports 65.28 weighted F1 against 64.76 for a RoBERTa DialogueRNN baseline 2010.02795 tab. 4. Its ablations that drop the commonsense inputs lower the IEMOCAP score 2010.02795 tab. 4. The recurrent cells are one part of a larger system with pretrained encoders and commonsense features. Card: COSMIC: COmmonSense knowledge for eMotion Identification in Conversations.

Convolutional GRU and LSTM

The recurrent cells here are convolutional, with kernel size 3, and read a Sentinel-2 time series in both directions 1802.02080 tab. 1 1802.02080 §5.2. The main model is a ConvGRU with 256 cells, trained for 60 epochs with batch size 32; a ConvLSTM was also trained, with batch size 28 1802.02080 §5.2. The reported results are for the GRU encoder only 1802.02080 tab. 2. See convolutional-recurrent-network for the general pattern. Card: Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders.

LSTM encoder-decoder for sequence-to-sequence

An encoder reads the input, and a decoder writes the output, usually with RNN or LSTM cells 2003.01200 §3.2. Training uses teacher forcing and a cross-entropy loss 2003.01200 eq. 1 2003.01200 §3.2. The survey gives the LSTM objective for translation 2003.01200 eq. 8. It runs no experiments of its own, and its tables copy scores from the works it cites 2003.01200 tab. 2. Card: Natural Language Processing Advancements By Deep Learning: A Survey.

Sequence memory without backpropagation (HTM)

This model is recurrent only in the sense that it carries context from one element to the next. It is not trained with gradients. Each element is a sparse binary code over 2048 columns with 32 cells per column 1512.05463 tab. 1. Its learning rule is Hebbian-like and runs in a single pass over the stream 1512.05463 §3.3 1512.05463 §6.1. The authors report NYC taxi demand error comparable to the best LSTM, and below ARIMA, ELM and ESN 1512.05463 §5 1512.05463 fig. 10. Under temporal noise present throughout training, LSTM is slightly better 1512.05463 §4.5. On the Reber grammar task HTM is below LSTM 1512.05463 §6.4. Card: Continuous online sequence learning with an unsupervised neural network model.

Feed-forward attention pooling (no recurrence)

Each step is mapped to a state on its own, and a learned attention score averages the states over time into one vector 1512.08756 §2.1. The paper's stated motivation is that recurrent networks are trained by backpropagation through time and cannot be fully parallelised over long inputs 1512.08756 §1. On two toy tasks it reaches perfect accuracy for lengths from 50 to 10000 in its fixed-length experiment 1512.08756 tab. 1. One epoch at length 10000 took 254 s, against 917 s for a single-layer vanilla RNN of hidden size 100 on the same GPU 1512.08756 §2.2. The RNN is a speed reference only. The paper reports no tuned RNN baseline on these tasks 1512.08756 §2.2. Card: Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems.

RNN motion model with LSTM data association (online tracking)

Two learned blocks track an unknown number of pedestrians frame by frame. An RNN predicts each target's next state and its existence probability. A per-target update then mixes the prediction with its assigned detection 1604.03635 § Multi-Target Tracking with RNNs 1604.03635 § Target Motion. An LSTM reads each target's row of distances to the detections and outputs an assignment over the detections plus one "missing" column 1604.03635 § Data Association with LSTMs. The models are trained only on synthetic trajectories 1604.03635 § Training Data. On the MOTChallenge 2015 test set it reports 19.0% MOTA and 165.2 FPS, which the authors say is not competitive on accuracy but faster than the methods they list 1604.03635 tab. 2 1604.03635 § Benchmark results. On the training set, the variant with Hungarian assignment has higher MOTA than the LSTM variant, 24.0 against 22.3 1604.03635 tab. 1. Card: Online Multi-Target Tracking Using Recurrent Neural Networks.

RNN memory inside unrolled EM clustering (RNN-EM)

Neural EM unrolls the Expectation Maximization steps into a differentiable clustering over pixels. RNN-EM replaces the gradient step that EM derives with an unrestricted RNN. Each component's hidden state receives its responsibility-weighted residual 1708.03498 §2.3 1708.03498 fig. 2. Because the weights are not restricted, the likelihood no longer carries a convergence guarantee, but the recurrent form can predict the next frame of a sequence 1708.03498 §2.3. The recurrent unit has 250 sigmoid units for static shapes and 100 for flying shapes 1708.03498 app. A.1 1708.03498 app. A.2. On static shapes, RNN-EM scores AMI 0.826 ± 0.005 against 0.475 ± 0.043 for N-EM 1708.03498 §4.1. On flying shapes with three objects it scores 0.970 ± 0.005 1708.03498 tab. 1. These are author-reported results on synthetic shapes and digits. Card: Neural Expectation Maximization.

LSTM conditioner for a diffusion forecaster (TimeGrad)

An LSTM encodes the past of a multivariate series, and a conditional denoising diffusion model samples each future step. The noise predictor is conditioned on the RNN state from the previous step 2101.12072 §3 2101.12072 eq. 10. The RNN has two LSTM layers of hidden size 40 2101.12072 §4.2. The model draws 100 sample paths, each with 100 reverse diffusion steps per forecast step 2101.12072 §4.2 2101.12072 alg. 2. The authors report that the number of diffusion steps can fall to about 10 with little loss, against a best value near 100 on Electricity 2101.12072 §4.4 2101.12072 fig. 3. On CRPS_sum it has the best score on five of six datasets, and on Exchange, the smallest, it does not 2101.12072 tab. 2. These are author-reported figures. The card has no timing comparison with the other models. The code's default RNN is an LSTM, while the example notebook uses a GRU time_grad_estimator.py Time-Grad-Electricity.ipynb. Card: Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting.

Char-level LSTM and BiLSTM tagger with a language vector (cross-lingual NER)

Each word is encoded by a character LSTM, and its final state is concatenated with a word embedding. A sentence-level BiLSTM reads the words, and a conditional random field scores tag transitions and tag outputs 2404.09383 §2.3 2404.09383 eq. 4 2404.09383 eq. 5. For transfer, the character LSTM, BiLSTM, tag layers and transition weights are shared across languages, and a learned language vector conditions the scores 2404.09383 §3.1 2404.09383 eq. 6. The training objective adds a weighted source-language likelihood to the target-language likelihood 2404.09383 eq. 7. With 100 target sentences, the neural model beats the log-linear CRF on every listed transfer pair, by 3.97 to 9.80 F1. Without a source language it loses on all five targets, by 0.75 to 8.45 F1 2404.09383 tab. 2. With 10,000 target sentences and no transfer, it beats the log-linear CRF on all five targets 2404.09383 tab. 2. Card: Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields.

CNN features into one LSTM (LRCN, as surveyed)

Per-frame CNN features, from RGB frames and from optical flow, feed one LSTM layer of 256 hidden units. The frame predictions are then averaged 1601.06615 §3.4.1 1601.06615 fig. 6. The survey is a recipe overview and runs no experiments, so it gives no accuracy for this model 1601.06615 §1. The survey's toy example is a single-hidden-layer RNN that outputs a running sum with unit weights and ReLU 1601.06615 §3.4. Card: A Taxonomy of Deep Convolutional Neural Nets for Computer Vision.

Shared LSTM with stance, detection and veracity heads

One LSTM stack reads each reply branch and is shared by three tasks: stance for every tweet, rumour detection and veracity for the whole branch. Each task then has its own dense layers and output 1806.03713 §4.2 1806.03713 fig. 3. The official code adds a private LSTM layer in front of the veracity and detection heads, which the paper does not describe MTL3_CV_VeracityStanceDetection.py. On PHEME with five events, the three-task model reaches macro-F 0.396. The single-task branchLSTM reaches 0.336 and NileTMRG* 0.339 1806.03713 tab. 3. On RumourEval, the veracity-plus-stance model reaches 0.558, against 0.491 for branchLSTM and 0.539 for NileTMRG* 1806.03713 tab. 3. These are author-reported single runs with no variance. Card: All-in-one: Multi-task Learning for Rumour Verification.

Backward plan GRU guiding a forward body GRU

A backward-running GRU starts from a random draw and the action label. It emits one guide vector per future step. A forward body GRU reads the pose and the guide at each step and predicts the next pose 1809.03036 §3.2 1809.03036 fig. 1. Training adds an open-loop error to a closed-loop error, and the closed-loop weight starts at zero and steps up, to limit drift 1809.03036 §3.4 1809.03036 eq. 6. In the long-term NPSS table, the lowest score on each of walking, eating and smoking belongs to one of the two proposed models 1809.03036 tab. 4. In a user study, NPSS agreed with the majority vote on 65 of 80 comparisons, against 39 of 80 for MSE 1809.03036 tab. 3. These are author-reported results on Human3.6M. Card: A Neural Temporal Model for Human Motion Prediction.

GRU encoders over a lane graph

Three one-layer GRUs, each with hidden size 32, encode the target's motion, the other agents and the lane-graph nodes 2106.15004 app. A.2. The recurrence is only the encoder: graph layers, agent-to-node attention and a route-conditioned attention decoder follow 2106.15004 §4.1 2106.15004 §4.3. On nuScenes the model reports MinADE10 1.00 and MissRate10,2 0.37, author-reported 2106.15004 tab. 1. The ablations in tab. 2 do not remove the GRU encoders, so their effect is not measured 2106.15004 tab. 2. Card: Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals.

Bidirectional recurrent graph network with graph-gated GRUs (GRIL)

Each GRU gate is a message-passing network over a sensor graph. Each time step has two imputation stages: a first guess from the hidden state, then a second guess from neighbouring sensors. A forward and a backward unit are combined by an MLP 2108.00298 §4 2108.00298 eq. 11. Against BRITS, the author-reported MAE on PEMS-BAY with point missing values is 0.67 against 1.47, and out-of-sample on AQI it is 14.73 against 20.21 2108.00298 tab. 2 2108.00298 tab. 1. The percentage reductions are computed on the card, not reported by the paper. Card: Filling the G_ap_s.

Spiking recurrent layer with learned delays (DelRec)

A leaky integrate-and-fire recurrent layer learns a real-valued delay per neuron (axonal) or per connection (synaptic). It is trained with surrogate gradients 2509.24852 §2.1. During training each spike is spread over neighbouring steps by a triangular kernel, whose width is annealed towards zero. Delays are rounded to integers at inference, except on AL 2509.24852 §4.4. Author-reported test accuracy: 82.42% on SSC and 85.74% on AL with axonal delays, 96.63% on PS-MNIST with synaptic delays, and 83.53% on HAR with axonal delays 2509.24852 tab. 1 2509.24852 tab. 2. The cell is spiking and has no gates, so it is not an LSTM or GRU. Card: DelRec: learning delays in recurrent spiking neural networks.

LSTM policy in a sequence GAN (SeqGAN)

The generator is an LSTM that picks each token as an action. A CNN discriminator scores each finished sequence. Monte Carlo roll-outs turn that score into a value for each earlier token, and the policy gradient uses it 1609.05473 § SeqGAN via Policy Gradient 1609.05473 eq. 4. The paper says GRU cells or attention could replace the LSTM [arxiv:1609.05473 § The Generative Model for Sequences]. Cell sizes are not stated in the paper. The code uses embedding and hidden sizes of 32 sequence_gan.py. On the synthetic oracle task it reports NLL 8.736, against 9.038 for MLE 1609.05473 tab. 1. These are author-reported. Card: SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient.

LSTM temporal integration in a driving pipeline

Sensor fusion and an attention glimpse feed a CNN. An LSTM-style gate update then keeps a hidden state over time, so the planner can cope with what it cannot see 1704.02532 §4.2. The planner is a DQN or DDAC head 1704.02532 §4.3. The test is lane keeping in TORCS, and the results are qualitative. The paper gives no measurement for the recurrent part 1704.02532 §5. Card: Deep Reinforcement Learning framework for Autonomous Driving.

Entity-aware LSTM with a static input gate (EA-LSTM)

The static attributes of each basin drive only the input gate. The weather inputs and the previous hidden state drive the other gates, and the cell and hidden updates are standard 1907.08456 §2.2 1907.08456 eq. 7-12. The read-out is the last hidden state, with dropout 0.4 and one layer 1907.08456 app. B 1907.08456 §2.6. On 447 validation basins the single model has mean NSE 0.674 and the 8-model ensemble 0.705. The paper's LSTM with static inputs concatenated scores 0.685 on the same set, so the EA-LSTM is not the best row in that table 1907.08456 tab. 3. Card: Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets.

Three GRU states per conversation (DialogueRNN)

A global GRU tracks the conversation context. One party GRU per speaker tracks that speaker's state, and an emotion GRU reads the speaker's updated party state to label each utterance 1811.00405 §3.3.1 1811.00405 §3.3.3 1811.00405 §3.3.5. Attention over earlier global states supplies context 1811.00405 §3.3.3. The bidirectional variant needs the whole dialogue 1811.00405 §3.4.2. On IEMOCAP text, the bidirectional model with attention reaches 63.40 weighted accuracy and 62.75 weighted F1. The unidirectional model reaches 59.33 and 59.89 1811.00405 tab. 2. Card: DialogueRNN: An Attentive RNN for Emotion Detection in Conversations.

LSTM mixture-density model in a world model (MDN-RNN)

An LSTM with a mixture-density output predicts the next latent vector from the current latent, the action and its own state 1809.01999 §2 1809.01999 app. A.3. The mixture has 5 Gaussians per latent dimension, and the hidden size is 256 for car racing and 512 for Doom [arxiv:1809.01999 app. A.3]. A linear controller reads the latent and the LSTM hidden state 1809.01999 app. A.4. Sampling divides the mixture logits by a temperature τ, which sets how noisy the imagined environment is 1809.01999 §2 rnn.py. A controller trained only inside the model scores 1092 ± 556 on the real DoomTakeCover task at τ = 1.15 1809.01999 tab. 2 1809.01999 §4.2. The car racing full model scores 906 ± 21 1809.01999 tab. 1. These are author-reported. Card: Recurrent World Models Facilitate Policy Evolution.

Diagonal linear recurrence with a parallel scan (S5)

S5 is a multi-input, multi-output diagonal state-space layer. It runs over time as a linear recurrence, computed with a parallel scan 2208.04933 §2.2 2208.04933 §3.4. The recurrence has no gates, so it is not an LSTM or GRU. Its reported LRA average is 87.46%, against 86.09% for S4-LegS, and 98.58% on Path-X 2208.04933 tab. 1. These are author-reported. Card: Simplified State Space Layers for Sequence Modeling.

Selective state space with a gated-RNN reading (Mamba)

Mamba makes the step size and the input and output matrices depend on each token 2312.00752 §3.2. For one state dimension, fixed A and B, and a linear step size, the update is a gated recurrence 2312.00752 §3.5 2312.00752 eq. 5. The authors report 4 to 5 times the inference throughput of a Transformer of similar size 2312.00752 §4.5. That throughput test uses an untrained 6.9B model 2312.00752 app. E.5. Card: Mamba: Linear-Time Sequence Modeling with Selective State Spaces.

Bidirectional recurrent propagation with feature warping (BasicVSR, IconVSR)

Two recurrent branches sweep a low-resolution clip in opposite directions and do not exchange information. Each branch warps its carried state onto the current frame with optical flow, refines it with 30 residual blocks, and an upsampler combines the two final states 2012.02181 §3.1 2012.02181 app. A. Warping features rather than pixels is the authors' main alignment choice: image warping lost 0.17 dB in their test, and removing alignment lost 1.19 dB 2012.02181 §3.1. IconVSR adds a keyframe information-refill every five frames, and lets the backward state feed the forward branch 2012.02181 §3.2 2012.02181 app. A. On REDS4 (BI, 4×), the authors report 31.42 dB for BasicVSR with 6.3M parameters and 63 ms, against 31.09 dB, 20.6M and 378 ms for EDVR. IconVSR reports 31.67 dB with 8.7M parameters 2012.02181 tab. 2. Inference runs on whole sequences, and memory use for long videos is not reported 2012.02181 app. A. Card: BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond.

Second-order grid propagation with flow-guided deformable alignment (BasicVSR++)

BasicVSR++ keeps BasicVSR's bidirectional recurrence and changes two parts. The propagation runs two rounds of backward and forward sweeps, and each step uses the two previous states of its sweep 2104.13371 §3.1. The alignment uses optical flow as the base of deformable offsets, and a small network learns only a residue on top of the flow 2104.13371 §3.2 2104.13371 eq. 6. The authors report 32.39 dB on REDS4 (BI, 4×) with 7.3M parameters and 77 ms, against 31.42 dB for BasicVSR with 6.3M and 63 ms 2104.13371 tab. 1. The smaller BasicVSR++ (S) reaches 32.24 dB with 6.4M parameters and 69 ms 2104.13371 tab. 2. In the ablation, whose dataset the caption does not name, flow-guided alignment moves the baseline from 31.48 dB to 31.94 dB, and second-order propagation adds a further 0.14 dB 2104.13371 tab. 3. The runtime is for 180 × 320 input 2104.13371 tab. 1. The backward branch starts at the last frame, so outputs depend on the whole clip 2104.13371 §3.1. Card: BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment.

Spatiotemporal memory flow with decoupled memories (PredRNN)

Each unit keeps a temporal memory C that runs along time within its layer, and a spatiotemporal memory M that runs up from the layer below. Each has its own gates, and the output gate reads both new memories 2103.09504 §4.2 2103.09504 eq. 4. A penalty on the absolute cosine similarity of the two memory increments is used in training only, and the authors say it adds no parameters at inference 2103.09504 §4.2 2103.09504 eq. 5. Reverse scheduled sampling raises the chance that the encoder sees true frames over training. The best run uses an exponential schedule from 0.5 to 1.0 2103.09504 §4.4 2103.09504 tab. 6. On fixed Moving MNIST, the authors report MSE 48.4 for PredRNN-V2 against 103.3 for ConvLSTM 2103.09504 tab. 2. Adding decoupling and RSS lowers PredRNN from 56.8 to 48.4, and the authors apply the two techniques to several other backbones 2103.09504 tab. 5. PredRNN-V2 has 23.86 MB of parameters, against 16.60 MB for ConvLSTM 2103.09504 tab. 2. Card: PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning.

GRU with k-means class features (DCGRU)

Each reading is paired with class features from a k-means clustering of the history. The paper picks m = 4 classes from an elbow plot 2512.06714 §3.3 2512.06714 §4.1. Each row passes through three time-distributed dense layers, with sizes 5, 50, 10 and 1. A 32-unit GRU reads the 96 rows, and a one-unit GRU cell gives the output 2512.06714 §3.4 2512.06714 tab. 5. The EDCGRU variant inserts one virtual value between readings, which doubles the window to 192 rows 2512.06714 §3.5 2512.06714 §4.2. On two district metering areas, the one-step MAPE is 1.26% and 1.31% for DCGRU. It is 2.49% and 3.58% for the baseline BGRU, which the paper describes as the same GRU block without class features 2512.06714 §3.7.1 2512.06714 tab. 6. GRUN, the other baseline, scores 2.06% and 2.46% 2512.06714 tab. 6. The DCGRU model has 4,187 parameters, against 30,403 for GRUN 2512.06714 tab. 6. All numbers come from one data set, with training and testing on the same DMA 2512.06714 §2. Card: A Novel Deep Neural Network Architecture for Real-Time Water Demand Forecasting.

LSTM generator and bidirectional LSTM discriminator (C-RNN-GAN)

The generator is a two-layer unidirectional LSTM. Each step takes a random vector concatenated with the previous output. The discriminator is a two-layer bidirectional LSTM that scores a whole sequence as real or generated 1611.09904 §2 1611.09904 §3. The paper sets 350 units per layer for both networks. Training starts with six epochs of next-event prediction, then adversarial training with feature matching, and a rule that pauses the discriminator when its loss falls below 70% of the generator's 1611.09904 §3. Results are curves only. The author reports more varied tones than a maximum-likelihood baseline 1611.09904 §4 1611.09904 fig. 3a–b. Card: C-RNN-GAN: Continuous recurrent neural networks with adversarial training.

Word-plus-character recurrent tagger with affix embeddings (NER survey)

Each word vector joins a word embedding with a recurrent summary of the word's characters. A sentence-level recurrent layer then reads the joined vectors. The affix variant adds learned embeddings of frequent prefixes and suffixes 1910.11470 §6.4.3 1910.11470 §6.4.4. The survey copies these numbers from Yadav et al., with no experiments of its own. The best author-reported CoNLL F1 is 87.26 in Spanish, 87.54 in Dutch and 79.01 in German, at 150 epochs, and 90.86 in English, which is below two other English rows in the same table 1910.11470 tab. 1. The layer widths and the joining operation are not stated 1910.11470 §6.4.4. Card: A Survey on Recent Advances in Named Entity Recognition from Deep Learning models.

GRU in a factorized or coupled spatio-temporal block (urban graph survey)

In the factorized design, a graph convolution runs first and a GRU then reads its output at each time step 2303.14483 §5.5.1. In the coupled design, the graph convolution replaces the linear maps inside the GRU gates, as in DCRNN 2303.14483 §5.5.2. The survey gives both designs as equations and runs no experiments, so it does not rank them 2303.14483 §1. Card: Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey.

LSTM or CNN encoder with a history window (sequential short-text classification)

Each short text is turned into one vector, by an LSTM with max pooling over its hidden states or by a CNN with max pooling over its windows. A two-layer head then reads the vectors of the current text and a few earlier ones 1603.03827 §2.1 1603.03827 §2.2. Author-reported best accuracies are 66.2% on DSTC 4 (LSTM), 84.6% on MRDA (CNN) and 73.1% on SwDA (CNN) 1603.03827 tab. 4. Raising the history of the first layer helps more than raising the second, and using history at both layers "does not help in most cases" 1603.03827 §4. Card: Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks.

Recurrent loop with back-projection (RBPN)

For each neighbouring frame, RBPN estimates the optical flow to the target frame and passes it through a loop that updates a high-resolution feature estimate. The features from all neighbours are concatenated and one convolution reconstructs the frame 1903.10128 §3. The recurrent state is a feature map, not a gated cell. In the paper's ablation, swapping the concatenation for a convolutional LSTM lowered PSNR on SPMCS-32 from 31.64 dB to 31.46 dB 1903.10128 tab. 3. Author-reported 4x result: 30.10 dB on SPMCS-11 for RBPN/6-PF, against 29.42 dB for VSR-DUF 1903.10128 tab. 8. Card: Recurrent Back-Projection Network for Video Super-Resolution.

GRU memory in an end-to-end mobile-manipulation policy (HarmonicMM)

The policy takes the joint embedding of two frozen-backbone camera views and the arm proprioception, then passes it through one GRU layer with 512 units. Linear heads read the hidden state for actions and value [arxiv:2312.06639 app. Network Architecture]. The GRU is one component of the policy, and the paper's claims are about the whole policy. The ablations it reports remove the proprioception and the visual backbone, not the GRU, so the recurrent layer's effect is not isolated 2312.06639 tab. II 2312.06639 tab. IV. The author-reported average success is 47% over four simulated tasks in unseen houses, 17.6 points above the ReLMoGen adaptation 2312.06639 §V-B 2312.06639 tab. I. Card: Harmonic Mobile Manipulation.

Bi-GRU encoders with attention over attention (AoA Reader)

Cloze-style reading comprehension. A shared embedding feeds one bi-directional GRU for the passage and one for the question. A passage-by-question matching matrix is normalised twice: over passage positions for each question word, and over question positions for each passage word. The second map is averaged over the passage to give one weight per question word, which weights the first maps. The result is summed per vocabulary word 1607.04423 §3 1607.04423 eq. 6-11. The authors say this adds a few steps to the AS Reader and no weights 1607.04423 §3. On the CBTest NE test split the single model scores 72.0 and the version with 5-best re-ranking scores 74.0, against 68.6 for the AS Reader 1607.04423 tab. 2. Re-ranking uses three language-model features, one of them trained per test sample 1607.04423 §4. Results are author-reported, and no official code was found 1607.04423 Paper vs code. Card: Attention-over-Attention Neural Networks for Reading Comprehension.

Two-stream LSTM over skeleton joints

Skeleton-based action recognition. A temporal stream reads per-frame joint coordinates, either as a stack of two LSTM layers or as five body-part LSTMs feeding a whole-body LSTM 1704.02581 §4.1. A spatial stream reads the joints as one ordered sequence, each step holding one joint's coordinates over a time window, and the order is either a chain or a graph traversal 1704.02581 §4.2. The two streams' class posteriors are fused, and the paper does not write the fusion formula out 1704.02581 §4. Random rotation, scaling and shear, applied only during training, add 2.7 points on NTU cross-subject and 7.8 on cross-view 1704.02581 tab. 1. The spatial stream alone scores 55.2 and 60.5 on NTU, against 67.8 and 70.5 for the hierarchical temporal stream 1704.02581 tab. 1. The full model reports 71.3 and 79.5 on NTU, 94.8 on SBU and 91.7 F1 on ChaLearn [arxiv:1704.02581 tab. 3] 1704.02581 tab. 4 1704.02581 tab. 5. These are author-reported, and the card finds no official code. Card: Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks.

ConvGRU fusion across key-frame fragments (NeuralRecon)

Real-time 3D reconstruction from monocular video. Each fragment of nine key frames is decoded into one sparse signed-distance volume in a single pass 2104.00681 §3. A ConvGRU, written with sparse convolutions, combines the fragment's features with a hidden state held over the global volume, so later fragments update earlier geometry 2104.00681 §3.2 GRU Fusion. With five views and the same occupancy area, the GRU reaches an F-score of 0.488, against 0.462 for a running-average linear fusion 2104.00681 tab. 5. The released code adds a pointwise linear term inside each sparse convolution that the paper's four equations do not show 2104.00681 Paper vs code. Card: NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video.

GRU context and fusion GRUs in a multimodal sentiment model (HFusion)

Each modality has its own GRU, which runs over the utterances of a clip, so every utterance vector carries context from the clip 1806.06228 §3.3. After each pair of modalities is fused, a GRU with output size 500 reads the pair sequence. A GRU with output size 550 then reads the trimodal fusion 1806.06228 §3.4. The unimodal GRU width is not stated 1806.06228 §3.3. A footnote says an LSTM "does not perform well" as the context model, and gives no number 1806.06228 §3.3. With all three modalities and the paper's own features on CMU-MOSI, the model reaches 77.9% accuracy, against 77.0% for early fusion and 77.3% for the second baseline 1806.06228 tab. 2. These are author-reported. Card: Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling.

LSTM on raw satellite series (crop types)

The LSTM is the authors' earlier crop-type model. It reads each parcel's series, one vector of 13 spectral bands per date 1910.10536 §4.1 1910.10536 §7.2.1. On raw data, the 12-class accuracy is 0.90 for the LSTM, against 0.87 for MS-ResNet and 0.83 for TempCNN. On 23 classes, the raw kappa is 0.71 for both the LSTM and the Transformer 1910.10536 tab. 1. The paper calls the RNN and the Transformer slightly better than the convolutional networks on raw data 1910.10536 §8.1. Preprocessing raised every model's score, so the LSTM result is for raw data only 1910.10536 §8.1. Card: Self-attention for raw optical Satellite Time Series Classification.

When to use which

The paper gives no single answer for forecasting. Its own conclusion depends on how accurate the forecasts must be, how fast they must come, and how much tuning is affordable 2103.12057 §4.3. The table below uses the constraints the cited papers name or measure.

The evidence supports these rows only for the tasks and grids in each cited paper. None of the papers tests very long sequences with a tuned recurrent baseline, and the forecasting paper does not test multivariate series or online learning.

Implementation sketch

The recurrent forecaster in the paper is a stack of recurrent layers followed by one dense output with one neuron per horizon step 2103.12057 §3.2.1. The cell type, layer count, unit count and return-sequence flag are the hyperparameters to grid over 2103.12057 tab. 9. Pseudocode in our own words:

# Recurrent forecaster for one window of past values  [arxiv:2103.12057 §3.2.1]
def recurrent_forecaster(cell, layers, units, return_sequences, horizon, window):
    x = window  # shape (past_history, 1)
    for index in range(layers):
        is_last = index == layers - 1
        # Earlier layers must pass the full sequence to the next layer
        # [code: experiments/models.py#L40-L49]
        keep_sequence = return_sequences if is_last else True
        x = cell(units, return_sequences=keep_sequence)(x)
    if return_sequences:
        # Flatten the sequence of hidden states before the output
        # [code: experiments/models.py#L54-L55]
        x = flatten(x)
    # One output per horizon step  [arxiv:2103.12057 §3.2.1]
    return dense(horizon)(x)

Training uses Adam and a mean absolute error loss on normalized targets in the shipped code. The paper does not name the loss 2103.12057 §3.3 main.py. Normalize each series with its own training statistics, then undo the scaling before scoring 2103.12057 §3.3.

Contradictions

• Layer count. The text says fewer stacked layers did better within the recurrent models, except for ESN 2103.12057 §4.4.2. The selection table marks the two-layer LSTM as significantly best, and gives both one and two layers for GRU as significantly best 2103.12057 tab. 17. The paper does not reconcile the two.
• Best cell versus mean. The paper's headline best cells favour LSTM, while its mean across configurations favours CNN 2103.12057 §4.1. Both statements come from the same study, so they answer different questions.
• LSTM or GRU. The motion paper picks LSTM over GRU on its own empirical test 2102.04942 §5.5. The forecasting paper finds no significant difference between the best LSTM and GRU cells 2103.12057 §4.3. The tasks differ, and neither paper resolves the other's result. The NLP survey also finds neither gated cell clearly better, on music and speech 1708.02709 §4.2.
• LSTM in the sentiment paper. Its footnote says an LSTM context model "does not perform well", yet the state-of-the-art baseline it reports is an LSTM context model 1806.06228 §3.3 1806.06228 §4.2. The paper gives no number for the footnote claim, so the two statements are not measured against each other.
• Number of clusters. The text of the clustering paper uses K = 3 for static shapes, while its appendix and code use K = 4 1708.03498 §4.1 1708.03498 app. A.1.
• CNN or LSTM on sequential short-text classification. The paper's grid caption says the CNN beats the LSTM on every dataset. On DSTC 4 the grid averages tie at 65.8, and the headline table gives the LSTM 66.2 against 65.5 for the CNN 1603.03827 tab. 3 1603.03827 tab. 4. The headline runs are single runs and the grid values are averages, so the two tables do not compare like with like 1603.03827 tab. 3 1603.03827 tab. 4.

Pitfalls

• Selection on the evaluation split. The paper reports the best WAPE over each configuration grid, and the shipped code passes the test windows as validation data 2103.12057 §4.4 main.py. The best-cell numbers are therefore optimistic. Compare cells by mean and spread 2103.12057 §4.1.
• Single runs. The paper says each configuration was trained several times 2103.12057 §4.4, but the code runs each one once and sets no random seed main.py. Exact numbers may not reproduce.
• Timing mixes hardware. The paper used mixed GPUs, and the code selects a GPU by index. Timing comparisons between cells may mix hardware 2103.12057 §4 intro main.py.
• Few series. Each series contributes one test window, so datasets with few series give noisy rankings 2103.12057 §3.3 2103.12057 tab. 8.
• Sizes left out. The motion paper gives no LSTM hidden size or depth 2102.04942 §3.2. The text-recognition paper gives no encoder hidden size 1711.04226 §4.1. Take these from the code or choose them yourself, and say which.
• Layers and normalisation only in code. The dialogue code runs a bidirectional GRU whose output it discards, and it applies batch normalisation on the IEMOCAP setting only commonsense_model.py commonsense_model.py. The paper describes neither 2010.02795 §3.4. Dropout also differs by dataset in the code train_iemocap.py.
• Survey numbers. The survey's score tables are copied from works that used different setups and do not rank the recurrent models 2003.01200 tab. 2.
• Gate equations with typos in the NLP survey. Its LSTM candidate term uses an undefined capital X, which should be the concatenated vector [h_{t-1}; x_t]. Its GRU candidate uses the update gate's weight U_z, where a separate candidate weight is expected. Check the original GRU paper before you implement either 1708.02709 §4.2.
• Recurrent variant without code. The bike-demand paper's best model is the recurrent one, and its public repository implements only the regular variant. Implement it from the text and Table 2. The gate equations come from outside the paper, which only refers to earlier work 1712.04997 Paper vs code 1712.04997 §3.3.2.
• Random RNN initial state. DeepPurpose's CNN+RNN encoders start the RNN from a random normal hidden state, not zeros. Outputs then depend on the random number state, so fix the seed if you need repeatable predictions encoders.py.
• Silent truncation. Those CNN+RNN encoders cut SMILES strings to 100 characters and protein sequences to 1000 residues. The paper does not mention this utils.py 2004.08919 app. A.
• Tracking loss sign. The paper's existence term is written as a log-likelihood without a minus sign. It has to be negated before it is minimised 1604.03635 eq. 4.
• Tracking network sizes. The paper gives 300 hidden units for the motion RNN and 500 for the association LSTM. The shipped configs use 50 for both 1604.03635 § Network size configBF.txt configDA.txt. Choose the size and say which.
• Clustering loss sign. The sign of the inter-cluster term in the paper's loss (eq. 5) differs from the code, which adds it 1708.03498 eq. 5 nem_model.py.
• Forecast loss window. The paper sums the TimeGrad loss over the prediction window only. The code computes the RNN, the target and the loss over the context plus the prediction 2101.12072 §3.1 time_grad_network.py.
• Tagger sizes are ambiguous, and there is no code. The sentence BiLSTM is given as 2 layers of 200 units each, and it is unclear whether that is per direction 2404.09383 §2.3 fn. 1. The character LSTM size is not stated. No official code is published, so these values must be chosen and reported 2404.09383 Paper vs code.
• Test-set selection in DelRec. The HAR setting is picked by a sweep that maximises test accuracy. AL has no validation split, so its number is the best checkpoint over the training epochs 2509.24852 §4.4. Both numbers are optimistic.
• DelRec delay settings. The text gives a delay learning rate of 1e-1 for Mackey-Glass, but the config uses 5e-2 2509.24852 §4.3 perf_MG.py. Sigma also never reaches zero in training: the last epoch leaves about 0.006 of its initial value on SSC 2509.24852 eq. 13. Evaluation still sets it to zero train.py.
• Motion model: derivatives and loss terms. The paper appends finite differences of orders 1 to 3, but the released code computes none 1809.03036 §3.3 motion_rnn_lm_v2.py. The paper's eq. 5 also lacks the alternating sign of a finite difference 1809.03036 eq. 5. The code averages the open-loop loss over observed and future frames, while the paper uses future frames only motion_rnn_lm_v2.py 1809.03036 §3.4.
• Rumour model: unused learning rate and dense activations. The code searches learning rates 1e-4 and 1e-3, but never passes them to the model, so Adam's default applies parameter_search.py. The paper says the dense layers use ReLU, while the code's dense layers have no activation 1806.03713 §4.1 MTL3_CV_VeracityStanceDetection.py.
• Code without a license. The GRIN repository has no LICENSE file. Do not copy its code into a page. Cite it by path and implement from the paper's equations 2108.00298 Paper vs code.
• Trajectory clustering at test time. The paper clusters 200 samples into K = 10 during training. It is unclear whether test-time prediction clusters too, and the number of rollouts per instance is not stated 2106.15004 app. A.6 2106.15004 §4.2.
• Roll-outs and no baseline in SeqGAN. The reward enters the policy gradient with no baseline subtracted, and the paper does not discuss the variance this causes sequence_gan.py. Count the roll-out cost before comparing wall-clock times rollout.py.
• No license on the SeqGAN code. The repository has no LICENSE file. Do not copy its code into a page. Cite it by path and implement from the paper 1609.05473 Paper vs code.
• Temperature is the exploit knob in the world model. At τ = 0.10 the imagined game scores 2086, but the real score is 193 [arxiv:1809.01999 tab. 2]. Choose τ by the real score. The paper's Doom controller uses τ = 1.15, while the code's default is 1.25 doomrnn.py.
• Epoch counts differ in the world model. The paper trains the VAE for one epoch 1809.01999 app. A.2. The released script runs 10 vae_train.py. Say which count you used.
• Test-split epoch selection in DialogueRNN. The IEMOCAP and AVEC scripts keep the epoch with the lowest test loss, so the reported numbers may be optimistic train_IEMOCAP.py. The script also applies hard-coded class weights that the paper's loss does not have train_IEMOCAP.py.
• EA-LSTM forget-gate bias. The code sets the initial forget bias to 5 on part of the bias vector. The paper does not state an initial bias ealstm.py.
• Qualitative driving claims. "Successful", "faster" and "better performance" in the driving paper have no measurements behind them 1704.02532 §5. Do not cite them as numerical evidence.
• Linear recurrences train differently. S5 and Mamba use a parallel scan over time, not a step-by-step loop, so the cost comparisons above do not carry over directly. Mamba's gated-RNN reading holds only for a special case: one state dimension, A = −1, B = 1 and a linear step size 2312.00752 §3.5.
• Staged sentiment model. The paper describes an end-to-end method 1806.06228 §2. The released code trains each pair model with its own four-way loss, saves its activations, and trains the trimodal model on them as fixed inputs hfusion.py. The code's bimodal GRU is 600 units followed by a 500-unit dense layer, where the paper uses a 500-unit GRU hfusion.py 1806.06228 §3.4.
• Optimiser settings in the satellite LSTM. The paper gives β2 = 0.98 for the RNN, MS-ResNet and TempCNN, while the code uses β2 = 0.999 for those three 1910.10536 §5 train.py.
• Unreliable citations in the Brain Intelligence paper. Its citation for a "dependency neural network" describes an LSTM, but the reference it gives is a dependency parser 1706.01040 §2.8. Verify citations before use.
• Sequence ends in bidirectional video models. The recurrences read frames beyond the first and last, and the paper does not say how those ends are filled 2012.02181 eq. 1. The linked repository has no model code at the cited commit, so no choice can be checked against it 2012.02181 Paper vs code.
• PredRNN decoupling weight. The paper's objective uses weight 1 on the decoupling term with a sum of squared errors. The code uses 0.1 or 0.01 with a mean-based MSE, so the penalty is much smaller relative to reconstruction 2103.09504 eq. 6 predrnn_v2.py.
• PredRNN untested pipelines. The radar and Traffic4Cast results depend on backbones and data loaders that are not in the repository, so they cannot be reproduced from it 2103.09504 Paper vs code.
• Class features in DCGRU are ambiguous. The text calls them membership indicators, yet also calls them "values of the class". The virtual-value method is not given at all 2512.06714 §3.3 2512.06714 §3.5.
• Dense activations conflict within one paper. The text says the second dense layer is linear, while the table says ReLU 2512.06714 §4.1 2512.06714 tab. 5. The card follows the table.
• Adversarial music model: paper and code differ. The code feeds the ground-truth event during pretraining, adds a second freezing threshold of 2.0, and uses 100 units where the paper uses 350 1611.09904 Paper vs code. The paper's own pretraining wording does not say which input it means 1611.09904 §3. The code has no LICENSE file, so cite it by path only.
• Survey text and survey table disagree. Some gains the NER survey quotes in its text do not match its Table 1, such as the English gain over Chiu and Nichols 1910.11470 §7 1910.11470 tab. 1. Use the table values.
• DCRNN equations versus text. The urban graph survey says DCRNN replaces the linear units of an LSTM, but its equation is in GRU form 2303.14483 §5.5.2 2303.14483 eq. 15. Implement the equation.
• Stabiliser and gate choices in xLSTM. The exponential gates overflow without the stabiliser, so use the running-maximum subtraction 2405.04517 §2.2. The released code computes mLSTM gates from q, k and v and uses a SiLU output gate with a skip and group norm, not the paper's linear gates and sigmoid output gate 2405.04517 Paper vs code. Pick one form and say which.
• Two-step flow and offset residue in BasicVSR++. The paper uses a flow from frame i to frame i-2 without defining it, and its offset equation has no bound on the residue. The released code composes the two-step flow and bounds the residue, and it also swaps the flow channels before adding them to the offsets 2104.13371 Paper vs code. An implementation that follows the paper alone will differ from the released code.

Papers

• An Experimental Review on Deep Learning Architectures for Time Series Forecasting: benchmarks LSTM, GRU, ERNN and ESN against MLP, CNN and TCN on 12 univariate datasets, with a grid of 38,000-plus models.
• Continuous online sequence learning with an unsupervised neural network model: HTM sequence memory, learned in one pass without gradients, compared with LSTM, ELM, ESN and ARIMA on a taxi series.
• Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems: attention pooling over per-step states as an alternative to a recurrent read, tested on two toy tasks.
• AON: Towards Arbitrarily-Oriented Text Recognition: bidirectional LSTM encoder and attention LSTM decoder for scene text recognition.
• Predicting Station-level Hourly Demands in a Large-scale Bike-sharing Network: A Graph Convolutional Neural Network Approach: graph CNN with a learned graph filter, and an LSTM variant, for next-hour demand at 272 stations.
• Multi-Temporal Land Cover Classification with Sequential Recurrent Encoders: bidirectional convolutional GRU and LSTM encoders for satellite image time series.
• Natural Language Processing Advancements By Deep Learning: A Survey: survey of recurrent and attention models for NLP tasks, with copied benchmark tables.
• DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction: library of compound and protein encoders, including a CNN with a bidirectional GRU or LSTM, for drug-target prediction.
• COSMIC: COmmonSense knowledge for eMotion Identification in Conversations: GRU state trackers over a dialogue, with commonsense inputs, for emotion recognition.
• Robust Motion In-betweening: LSTM generator with adversarial critics for motion transitions between keyframes.
• Online Multi-Target Tracking Using Recurrent Neural Networks: RNN motion model and LSTM data association for online pedestrian tracking.
• Neural Expectation Maximization: unrolled EM clustering with an RNN-EM variant, tested on shapes and digits.
• Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecasting: TimeGrad, an LSTM-conditioned diffusion forecaster for multivariate series.
• Low-Resource Named Entity Recognition with Cross-Lingual, Character-Level Neural Conditional Random Fields: character LSTM and BiLSTM CRF tagger with a language vector for cross-lingual transfer.
• A Taxonomy of Deep Convolutional Neural Nets for Computer Vision: survey that describes CNN-to-LSTM designs such as LRCN, with no experiments of its own.
• All-in-one: Multi-task Learning for Rumour Verification: shared LSTM with stance, detection and veracity heads on Twitter reply branches.
• A Neural Temporal Model for Human Motion Prediction: backward plan GRU and forward body GRU with a closed-loop loss ramp for long motion synthesis.
• Multimodal Trajectory Prediction Conditioned on Lane-Graph Traversals: GRU encoders over a lane graph, inside a route-based trajectory model for nuScenes.
• Filling the G_ap_s: GRIL, a bidirectional recurrent graph network with graph-gated GRUs for imputing sensor gaps.
• DelRec: learning delays in recurrent spiking neural networks: spiking recurrent layers with learned delays, trained by surrogate gradients.
• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient: LSTM policy for discrete sequences, trained against a CNN discriminator by policy gradient.
• Deep Reinforcement Learning framework for Autonomous Driving: LSTM temporal integration inside an end-to-end driving pipeline, with qualitative lane keeping only.
• Brain Intelligence: Go Beyond Artificial Intelligence: position paper with no recurrent design and no results, cited for context only.
• Recurrent World Models Facilitate Policy Evolution: MDN-RNN world model with a linear controller evolved by CMA-ES, on car racing and Doom.
• Recent Trends in Deep Learning Based Natural Language Processing: NLP survey with a short section on the simple RNN, LSTM and GRU, and benchmark tables copied from cited work; no experiments.
• DialogueRNN: An Attentive RNN for Emotion Detection in Conversations: GRU states for context, each speaker and emotion, for utterance-level emotion labels.
• Towards Learning Universal, Regional, and Local Hydrological Behaviors via Machine-Learning Applied to Large-Sample Datasets: EA-LSTM with static catchment attributes in the input gate, for rainfall-runoff over 531 basins.
• Simplified State Space Layers for Sequence Modeling: S5, a diagonal state-space layer run over time as a linear recurrence with a parallel scan.
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces: selective state-space model whose recurrence reduces to a gated RNN in one special case.
• BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond: bidirectional recurrent propagation with flow-based feature warping for video super-resolution.
• PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning: ST-LSTM with memory flowing up the layers and two decoupled memories, trained with reverse scheduled sampling.
• A Novel Deep Neural Network Architecture for Real-Time Water Demand Forecasting: GRU with k-means class features for 15-minute water demand, with a virtual-value variant.
• C-RNN-GAN: Continuous recurrent neural networks with adversarial training: LSTM generator and bidirectional LSTM discriminator for continuous MIDI events, with feature matching.
• A Survey on Recent Advances in Named Entity Recognition from Deep Learning models: survey of NER systems, with copied CoNLL and DrugNER tables for word, character and affix recurrent taggers.
• Spatio-Temporal Graph Neural Networks for Predictive Learning in Urban Computing: A Survey: survey of STGNNs for city data, with GRU-based fusion designs, and no experiments.
• BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment: second-order grid propagation and flow-guided deformable alignment on top of BasicVSR.
• xLSTM: Extended Long Short-Term Memory: exponentially gated LSTM with scalar and matrix memories, scaled to language modelling and compared with Transformers and state-space models.
• Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks: LSTM or CNN encoder for each short text, with a history window over earlier texts, for dialog act labels.
• The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approaches: survey of deep learning with the RNN, LSTM, GRU and ConvLSTM equations, and no experiments of its own.
• Recurrent Back-Projection Network for Video Super-Resolution: recurrent loop over neighbouring frames with optical flow and back-projection, for 4x video super-resolution.
• Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances: survey of emotion recognition in conversation, with recurrent context models such as DialogueRNN, and copied results.
• Graph Neural Network for Traffic Forecasting: A Survey: survey of GNN methods for traffic forecasting, including RNN-based spatiotemporal GNNs such as DCRNN, with no experiments of its own.
• Harmonic Mobile Manipulation: one RL policy with a 512-unit GRU that drives a Stretch robot's base and arm together from two RGB views and proprioception.
• Attention-over-Attention Neural Networks for Reading Comprehension: bi-GRU reader that weights query-to-passage attention by a learned passage-to-query attention, for cloze questions on CNN and CBTest.
• Modeling Temporal Dynamics and Spatial Configurations of Actions Using Two-Stream Recurrent Neural Networks: temporal and spatial LSTM streams over skeleton joints, fused by class posteriors, for action recognition.
• NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video: sparse TSDF per key-frame fragment, fused with earlier fragments by a ConvGRU, for real-time monocular reconstruction.
• Multimodal Sentiment Analysis using Hierarchical Fusion with Context Modeling: per-modality GRU context and hierarchical GRU fusion for utterance-level sentiment on CMU-MOSI and IEMOCAP.
• Self-attention for raw optical Satellite Time Series Classification: compares a Transformer with an LSTM and two CNNs on raw and preprocessed Sentinel-2 crop-type series.