---
kind: concept
slug: backpropagation-through-time
title: Backpropagation through time
summary: >-
  Gradient-based training of a recurrent network by unrolling it over time steps and backpropagating through them. The central variant for spiking networks replaces the zero or infinite derivative of the spike threshold with a surrogate on the backward pass. Its cost is stored history, which online approximations trade for accuracy and locality. One card so far; the page is thin.
tags: [training, recurrent-networks, spiking-neural-networks, surrogate-gradients]
papers: ["2109.12894"]
related: [recurrent-neural-network, quantization-aware-training]
status: draft
updated_at: 2026-10-18
---

## Overview

The network is unrolled over the time steps of an input sequence, and the
total loss is the sum of the losses at each step. Because one weight is
shared across all steps, its gradient is the sum of its influence on every
step, including the influence carried forward through the state
[arxiv:2109.12894 §4.3] [arxiv:2109.12894 eq. 6] [arxiv:2109.12894 eq. 7].
The cited paper presents this for spiking neural networks, whose state is a
membrane potential that leaks and resets. Its recipe is the standard one
applied to that state: store the forward pass, then propagate the error
backward through the stored steps [arxiv:2109.12894 §4.3].

The one obstacle specific to spiking units is the threshold. A spike is a
step function, so its derivative is zero almost everywhere and undefined at
the threshold. Plain backpropagation therefore gives no useful signal to
neurons that have not fired [arxiv:2109.12894 §4.3] [arxiv:2109.12894 fig. 9].

## Variants

### Full BPTT with a surrogate gradient

The forward pass keeps the hard threshold. On the backward pass, the step is
replaced by a smooth function whose derivative is used instead
[arxiv:2109.12894 eq. 8] [arxiv:2109.12894 eq. 9]. The paper describes a
sigmoid form and an arctan form, and names the arctan derivative as the
default choice [arxiv:2109.12894 §4.3.1]. The reset term is excluded from
the backward pass [arxiv:2109.12894 §4.3.3]. The paper calls this the central
recipe for training SNNs with gradients [arxiv:2109.12894 §4.3]. Card:
[Training Spiking Neural Networks Using Lessons From Deep Learning](/papers/2109.12894).

Caveat from the same section: the surrogate lets error reach earlier layers,
but the weights on the output of a neuron that never fires still receive no
update [arxiv:2109.12894 §4.3.1].

### Online approximations (RTRL-derived)

Full BPTT needs the whole history. The paper turns to real-time recurrent
learning, which tracks the influence of each weight on each membrane
potential forward in time. The influence decays by the membrane factor and
gains the immediate input at each step [arxiv:2109.12894 eq. 10]
[arxiv:2109.12894 eq. 11] [arxiv:2109.12894 eq. 12]. The gradient increment
at a step is the credit at that step times the stored influence
[arxiv:2109.12894 §5.2]. The paper names six RTRL-based SNN methods, e-prop,
decolle, OSTL, ETLP, OSTTP and FPTT, and does not compare them in detail
[arxiv:2109.12894 §5.2.1]. None of them has a card in this wiki yet, so this
page does not describe them further.

### Local and random-feedback rules

Not a BPTT variant as such. The paper surveys perturbation learning, random
feedback, per-layer local losses and forward-forward learning as ways to
avoid the backward pass, which it considers less biologically implausible
but less accurate [arxiv:2109.12894 §3.4] [arxiv:2109.12894 fig. 8]. It also
relates spike-timing dependent plasticity to the BPTT update for a pair of
spikes [arxiv:2109.12894 §5.3]. Both are listed here for contrast only.

## When to use which

| Situation | Use | Why |
|---|---|---|
| Sequences fit in memory, accuracy matters most | Full BPTT with a surrogate gradient | The paper's default recipe; the backward pass is exact for the surrogate [arxiv:2109.12894 §4.3] |
| Sequence length or neuron count makes stored history too costly | An online RTRL-derived method | BPTT memory grows with neurons times time; RTRL-style influence avoids storing history [arxiv:2109.12894 §5.2] |
| Learning must be local in space and time, e.g. on-chip | An online method or a local rule | The paper ties biologically plausible learning to local information only [arxiv:2109.12894 §5.1] [arxiv:2109.12894 §5.3] |
| Static images, network can be converted from a non-spiking one | Shadow training (ANN-to-SNN conversion), not BPTT | Reported to lead on CIFAR-10 and ImageNet, but needs many time steps [arxiv:2109.12894 §4.1] |

The memory and locality statements are scaling arguments in the paper. It
gives no measured numbers for them [arxiv:2109.12894 §5.2].

## Implementation sketch

Surrogate-gradient BPTT for one mini-batch. The forward pass uses the hard
threshold, and the backward pass uses a smooth derivative in its place.

```pseudocode
# Surrogate-gradient BPTT for one mini-batch  [§4.3] [eq. 6] [eq. 8]
# inputs: (T, B, N_in); targets: whatever the chosen loss expects
def bptt_step(inputs, targets, weights, T, beta, theta):
    # Start every unit at rest  [§2.1]
    membrane = zeros(B, N)
    spikes = zeros(B, N)
    # Keep each step's state for the backward pass  [§4.3]
    spike_record = []
    membrane_record = []
    for t in range(T):
        # Forward pass uses the hard threshold  [eq. 5]
        spikes, membrane = lif_step(
            inputs[t] @ weights, membrane, spikes, beta, theta
        )
        spike_record.append(spikes)
        membrane_record.append(membrane)
    # Loss summed over the sequence  [§3.3] [eq. 6]
    loss = output_loss(spike_record, membrane_record, targets)
    # Backward through time; the threshold derivative is replaced by a
    # surrogate, and the reset term is not differentiated  [§4.3.1] [§4.3.3]
    return backprop_through_time(loss, surrogate_derivative)
```

The `lif_step` update and the output loss are defined in the paper's
discrete leaky integrate-and-fire model and its loss choices [§2.1] [§3.3].
The surrogate is an arctan-shaped derivative, peaking at the threshold
[§4.3.1]. The exact scale of that derivative differs between the printed
formula and the authors' library (see Contradictions).

Online variant, scalar case: one unit, one weight. The influence is updated
forward in time, and the gradient increment is the credit at each step times
the influence [arxiv:2109.12894 §5.2] [arxiv:2109.12894 eq. 12].

```pseudocode
# Online influence of one weight on one membrane potential  [§5.2] [eq. 12]
def influence_step(influence, input_t, beta):
    # Decay the past influence and add the immediate influence  [eq. 12]
    return beta * influence + input_t
```

## Contradictions

- **Surrogate scale.** The paper's printed arctan derivative carries a
  factor of 1/π at its peak. Its own integral has a derivative without that
  factor, and the authors' library uses the latter, with peak 1
  [arxiv:2109.12894 §4.3.1]. An implementation that copies the printed
  formula will produce surrogate gradients about π times smaller than the
  library's. Which scale works better is not tested in the paper.
- **Reset in the printed listing.** The paper's Python listing for the
  neuron sets the previous spike to zero inside the function, so the reset
  never takes effect [arxiv:2109.12894 §2.1]. The same paper says the reset
  uses the previous step's spike. A port of the listing as written does not
  implement the model in Eq. 4.

## Papers

- [Training Spiking Neural Networks Using Lessons From Deep Learning](/papers/2109.12894): tutorial and perspective on BPTT with surrogate gradients for SNNs, with an online RTRL-derived alternative; presents no new method and no experiments.