Backpropagation through time

Overview

The network is unrolled over the time steps of an input sequence, and the total loss is the sum of the losses at each step. Because one weight is shared across all steps, its gradient is the sum of its influence on every step, including the influence carried forward through the state 2109.12894 §4.3 2109.12894 eq. 6 2109.12894 eq. 7. The cited paper presents this for spiking neural networks, whose state is a membrane potential that leaks and resets. Its recipe is the standard one applied to that state: store the forward pass, then propagate the error backward through the stored steps 2109.12894 §4.3.

The one obstacle specific to spiking units is the threshold. A spike is a step function, so its derivative is zero almost everywhere and undefined at the threshold. Plain backpropagation therefore gives no useful signal to neurons that have not fired 2109.12894 §4.3 2109.12894 fig. 9.

Variants

Full BPTT with a surrogate gradient

The forward pass keeps the hard threshold. On the backward pass, the step is replaced by a smooth function whose derivative is used instead 2109.12894 eq. 8 2109.12894 eq. 9. The paper describes a sigmoid form and an arctan form, and names the arctan derivative as the default choice 2109.12894 §4.3.1. The reset term is excluded from the backward pass 2109.12894 §4.3.3. The paper calls this the central recipe for training SNNs with gradients 2109.12894 §4.3. Card: Training Spiking Neural Networks Using Lessons From Deep Learning.

Caveat from the same section: the surrogate lets error reach earlier layers, but the weights on the output of a neuron that never fires still receive no update 2109.12894 §4.3.1.

Online approximations (RTRL-derived)

Full BPTT needs the whole history. The paper turns to real-time recurrent learning, which tracks the influence of each weight on each membrane potential forward in time. The influence decays by the membrane factor and gains the immediate input at each step 2109.12894 eq. 10 2109.12894 eq. 11 2109.12894 eq. 12. The gradient increment at a step is the credit at that step times the stored influence 2109.12894 §5.2. The paper names six RTRL-based SNN methods, e-prop, decolle, OSTL, ETLP, OSTTP and FPTT, and does not compare them in detail 2109.12894 §5.2.1. None of them has a card in this wiki yet, so this page does not describe them further.

Local and random-feedback rules

Not a BPTT variant as such. The paper surveys perturbation learning, random feedback, per-layer local losses and forward-forward learning as ways to avoid the backward pass, which it considers less biologically implausible but less accurate 2109.12894 §3.4 2109.12894 fig. 8. It also relates spike-timing dependent plasticity to the BPTT update for a pair of spikes 2109.12894 §5.3. Both are listed here for contrast only.

When to use which

The memory and locality statements are scaling arguments in the paper. It gives no measured numbers for them 2109.12894 §5.2.

Implementation sketch

Surrogate-gradient BPTT for one mini-batch. The forward pass uses the hard threshold, and the backward pass uses a smooth derivative in its place.

# Surrogate-gradient BPTT for one mini-batch  [§4.3] [eq. 6] [eq. 8]
# inputs: (T, B, N_in); targets: whatever the chosen loss expects
def bptt_step(inputs, targets, weights, T, beta, theta):
    # Start every unit at rest  [§2.1]
    membrane = zeros(B, N)
    spikes = zeros(B, N)
    # Keep each step's state for the backward pass  [§4.3]
    spike_record = []
    membrane_record = []
    for t in range(T):
        # Forward pass uses the hard threshold  [eq. 5]
        spikes, membrane = lif_step(
            inputs[t] @ weights, membrane, spikes, beta, theta
        )
        spike_record.append(spikes)
        membrane_record.append(membrane)
    # Loss summed over the sequence  [§3.3] [eq. 6]
    loss = output_loss(spike_record, membrane_record, targets)
    # Backward through time; the threshold derivative is replaced by a
    # surrogate, and the reset term is not differentiated  [§4.3.1] [§4.3.3]
    return backprop_through_time(loss, surrogate_derivative)

The lif_step update and the output loss are defined in the paper's discrete leaky integrate-and-fire model and its loss choices §2.1 §3.3. The surrogate is an arctan-shaped derivative, peaking at the threshold §4.3.1. The exact scale of that derivative differs between the printed formula and the authors' library (see Contradictions).

Online variant, scalar case: one unit, one weight. The influence is updated forward in time, and the gradient increment is the credit at each step times the influence 2109.12894 §5.2 2109.12894 eq. 12.

# Online influence of one weight on one membrane potential  [§5.2] [eq. 12]
def influence_step(influence, input_t, beta):
    # Decay the past influence and add the immediate influence  [eq. 12]
    return beta * influence + input_t

Contradictions

• Surrogate scale. The paper's printed arctan derivative carries a factor of 1/π at its peak. Its own integral has a derivative without that factor, and the authors' library uses the latter, with peak 1 2109.12894 §4.3.1. An implementation that copies the printed formula will produce surrogate gradients about π times smaller than the library's. Which scale works better is not tested in the paper.
• Reset in the printed listing. The paper's Python listing for the neuron sets the previous spike to zero inside the function, so the reset never takes effect 2109.12894 §2.1. The same paper says the reset uses the previous step's spike. A port of the listing as written does not implement the model in Eq. 4.

Papers

• Training Spiking Neural Networks Using Lessons From Deep Learning: tutorial and perspective on BPTT with surrogate gradients for SNNs, with an online RTRL-derived alternative; presents no new method and no experiments.