Paper · 2021-04-05 · cs.LG, cs.AR
A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays
Presents IBM's open-source PyTorch toolkit (aihwkit) that simulates analog crossbar arrays through "analog tiles": configurable device response, pulsed parallel rank updates, Tiki-Taka optimizers, hardware-aware training and PCM inference noise and drift. The paper reports no accuracy benchmarks. Its only numbers are a 2-5x training slowdown and 60 s versus 15 s per epoch for VGG-8 on CIFAR-10.
Processing-in-memoryHardware-aware traininganalog-hardwarein-memory-computingsimulationpytorchinference-noise
arXivPDFCodeMarkdown9 min read
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Some proposed AI chips store the weights of a neural network as electrical resistances inside a grid of memory cells. The grid then does the core multiply-and-add in place. The real devices are messy: values vary from device to device, respond unevenly to programming pulses, and drift over time. To judge whether a network would still work on such hardware, you need a simulator that imitates these imperfections.
This paper presents a PyTorch toolkit that does exactly that. You swap a normal layer for an "analog" layer, choose a model of the device and of the noise, and train or test the network as usual. The toolkit can also train the network with the imperfections in place, so it becomes robust to them. A separate model imitates how stored values drift after training, which matters for chips that only run inference. The code runs on the GPU.
TL;DR
- aihwkit is a PyTorch toolkit that simulates analog crossbar tiles for training and inference §1, §2.
- Device response, variation between devices and between pulses, and weight and output noise are configurable §3, §4.
- Pulsed, parallel rank updates and Tiki-Taka optimizers are included; inference noise and drift follow a PCM-calibrated model §3, §4, §5.
- The paper has no accuracy experiments; its only numbers are runtime figures §3.
Problem
Analog in-memory hardware promises fast matrix-vector products and rank updates, but device non-idealities and fixed-weight constraints change what networks can learn. Existing simulators either do not link to a modern framework (MLP-NeuroSim, RxNN) or lack fast, in-situ parallel analog update (DNN+NeuroSim) §1. Simulation therefore must run on realistically sized DNNs and datasets to inform hardware design §1.
Method
The toolkit centres on the analog tile, a 2D weight matrix stored on a resistive crossbar. A tile wraps pre- and post-processing (dynamic input scaling, DAC and ADC conversion) around the matrix-vector product §3. The forward pass follows the noisy model in eq. 1; the paper states that the noise sources can be turned on or off through the tile configuration §3.
Steps the paper describes:
- Forward pass. Input is discretised, input noise is added, the noisy weights multiply it, output noise is added, and the result is discretised again §3, eq. 1. Only parallel read-out and linear ADC are supported; iterative read-out and shift-and-add are named as possible extensions §3.
- Backward pass. It is the transposed forward pass and can carry its own noise settings, which may differ from the forward ones §3.
- Pulsed update. The rank update eq. 2 is applied with stochastic (or deterministic) pulse trains. A pulse is drawn with a probability proportional to |x_j| and |d_i|. Where pulses coincide, the crosspoint moves by a finite step that depends on the device model. The pulses are applied one after another, each following the device's response curve §3, eq. 2. Pulse length (fixed or variable) is not fixed in the paper §3.
- Device model. Each crosspoint can have device-to-device offsets in step size and bias, and pulse-to-pulse variation. Presets are calibrated on hardware data, for example ReRAM §3, fig. 3; the PCM noise model is calibrated separately §5.
- Unit cells and optimizers. A unit cell can hold several devices (for example two unidirectional devices, or a bidirectional device with references). Tiki-Taka couples two tiles so that information is exchanged between them §4, fig. 4.
- Temporal processes. Conductance decay, reset and diffusion are applied at mini-batch granularity and taken to be too small to matter within one mini-batch §4.
- Hardware-aware training. The forward pass carries the analog noise while the backward and update passes can be set to perfect. Weight-noise functions add noise to the weights during a mini-batch in the forward and backward passes, but not in the update §5.
- Inference noise. A PCM-calibrated model adds conductance-dependent programming noise, read noise and drift to trained weights, with automatic global drift compensation §5, fig. 3.
Core equations as stated in the paper:
y_i = f_\text{adc}\Big(\sum_j (w_{ij} + \sigma_\text{w}\xi_{ij}) (f_\text{dac}(x_j) + \sigma_\text{inp}\xi_j) + \sigma_\text{out}\xi_i\Big) eq. 1 — f_\text{adc}, f_\text{dac} are discretisation functions with scaling and clipping; \xi are Gaussian noise terms §3.
w_{ij}^{(t+1)} = w_{ij}^{(t)} + \lambda\, d_i x_j eq. 2 — d_i are back-propagated error signals; \lambda is not given a value in the paper §3.
Pulsed update, step by step, as the paper describes it §3, eq. 2:
# Stochastic pulsed rank update [§3] [eq. 2]
# Input: x (in_size,), d (out_size,), W (out_size, in_size)
# Output: updated W (out_size, in_size)
def pulsed_update(W, x, d, device):
# Draw pulse trains; probability scales with |x_j| and |d_i| [§3]
pulses_x = draw_pulse_train(abs(x)) # (in_size, length)
pulses_d = draw_pulse_train(abs(d)) # (out_size, length)
# Apply the pulses one after another; coincidences move the device [§3]
for k in range(length):
coincidence = outer(pulses_d[:, k], pulses_x[:, k])
# Step follows the device response curve (sign of d*x) [§3]
W = device.apply_pulses(W, coincidence, sign(outer(d, x)))
return WThe paper's PCM model is only described in words; the coefficients below come from the code, not from the paper's text.
# PCM programming noise, read noise and drift [code: pcm.py#L109-L115]
# Input: g_target (conductance, same shape as the weights)
# Output: conductance after t_inference seconds (same shape)
def program_and_age(g_target, t_inference):
# Programming noise std: quadratic in r = g / g_max [code: pcm.py#L89]
r = g_target / g_max
sig_prog = 0.26348 + 1.9650 * r - 1.1731 * r * r
# Rescale from the 25 uS coefficient reference [code: pcm.py#L114]
sig_prog = sig_prog * g_max / 25.0
gauss = sample_standard_normal(shape_of(g_target))
g_prog = g_target + sig_prog * gauss
g_prog = clamp(g_prog, min=0.0)
# Drift exponent nu is sampled per device [code: pcm.py#L126-L128]
nu = sample_drift_exponent(abs(g_prog) / g_max)
# Drift time measured from t_0 = 20 s [code: pcm.py#L138-L144]
t = t_inference + t_0
g_drift = g_prog * (t / t_0) ** (-nu)
# Accumulated 1/f read noise [code: pcm.py#L148-L153]
q_s = min(0.2, 0.0088 / max(r ** 0.65, 1e-3))
sig_noise = q_s * sqrt(ln((t + t_read) / (2 * t_read)))
g_final = g_drift + abs(g_drift) * sig_noise * randn_like(g_drift)
return clamp(g_final, min=0.0)Hyperparameters
The paper gives almost no numeric settings. Values from the paper are listed first; values from the code follow, marked as code defaults at the commit in the frontmatter.
| Name | Value | Where | Notes |
|---|---|---|---|
| Example network | one AnalogLinear(4, 2) layer | fig. 2 | 4 inputs, 2 outputs, bias on |
| Optimizer | AnalogSGD, lr = 0.1 | fig. 2 | Example only; not a tuned setting |
| Training length in example | 100 epochs | fig. 2 | Example loop only |
| Example device | ReRamESPresetDevice with default parameters | fig. 2, §2 | Values not stated in the paper |
| Tiki-Taka example | transfer_every=2, units_in_mbatch=True | fig. 4 | Example listing |
| Tiki-Taka devices | ReRamSBPresetDevice(dw_min_dtod=0.1) and dw_min_std=0.2 | fig. 4 | Second device name is misspelled in the listing; see Paper vs code |
| Eq. 2 learning rate \lambda | not stated | §3, eq. 2 | Only symbol given |
| Pulse train length | not stated | §3 | "fixed or variable" length; no value |
| Input DAC resolution | 7 bit (inp_res = 1/(2^7-2)) | io.py | Code default for IO parameters; paper says "not stated" |
| Output ADC resolution | 9 bit (out_res = 1/(2^9-2)) | io.py | Code default; paper says "not stated" |
| Output bound | 12.0 | io.py | Code default; paper says "not stated" |
| Output noise | 0.06 | io.py | Code default, σ_out in eq. 1; paper does not give a value |
| Input and weight noise | 0.0 (off) | io.py | Code defaults |
ReRAM-ES preset: dw_min, up_down | 0.00135, 0.259359 | devices.py | Fit to Gong et al. §3; the paper states no values |
| ReRAM-ES preset: w_max, w_min | 1.0, -1.0 | devices.py | |
| ReRAM-ES preset: a, b, γ up/down, A up/down | -0.5, -0.5, 5.0, 5.0, -1.18445, -0.081404 | devices.py | |
| ReRAM-ES preset: device-to-device | dw_min 0.2, up_down 0.05, w_max 0.3, w_min 0.3 | devices.py | Units not stated in the paper |
| ReRAM-ES preset: cycle-to-cycle | dw_min_std 5.0, write_noise_std 75.0 | devices.py | Units not stated in the paper |
| PCM programming polynomial | 0.26348, 1.9650, -1.1731 at 25 uS reference | pcm.py | Code only; paper gives no coefficient |
| PCM read-time constants | t_read = 250e-9 s, t_0 = 20 s | pcm.py | Code defaults |
| PCM noise scales | programming 1.0, read 1.0 | pcm.py | Code defaults |
| Transfer period (TransferCompound) | 1.0, in units of mini-batches | compounds.py and compounds.py | Code default; the Tiki-Taka example sets 2 |
Results
Author-reported. The paper has no accuracy experiments, benchmarks or ablations. Its only performance statements concern run time.
Setup: a VGG-8 network on CIFAR-10, one V100 GPU, per-epoch training time. Baseline is floating-point training §3, timing footnote.
| Setting | Metric | Baseline | This paper | Source |
|---|---|---|---|---|
| VGG-8 / CIFAR-10, V100 GPU | seconds per epoch | 15 s (floating point) | 60 s (parallel pulsed update) | §3, timing footnote |
| General DNN training | runtime relative to floating point | 1x | 2-5x longer (stated range) | §3 |
| PCM conductance over time (qualitative) | conductance evolution | measured PCM | simulated inference model | fig. 3 |
Fig. 3C compares the PCM temporal evolution with the simulated inference model. The authors note a visible mismatch for times below 1000 s, which they attribute to devices converging at different programming iterations fig. 3. The comparison is qualitative; no error numbers are given.
Paper vs code
The official repository is IBM/aihwkit (MIT). The commit in the frontmatter
is the current shallow clone, which is far newer than this 2021 paper.
Several divergences follow from that, so pin the commit when you implement
anything from the code.
- Listing names. The paper's Fig. 4 spells
ReramSBPresetDevice; the code class isReRamSBPresetDevicedevices.py. The listing, as printed, would not run. - Listing variable. Fig. 2 defines
rpu_configbut then passesconfigtoAnalogLinearfig. 2. The code's constructor takesrpu_configlinear.py. - Optimizer method. Fig. 2 calls
regroup_param_groups(model)on the optimizer fig. 2. The code defines it on the analog optimizer mixin analog_optimizer.py, andAnalogSGDis at analog_optimizer.py. - Output noise placement. Eq. 1 puts σ_out inside f_adc eq. 1. The code adds the output noise before discretisation and before clipping, which matches the equation rpu_forward_backward_pass.cpp.
- Ideal backward for inference. The paper says backward and update can be set to ideal for inference chips §5. The inference configuration sets the device to ideal and the backward IO to perfect configs.py. The update path was not checked in this pass.
- Features beyond the paper. The code has presets the paper does not describe (for example TTv2, AGAD and chopped Tiki-Taka variants) [code: src/aihwkit/simulator/presets/configs.py#L772, L1215, L994]. It also has IR-drop options io.py where the paper says only parallel read-out is supported §3.
- PCM constants. The paper says the inference model is calibrated on a 1M PCM array §5. It gives no coefficients. The coefficients used in the code (see Hyperparameters) are not explained in the code, so the calibration cannot be checked from either source.
No discrepancies were found in the forward-noise order, the input and output bound handling, or the PCM drift and read-noise formulas as far as they were read. Those formulas are not in the paper, so there is nothing to compare them with.
Pitfalls
- No accuracy evidence. The paper shows a toolkit and two runtime numbers. Accuracy claims for any network need a separate run.
- Runtime numbers depend on setup. The 60 s versus 15 s figure is for VGG-8 on CIFAR-10 on one V100, with the GPU kernels the authors tuned [§3, timing footnote]. The 2-5x range is a general statement. Expect different numbers on other GPUs, networks and device settings.
- Defaults are not in the paper. Every device, IO and PCM number that an implementer will run with comes from code defaults, and the paper gives none of them. Read the code default and override it on purpose.
- Noise is off by default for inputs and weights. The IO defaults set input and weight noise to zero, but output noise is 0.06 io.py. A model built with defaults therefore has some output noise but no input or weight noise.
- Pulse sequence is ambiguous. Pulse length (fixed or variable) and the exact pulse-generation scheme are not given in the paper §3. The code decides them; read the update kernels before reimplementing §3, eq. 2.
- Analog update is not the same as digital gradient accumulation. The paper argues that digital accumulation of outer products underestimates update noise §3. A reimplementation that accumulates gradients in digital will not reproduce the noise the toolkit models.
- Temporal processes once per mini-batch. Decay, reset and diffusion are updated at mini-batch granularity and assumed too small to matter inside one §4. This is an assumption, not something the code checks.
- Learning rate in Eq. 2 is undefined. \lambda has no value and no reference in the paper §3, eq. 2.
- The example listing will not run as printed. Its typos are listed under
Paper vs code.
mse_lossis also not imported; the caption says imports are omitted fig. 2. - Version drift. The 2021 paper describes a much smaller API than the
current code. Pin
code_commitand check that class and parameter names still exist before you use them.
Lineage
- Builds on: the RPU concept and pulsed update of Gokmen and Vlasov (2016) §1, §3; the Tiki-Taka optimizer of Gokmen and Haensch (2020) §4; the ReRAM device characterisation of Gong et al. (2018) §3; the PCM drift measurements of Joshi et al. (2020) §5.
- Compared against: MLP-NeuroSim and RxNN, which do not use a modern framework, and DNN+NeuroSim, which uses digital accumulation of updates §1, §3.
- Superseded by: not stated in this card.
Source and license
Adapted from “A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays” by Malte J. Rasch, Diego Moreda, Tayfun Gokmen, Manuel Le Gallo, Fabio Carta, Cindy Goldberg, Kaoutar El Maghraoui, Abu Sebastian, Vijay Narayanan (arXiv:2104.02184), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/IBM/aihwkit at commit 350ff34a9b6b (MIT). Any quoted code is shown under that license.