Processing-in-memory

Overview

Processing-in-memory (PIM) puts computation inside or next to the memory that stores the data, so the data does not have to travel to a separate processor first 2411.10291 §V-C. The survey's motivation is that the bandwidth between an accelerator and off-chip DRAM often limits throughput, most of all when several cores need memory at the same time. Moving data also costs energy and time 2411.10291 §V-C. Better dataflow reduces the movement but does not remove the bandwidth limit, so the survey looks at moving the logic instead 2411.10291 §V-C.

The idea is about fifty years old. Early designs were held back by the technology of the time. Interest was renewed by 3D-stacked memory, such as HBM and HMC, where DRAM layers are linked by vertical vias. That gives high internal bandwidth and shorter data paths 2411.10291 §V-C. The survey cites a comparison in which one HBM part offers about 1 TB/s against 32 GB/s for GDDR5. The figure comes from another source and is not checked here 2411.10291 §V-C.

The survey is the only source on this page, and its PIM section is a literature overview plus an argument. It reports no PIM measurements. Its benchmark covers three driving models on four CPU and GPU systems and includes no PIM system 2411.10291 §V-A. Every design below is therefore a described architecture, not a measured one.

Variants

The survey groups the designs it names by where the computation sits. The grouping is ours, drawn from its descriptions 2411.10291 §V-C.

Near-memory logic under stacked DRAM

Processing elements sit on the logic die beneath a stack of DRAM. Each vertical slice of the stack (a vault, with its own controller) can reach its own processing element over the internal links, and the elements talk through an on-chip mesh. The survey describes two examples.

• Neurocube. Each processing element holds several multiply-accumulate units, a cache, a temporal buffer and a memory module for weight reuse. A programmable sequence generator drives each vault. The survey reports up to 132 GOPS for convolutional inference at 3.41 W, in a 15 nm process 2411.10291 §V-C. Card: Moving Forward.
• TETRIS. The crossbar in each vault's controller is replaced by a 2D mesh network. Each processing element has a small register file and a fixed-point ALU, and a global buffer is shared by all of them. The survey reports a 4.1x speed-up and 1.5x energy saving against a 2D accelerator with 1024 processing elements and four low-power DRAM channels 2411.10291 §V-C 2411.10291 fig. 10. Card: Moving Forward.

Bitwise operations inside the DRAM array

Here the memory cells and their control circuits do the arithmetic, with only small changes to the DRAM design. The survey describes three examples.

• Ambit. A change to the control logic lets three rows be activated together, which gives bitwise AND and OR. A change to the sense amplifier gives NOT. More complex operations, such as XOR, addition and multiplication, can then be built from these 2411.10291 §V-C. Card: Moving Forward.
• SIMDRAM. Complex operations are split into majority and NOT operations, so that arbitrary functions can be built from them. It adds a programming interface, instruction-set support and hardware parts for this. The survey says it handles multiplication and addition more efficiently than Ambit 2411.10291 §V-C. Card: Moving Forward.
• DrAcc. Built on Ambit, it adds a carry look-ahead adder inside the DRAM. It targets networks with ternary weights, and it builds multiplication for those weights from the shift-and-add circuit 2411.10291 §V-C. Card: Moving Forward.

When to use which

The survey supports these choices only in general terms. It reports no PIM measurements, so the pick depends on the constraints. Check each claim against the cited section before relying on it.

Implementation sketch

There is no code in the survey. The sketch shows only the data path that separates the placements. Its steps are the survey's descriptions 2411.10291 §V-C.

# Baseline: operands cross the off-chip DRAM bus to the accelerator  [§V-C]
def off_chip(layer, weights, activations):
    # Bandwidth to off-chip DRAM often limits throughput  [§V-C]
    w = dram_read(weights)
    a = dram_read(activations)
    return accelerator_compute(layer, w, a)

# Near-memory logic: each vault's processing element reads its own banks  [§V-C]
def near_memory(layer, vault):
    # The internal links of the stack carry the data, not the external bus
    w = vault.read_local(layer.weights)
    a = vault.read_local(layer.activations)
    return vault.pe_compute(layer, w, a)

# In-DRAM bitwise: row-wide operations run where the data is stored  [§V-C]
def in_dram(op, rows):
    # Three rows at once give AND or OR; NOT comes from the sense amplifier
    return dram_row_ops(op, rows)

The two near-memory and in-DRAM paths are the designs' own mechanisms, not general methods. The survey gives no code for them, so the sketch is not an implementation spec.

Contradictions

None known. The survey is the only source here, and it does not compare the PIM designs against each other on the same workload.

Papers

• Moving Forward: A Review of Autonomous Driving Software and Hardware Systems: survey of autonomous-driving software and hardware. It names five PIM designs (Neurocube, TETRIS, Ambit, SIMDRAM, DrAcc) and argues that future self-driving chips may include PIM-based cores.