Loop tiling

Overview

A DNN layer's tensors are usually larger than the memories on a small accelerator. Loop tiling splits the loop nest into outer tiles, which stay in off-chip memory, and inner tiles, which are copied into fast on-chip buffers and fully processed there 2505.08992 §IV.A. The question it answers is how large each piece should be. The second question, which operands stay put and get reused, belongs to dataflow mapping 2505.08992 §I.

The invariant the variants share is that the result does not change: tiling only reorders the same arithmetic and the same data movement. The survey frames tiling as one of two design choices, the other being dataflow 2505.08992 §I. Its tiling text covers two-level and multi-level schemes, double buffering, and loop unrolling and pipelining of the inner loops 2505.08992 §IV.

Variants

Two-level tiling

Outer tiles stay in DRAM or HBM, and inner tiles stay in BRAM or LUTRAM. Each inner tile is fully processed before the next one is fetched 2505.08992 §IV.A. The survey gives three sizing rules: the tile must fit its on-chip buffer, its dimensions should match the number of parallel multipliers, and it should be large enough that the transfer cost is amortised 2505.08992 §IV.A. It gives no numeric threshold for any of them. Its worked example uses a 16×16 inner tile for the PE-array shape 2505.08992 §IV.A. Card: FPGA dataflow and tiling survey.

Multi-level tiling

The same idea extended to three or more tiers. The outermost tile sits in HBM, intermediate tiles in URAM or BRAM, and the innermost sub-tile in registers or LUTRAM. The survey names Versal and Alveo U280 as devices where this matters 2505.08992 §IV.D. It gives no algorithm for this scheme, so the structure above is the survey's description only.

Double-buffered tiles

Two buffers alternate roles. While tile n is computed from one buffer, tile n+1 is fetched into the other, which hides memory latency 2505.08992 §IV.C. This needs the on-chip capacity for two tiles. The survey's listing has no stop condition and no first fill, so an implementation must add both. See Dataflow mapping for a sketch that includes them.

Unrolled and pipelined inner loops

Unrolling duplicates the hardware for several inner iterations. Throughput grows with the unroll factor until LUT or DSP resources run out 2505.08992 §IV.B. Pipelining aims at an initiation interval of one, so a new iteration starts every cycle. The survey says the interval falls to one when dependencies are resolved, and that memory partitioning and tree-shaped accumulation help 2505.08992 §IV.B. Its example unroll factor of 16 is an HLS pragma, not an algorithm 2505.08992 alg. 5.

Mapper-searched tiling

Tile factors are chosen by a tool rather than the designer. In the survey's comparison, Timeloop enumerates loop nests and searches for tilings itself, while MAESTRO requires the user to supply tiling factors 2505.08992 tab. 1. See When to use which.

When to use which

The survey gives trade-offs, not a decision rule. Each row is what the cited text supports.

Tile size changes energy as well as speed. The survey reports that, for its worked layer, Timeloop energy varies by up to 11× across tilings 2505.08992 §III. It attributes this to Sparseloop, not Timeloop itself 2505.08992 §III, so check the source before relying on the figure.

The survey compares high-level synthesis, a domain-specific language and hand-written RTL on design effort and resource overhead 2505.08992 tab. 3. Those cells carry no source. Do not use them as evidence in a decision.

Implementation sketch

The common core is an outer loop over tiles, a fetch into a local buffer, a complete pass over the inner tile, and a write-back. The sketch below takes its shape from the survey's two-level scheme and the note that each inner tile is processed completely before the next fetch 2505.08992 §IV.A. It is not from any one design, and the tile shape is an input.

# Two-level tiled loop  [arxiv:2505.08992 §IV.A]
# Tile shape is an input; the survey gives no rule that derives it.

def tiled_layer(outer_tiles, tile_shape):
    # Visit each outer tile; it stays in DRAM or HBM  [§IV.A]
    for outer in outer_tiles:
        # Copy one inner tile into BRAM or LUTRAM  [§IV.A]
        inner = load_inner(outer, tile_shape)
        # Process the whole inner tile before the next fetch  [§IV.A]
        # Unrolling this loop trades LUT and DSP use for throughput  [§IV.B]
        for item in inner:
            compute(item)
        # Write the finished inner tile back to its outer tile  [§IV.A]
        store_inner(outer, inner)
    return outer_tiles

Contradictions

• Cycle counts for the same tiled layer. The survey's worked layer gives about 5×10⁵ cycles from MAESTRO and about 4.5×10⁵ from Timeloop, both for the hypothetical 256-PE array 2505.08992 §III. The survey does not explain the gap.
• Sourcing of the tile-mapping energy claim. The 11× energy variation is reported in the survey's own text but cited to Sparseloop 2505.08992 §III. The survey does not show that the claim comes from Sparseloop's results.

Papers

• Dataflow & Tiling Strategies in Edge-AI FPGA Accelerators, arXiv:2505.08992: covers two-level and multi-level tiling, double buffering, unrolling and pipelining, the MAESTRO and Timeloop tools, and four FPGA case studies. It runs no new experiments, and its numbers are quoted from other works.