---
kind: concept
slug: loop-tiling
title: Loop tiling
summary: >-
  Cutting a layer's loop nest into tiles so each piece fits in an accelerator's small on-chip memories, then overlapping fetches with compute and unrolling inner loops for throughput. The trade-off is on-chip buffer size against off-chip traffic and resource use, and tile sizes must be picked per layer. The only source is one FPGA survey that quotes numbers from other works.
tags: [fpga, tiling, dnn-accelerator, edge-ai, hardware]
papers: ["2505.08992"]
related: [dataflow-mapping, arithmetic-intensity, low-precision-arithmetic]
status: draft
updated_at: 2026-10-25
---

## Overview

A DNN layer's tensors are usually larger than the memories on a small
accelerator. Loop tiling splits the loop nest into outer tiles, which stay in
off-chip memory, and inner tiles, which are copied into fast on-chip buffers
and fully processed there [arxiv:2505.08992 §IV.A]. The question it answers is
how large each piece should be. The second question, which operands stay put
and get reused, belongs to [dataflow mapping](/concepts/dataflow-mapping)
[arxiv:2505.08992 §I].

The invariant the variants share is that the result does not change: tiling
only reorders the same arithmetic and the same data movement. The survey
frames tiling as one of two design choices, the other being dataflow
[arxiv:2505.08992 §I]. Its tiling text covers two-level and multi-level
schemes, double buffering, and loop unrolling and pipelining of the inner
loops [arxiv:2505.08992 §IV].

## Variants

### Two-level tiling

Outer tiles stay in DRAM or HBM, and inner tiles stay in BRAM or LUTRAM. Each
inner tile is fully processed before the next one is fetched
[arxiv:2505.08992 §IV.A]. The survey gives three sizing rules: the tile must
fit its on-chip buffer, its dimensions should match the number of parallel
multipliers, and it should be large enough that the transfer cost is
amortised [arxiv:2505.08992 §IV.A]. It gives no numeric threshold for any of
them. Its worked example uses a 16×16 inner tile for the PE-array shape
[arxiv:2505.08992 §IV.A]. Card: [FPGA dataflow and tiling survey](/papers/2505.08992).

### Multi-level tiling

The same idea extended to three or more tiers. The outermost tile sits in HBM,
intermediate tiles in URAM or BRAM, and the innermost sub-tile in registers or
LUTRAM. The survey names Versal and Alveo U280 as devices where this matters
[arxiv:2505.08992 §IV.D]. It gives no algorithm for this scheme, so the
structure above is the survey's description only.

### Double-buffered tiles

Two buffers alternate roles. While tile n is computed from one buffer, tile
n+1 is fetched into the other, which hides memory latency
[arxiv:2505.08992 §IV.C]. This needs the on-chip capacity for two tiles. The
survey's listing has no stop condition and no first fill, so an
implementation must add both. See [Dataflow mapping](/concepts/dataflow-mapping)
for a sketch that includes them.

### Unrolled and pipelined inner loops

Unrolling duplicates the hardware for several inner iterations. Throughput
grows with the unroll factor until LUT or DSP resources run out
[arxiv:2505.08992 §IV.B]. Pipelining aims at an initiation interval of one,
so a new iteration starts every cycle. The survey says the interval falls to
one when dependencies are resolved, and that memory partitioning and
tree-shaped accumulation help [arxiv:2505.08992 §IV.B]. Its example unroll
factor of 16 is an HLS pragma, not an algorithm [arxiv:2505.08992 alg. 5].

### Mapper-searched tiling

Tile factors are chosen by a tool rather than the designer. In the survey's
comparison, Timeloop enumerates loop nests and searches for tilings itself,
while MAESTRO requires the user to supply tiling factors
[arxiv:2505.08992 tab. 1]. See [When to use which](#when-to-use-which).

## When to use which

The survey gives trade-offs, not a decision rule. Each row is what the cited
text supports.

| Situation | Use | Why |
|---|---|---|
| The layer's tensors do not fit in on-chip memory | Two-level tiling | Inner tile must fit the buffer [arxiv:2505.08992 §IV.A] |
| The device has more memory tiers, such as HBM plus URAM or BRAM | Multi-level tiling | Each tier holds one tile size [arxiv:2505.08992 §IV.D] |
| Memory latency stalls the compute and a second buffer fits | Double buffering | Next tile loads during compute [arxiv:2505.08992 §IV.C] |
| Throughput must rise and spare LUT or DSP resources exist | Unrolling and pipelining | Throughput grows with unroll factor until resources run out [arxiv:2505.08992 §IV.B] |
| You know the layer and want to set tile factors yourself | User-supplied tiling, MAESTRO-style | The user supplies tiling [arxiv:2505.08992 tab. 1] |
| You want a tool to search the tiling space | Mapper-searched tiling, Timeloop-style | Mapper finds tilings [arxiv:2505.08992 tab. 1] |
| You want an automated flow to pick unroll and tile parameters | FINN-R-style flow | Parameters chosen from throughput and resource goals [arxiv:2505.08992 §V.B] |

Tile size changes energy as well as speed. The survey reports that, for its
worked layer, Timeloop energy varies by up to 11× across tilings
[arxiv:2505.08992 §III]. It attributes this to Sparseloop, not Timeloop itself
[arxiv:2505.08992 §III], so check the source before relying on the figure.

The survey compares high-level synthesis, a domain-specific language and
hand-written RTL on design effort and resource overhead
[arxiv:2505.08992 tab. 3]. Those cells carry no source. Do not use them as evidence in a
decision.

## Implementation sketch

The common core is an outer loop over tiles, a fetch into a local buffer, a
complete pass over the inner tile, and a write-back. The sketch below takes
its shape from the survey's two-level scheme and the note that each inner tile
is processed completely before the next fetch [arxiv:2505.08992 §IV.A]. It
is not from any one design, and the tile shape is an input.

```pseudocode
# Two-level tiled loop  [arxiv:2505.08992 §IV.A]
# Tile shape is an input; the survey gives no rule that derives it.

def tiled_layer(outer_tiles, tile_shape):
    # Visit each outer tile; it stays in DRAM or HBM  [§IV.A]
    for outer in outer_tiles:
        # Copy one inner tile into BRAM or LUTRAM  [§IV.A]
        inner = load_inner(outer, tile_shape)
        # Process the whole inner tile before the next fetch  [§IV.A]
        # Unrolling this loop trades LUT and DSP use for throughput  [§IV.B]
        for item in inner:
            compute(item)
        # Write the finished inner tile back to its outer tile  [§IV.A]
        store_inner(outer, inner)
    return outer_tiles
```

## Contradictions

- **Cycle counts for the same tiled layer.** The survey's worked layer gives
  about 5×10⁵ cycles from MAESTRO and about 4.5×10⁵ from Timeloop, both for
  the hypothetical 256-PE array [arxiv:2505.08992 §III]. The survey does not
  explain the gap.
- **Sourcing of the tile-mapping energy claim.** The 11× energy variation is
  reported in the survey's own text but cited to Sparseloop
  [arxiv:2505.08992 §III]. The survey does not show that the claim comes from
  Sparseloop's results.

## Papers

- [Dataflow & Tiling Strategies in Edge-AI FPGA Accelerators, arXiv:2505.08992](/papers/2505.08992):
  covers two-level and multi-level tiling, double buffering, unrolling and
  pipelining, the MAESTRO and Timeloop tools, and four FPGA case studies. It
  runs no new experiments, and its numbers are quoted from other works.