---
kind: concept
slug: model-partitioning
title: Model partitioning
summary: >-
  Splitting one neural network across several devices so that no device has to hold a whole layer. The core idea is to give each worker a share of a layer's output positions or neurons, sized by its capability, and to route activations between workers. The trade-off: communication grows with the number of workers, and the one source here reports total latency rising as boards are added.
tags: [inference, microcontroller, tinyml, distributed-inference, memory, latency]
papers: ["2605.09357"]
related: [on-device-inference, computation-offloading, memory-planning-for-inference, batchnorm-folding, post-training-quantization]
status: draft
updated_at: 2026-11-07
---

## Overview

Model partitioning means running one trained network on several devices
at once, with each device holding only part of the weights and computing
only part of the outputs. The reason to do it is memory. A single device
may have enough flash for the weights but not enough working RAM for the
intermediate activations of one layer [arxiv:2605.09357 §1]. Cutting
between layers does not help when one layer alone is larger than a
device's RAM [arxiv:2605.09357 §1, Challenge 1].

The one source in this set cuts finer than a layer. It splits each
convolution by output position and each linear layer by output column
[arxiv:2605.09357 §4.2]. Workers receive their weight fragments once at
deployment. A coordinator then sends each worker the inputs it needs for
every layer and collects the partial outputs [arxiv:2605.09357 §3,
§4.4]. No worker holds the full activations or weights of a layer
[arxiv:2605.09357 §4.4].

The cost is communication. Every intermediate result passes through the
coordinator, so each activation crosses the network at least twice
[arxiv:2605.09357 §6.2]. On the testbed, computation time fell from 15.37 s on three
boards to 7.07 s on eight, while communication time rose from 27.60 s to
49.82 s [arxiv:2605.09357 §7.2.3].

## Variants

### Rating-weighted split across microcontrollers

Each worker gets a share of every layer that is proportional to a
capability rating. The rating combines clock frequency, per-kilobyte
communication delay, bandwidth and a communication coefficient
[arxiv:2605.09357 §5.1, eq. 5]. Convolution output positions are cut
into contiguous runs, and linear-layer columns into contiguous runs, one
run per worker [arxiv:2605.09357 §4.2, alg. 1, alg. 2]. Weight fragments
are sized the same way, with an overflow rule for workers whose storage
is too small [arxiv:2605.09357 §5.3, eq. 6–7]. Activations are routed
between consecutive layers by a map that records which inputs each
worker needs [arxiv:2605.09357 §4.3, alg. 3]. The paper deploys this on
eight Teensy 4.1 boards, each with a 600 MHz Cortex-M7 [arxiv:2605.09357
§6.1]. Card: [Split CNN Inference on Networked Microcontrollers](/papers/2605.09357).

Other partitioning schemes named in the paper for GPUs and phones assume
each worker can hold its sub-matrix or all channels of a feature map
[arxiv:2605.09357 §2].
This variant is aimed at the case where it cannot. The paper gives no
number for a single board, since one board cannot hold the model
[arxiv:2605.09357 §7.1, §7.2.1].

### Even split as a baseline

The same paper uses an even split, where every worker gets the same
share regardless of speed, and a frequency-only split as baselines
[arxiv:2605.09357 §7.2.1]. These are the simplest versions of the
variant above, and the paper's comparison in tab. 2 is the reference for
how much the rating adds [arxiv:2605.09357 tab. 2].

## When to use which

| Situation | Use | Why |
|---|---|---|
| One layer's peak RAM exceeds a device's budget | Split within layers, as in the paper | Layer-boundary cuts cannot fix a single oversized layer [arxiv:2605.09357 §1] |
| All boards have the same clock and link | Even split may be enough | With equal boards and no delay, even and optimized both took 9.80 s in case 1 [arxiv:2605.09357 tab. 2] |
| Boards differ in clock speed | Rating-weighted split, then measure | Optimized was fastest or tied for fastest in 7 of 8 cases; the exception is case 2, where frequency-only was faster [arxiv:2605.09357 tab. 2] |
| Links have different delays | Rating-weighted split, then measure | In the four cases with injected delays, optimized was fastest [arxiv:2605.09357 tab. 2] |
| Many workers on a slow network | Fewer workers with enough RAM | Total latency rose from 42.97 s on 3 boards to 56.89 s on 8 [arxiv:2605.09357 §7.2.3]. The paper does not test a fewer-workers setting as advice |
| Accuracy must be guaranteed | Measure on your own model | The paper reports no accuracy number for the split, quantised model [arxiv:2605.09357 §9] |

This set has one paper, so only the rating-weighted variant is covered.
The comparison with an even split rests on that paper's numbers alone. A reader should not expect the same gains
on other boards or links.

## Implementation sketch

The common core is to assign each worker a contiguous run of output
positions, sized by its rating [arxiv:2605.09357 alg. 1]. The linear case
is the same with columns in place of positions [arxiv:2605.09357 alg. 2].

```pseudocode
# Assign output positions of a conv layer to workers  [alg. 1]
# Input: output shape (c, h, w), ratings R, number of workers N
def split_conv(c, h, w, ratings, n_workers):
    start = 0
    for r in range(n_workers):
        # Quota of positions for worker r, proportional to its rating
        quota = ratings[r] / sum(ratings) * c * h * w
        i = start
        while i - start < quota:
            # Output channel that position i belongs to
            kernel = i // (h * w)
            if kernel not yet on worker r:
                send kernel weights to worker r
            else:
                increment usage count of kernel on worker r
            i = i + 1
        start = i
```

Notes on where the variant differs from the core:
- The released code rounds the per-unit size up, while alg. 1 uses the
  unrounded quota [arxiv:2605.09357 §4.2, code: pc_code/Algorithms/src/lib/operations.rs#L8].
- The rating in eq. 5 needs values for the communication coefficient,
  delay and bandwidth that the paper does not give
  [arxiv:2605.09357 §5.2].

## Contradictions

- **RAM budget of one board.** The paper gives the Teensy 4.1 as having
  1 MB of on-chip RAM [arxiv:2605.09357 §6.1]. Its evaluation calls
  512 KB the typical single-MCU RAM [arxiv:2605.09357 §7.2.1]. The text
  does not say which budget the per-layer RAM check uses.
- **Optimized versus frequency-only.** In case 2 the frequency-only split
  is faster (12.40 s) than the rating-weighted one (12.52 s), while in
  case 5 the frequency-only split is slower than an even split
  [arxiv:2605.09357 tab. 2]. The paper's text presents the rating as the
  better method, so these rows show it is not better in every case.
- **Rating formula in the released code.** The paper's rating does not
  appear in the released source. The simulation gives every worker an
  equal portion [arxiv:2605.09357 §5.2, code: pc_code/Simulation/src/main.rs#L20].
  Table 2's "Optimized" column therefore cannot be checked from the code
  alone.
- **Input size.** The paper evaluates 112 × 112 × 3 inputs
  [arxiv:2605.09357 §7.1]. The released code crops to 224 × 224
  [code: pc_code/Algorithms/src/lib/util.rs#L49-L57].

## Papers

- [Split CNN Inference on Networked Microcontrollers](/papers/2605.09357): splits MobileNetV2 across up to eight Teensy 4.1 boards by output position and weight column, with rating-weighted shares. Reports latency and RAM on the testbed, with no accuracy numbers.