AI-ANNE: (A) (N)eural (N)et for (E)xploration: Transferring Deep Learning Models onto Microcontrollers and Embedded Systems

In plain words

Small neural networks are usually trained and run on powerful computers with large numerical libraries. A microcontroller, the tiny chip inside a sensor or a small gadget, has far less memory and no such libraries. This working paper shows how to train a small network on a normal computer, copy its learned numbers (the weights and biases) onto a Raspberry Pi Pico, and run it there using MicroPython, a lightweight version of Python.

The paper builds the network from its basic parts, which are neurons, layers, and activation functions, written as plain Python lists and loops. It then runs two small example networks on an Iris flower classification task.

The paper reports only accuracy for these two examples and gives no speed or energy measurements. Treat the claims about efficiency as the authors' framing.

TL;DR

• Ports a pre-trained Keras/TensorFlow dense network to MicroPython on a Raspberry Pi Pico, implementing neurons, layers, and activations by hand §I app. A app. C.
• Two example nets: 8 neurons (2-3-2-1) at 90% and 6 neurons (3-2-1) at 95% accuracy on a binary Iris task §IV.
• Accuracy is the only metric. No baseline, no timing, no memory numbers.
• The public code differs from the paper in several activation choices and in how weights are laid out; see Paper vs code before implementing.

Problem

Frameworks such as TensorFlow and Keras need resources a microcontroller does not have, so trained networks cannot run there as they are §I. Training on the device is described as a complex challenge §I. The Raspberry Pi Pico (RP2040: 133 MHz, 264 KB SRAM, 2 MB flash) and the Pico 2 (RP2350: 150 MHz, 520 KB SRAM, 4 MB flash) are the target boards §II. The paper assumes the networks are small enough to be written as explicit MicroPython loops, and that training happens on a larger system beforehand §III.d §IV.

Method

The workflow is: pre-train in Python, export the parameters, then rebuild the forward pass on the board.

1. Pre-train a small dense network in TensorFlow and Keras on a computer. The paper names the activations used per layer for each example §IV.
2. Export each layer's weights and biases in a fixed order into MicroPython lists. The paper shows the 8-neuron and 6-neuron values tab. 2 tab. 4.
3. Implement the building blocks in MicroPython: zero and add helpers, transpose, matrix creation, a dense layer that loops over units, and a neuron that sums weighted inputs, adds the bias and applies an activation app. A app. C.
4. Implement the activation functions Sigmoid, ReLU, Leaky ReLU, Tanh and Softmax app. B. The formulas are in §III.d.
5. Feed the standardized Iris data through the layers in order. The example transposes the test matrix so each row of the input is one feature across all samples tab. 3 tab. 5.
6. Compute accuracy from the confusion matrix of predicted against true classes §III tab. 1.

A single neuron computes a weighted sum of its inputs, adds a bias, and applies an activation §III.a:

y = f\left(\sum_i w_i x_i + b\right) §III.a

The activation functions are defined as follows. The paper does not number these equations §III.d:

\mathrm{Sigmoid}(x) = \frac{1}{1 + e^{-x}} §III.d

\mathrm{ReLU}(x) = \begin{cases} x & x \geq 0 \\ 0 & x < 0 \end{cases} §III.d

\mathrm{LeakyReLU}(x) = \begin{cases} x & x \geq 0 \\ \alpha x & x < 0 \end{cases} §III.d

\tanh(x) = \frac{e^x - e^{-x}}{e^x + e^{-x}} §III.d

\mathrm{Softmax}(x_i) = \frac{e^{x_i}}{\sum_j e^{x_j}} §III.d

Dense layer, written in the paper's batch form. Weights are passed as (units, inputs), which is the transpose of the tables; see Pitfalls.

# Dense layer over a batch, as the paper's code does  [tab. A] [tab. C]
# inputs: (n_in, batch); weights: (n_units, n_in); biases: (n_units,)
# output: (n_units, batch)
def dense(n_units, inputs, weights, biases, activation):
    # one neuron per output unit  [tab. A]
    outputs = []
    for unit in range(n_units):
        # weighted sum over the inputs, for every sample  [§III.a] [tab. C]
        total = zeros(batch)
        for i in range(n_in):
            total = total + weights[unit][i] * inputs[i]
        # add the bias before the activation  [§III.a] [tab. C]
        total = total + biases[unit]
        # apply the activation to the vector over samples  [tab. C]
        outputs.append(activate(total, activation))
    return outputs

Forward pass of the 8-neuron example as the paper writes it. The values of w1 to w4 and b1 to b4 are in tab. 2. The paper's code transposes them first ki-enna-b.py.

# Forward pass of the 8-neuron example  [tab. 3]
# x: standardized features, shape (4, batch)
def predict_8_neurons(x):
    # input layer: 2 units, ReLU  [tab. 3]
    hidden_1 = dense(2, x, w1, b1, 'relu')
    # first hidden layer: 3 units, tanh  [tab. 3]
    hidden_2 = dense(3, hidden_1, w2, b2, 'tanh')
    # second hidden layer: 2 units, softmax  [tab. 3]
    hidden_3 = dense(2, hidden_2, w3, b3, 'softmax')
    # output layer: 1 unit, sigmoid  [tab. 3]
    return dense(1, hidden_3, w4, b4, 'sigmoid')

The class decision uses a threshold of 0.5. The paper does not state it; the code does ki-enna-b.py.

# Binary decision from the sigmoid output  [code: ki-enna-b.py#L1112]
# probs: output of the single output unit, one value per sample
def classify(probs):
    # the paper does not state this threshold
    return [1 if p > 0.5 else 0 for p in probs]

Hyperparameters

Results

Author-reported. Setup: binary Iris task (Versicolor vs Virginica), a test set of 20 samples implied by tab. 1, pre-trained TensorFlow/Keras nets exported to MicroPython. No baseline is reported. The paper names AIfES, another on-device framework, as a comparable approach §I but gives no comparison numbers.

The paper's confusion matrix is an illustrative example for the 6-neuron net tab. 1. The paper prints the counts with leading zeros.

Paper vs code

Checked against the repository statistical-thinking/KI.ENNA at code_commit (MIT, LICENSE). The README calls this version 2.0.1 [README.md#L131], so it is later than the paper.

• Weight values match. The 8-neuron and 6-neuron weights and biases in tab. 2 and tab. 4 match the values in the code on the entries I compared. ki-enna-b.py ki-enna-b.py
• 8-neuron activations differ. The paper uses ReLU, tanh, softmax and sigmoid tab. 3. The code uses tanh, sigmoid, ReLU and sigmoid ki-enna-b.py.
• 6-neuron activations differ. The paper uses ReLU, sigmoid and sigmoid tab. 5. The code uses ReLU in all three layers ki-enna-b.py.
• Weights are transposed in the code. The paper's tables store weights as inputs x units, but its dense indexes them per unit tab. 2 app. A. The code transposes them before use ki-enna-b.py ki-enna-b.py. The paper does not mention this step.
• The Leaky ReLU branch calls ReLU. The neuron function sends the leaky_relu branch to relu and ignores alpha tab. C ki-enna-b.py.
• The softmax branch calls tanh. The neuron function sends softmax to tanh tab. C ki-enna-b.py. The softmax helper in the code is a separate function ki-enna-b.py that is not used by the example nets.
• Accuracy and loss on the device are typed in. The device screen shows fixed strings for accuracy, loss, epochs and learning rate ki-enna-b.py ki-enna-b.py. They are not computed at run time.
• Confusion matrix order differs. The paper's table lists actual positive first tab. 1. The code builds [[TN, FP], [FN, TP]], with actual negative first ki-enna-b.py.
• A third network exists in the repository. ai-anne-b.py contains a 4-4-2-1 net with leaky ReLU, sigmoid and sigmoid, input min/max scaling, and a different weight layout (no transpose) ai-anne-b.py ai-anne-b.py. The paper does not describe it.
• The license statement differs. Appendix D names the German Free Software License app. D. The repository's LICENSE is MIT, and the README says the 2021 version used the German Free Software License while 2.0.1 is MIT [README.md#L62] [README.md#L130-L132] [LICENSE]. The card records the repo license as MIT.
• Training code is not in the clone. The paper says the repository holds Jupyter notebooks for pre-training §Open Source Code. The clone at code_commit has no .ipynb files, so the training recipe cannot be checked.

Pitfalls

• Weight layout. The tables are inputs x units tab. 2 tab. 4. The dense in the paper indexes weights per unit app. A. Transpose before use, as the repository does ki-enna-b.py. ai-anne-b.py takes the other layout ai-anne-b.py.
• Softmax normalises over samples, not units. The neuron function applies the activation to the vector over samples for one unit tab. C. As written, softmax therefore normalises across the batch for each unit. The paper's text describes normalising across classes §III.d. An implementer who follows the code gets the batch behaviour; one who follows the text gets the class behaviour. The paper does not say which it intends.
• The paper's softmax code is wrong. The code in tab. B takes max(x[val]) and exponentiates val - max_x over indices, not values. Use the formula in §III.d with the maximum subtracted per sample.
• Leaky ReLU is not implemented. tab. C and the code send the leaky_relu name to ReLU. Implement alpha yourself if you need it.
• Overflow in sigmoid and tanh. The formulas in tab. B call exp on the raw input. Large negative inputs overflow the float range. ai-anne-b.py branches on the sign of the input to avoid this ai-anne-b.py.
• Accuracy rests on 20 samples. One wrong sample moves accuracy by 5 percentage points tab. 1. The paper does not describe a split or cross-validation.
• Label mapping and threshold are not stated. Which class is 1 and the 0.5 cut-off are not in the paper. The code uses 0.5 ki-enna-b.py.
• Preprocessing is not stated. The Iris input is pre-standardized in the code ki-enna-b.py. The paper gives no means or scales.
• The two example nets do not match the code. Activations differ from the paper (see Paper vs code). Results from the paper's stated setup should not be assumed to hold for the code without rerunning.
• Cost is not measured. The paper claims low latency and low energy use but reports no timing, memory or power data. Memory use on the 264 KB Pico is not measured either §II.
• Interpreter cost. The layers are Python loops over lists. The paper does not measure the cost of this.

Lineage

• Builds on: none stated. The paper cites TinyML and MicroPython background only §I §II.
• Compared against: AIfES (Wulfert et al., 2024; no arXiv id in the paper) §I. The paper says AIfES outperforms similar tools in speed and memory for certain tasks, without giving the data.
• Superseded by: not known from this wiki.