Machine Learning for Microcontroller-Class Hardware: A Review

In plain words

Microcontrollers are the small, cheap, battery-powered chips inside sensors, wearables and gadgets. They have far less memory and power than a phone, so a typical machine-learning model does not fit on them. This review asks how an engineer should get a model onto such a chip. It describes a loop: collect and clean the sensor data, optionally shrink its inputs, pick a small model, search for a better one if needed, shrink the model further, turn it into code for the chip, and sometimes keep training it on the device.

The authors go through each step and list the tools and techniques used for it. They also compare published results per application, such as gesture recognition, keyword spotting and anomaly detection. The review does not test anything new. Its value is the map of options and the trade-offs that the reported numbers show.

TL;DR

A review of TinyML, meaning machine learning on microcontrollers. It proposes a closed-loop development workflow fig. 1 and surveys each stage: data engineering, feature projection, compression, lightweight blocks, neural architecture search, software suites, online learning and federated learning §III–§IX. Quantitative comparisons are collected from prior work, not measured by the authors §X–§XI. Use it as a map of options and their trade-offs, and check the original sources before relying on any single figure.

Problem

Conventional models need far more memory, compute and power than microcontrollers offer §I. A typical microcontroller in the review's table has 2–1024 kB of SRAM and 32–2048 kB of eFlash, at 0.1–0.3 W, compared with 16 GB HBM and 250 W for a cloud GPU tab. I. Sending sensor data to a server is not always possible, because of network bandwidth, latency, power or privacy limits §I.B–§I.C. Earlier surveys describe the stages qualitatively but do not compare methods quantitatively §I.

Method

The review's central object is a workflow, not an algorithm. It is shown as a loop of nine labelled stages on one sensor-to-microcontroller diagram fig. 1. Stages 3 to 8 repeat until the desired performance is reached fig. 1 caption. Phase 1 covers data, feature projection, the model zoo, neural architecture search (NAS) and hardware profiling. Phase 2 covers porting the model into a TinyML software suite, compression, code generation, and on-device training or federated learning §II.

The sketch below restates that loop in code. It is written from the workflow description in fig. 1 and §II; the paper contains no code for it, and the helper functions are placeholders.

# Closed-loop TinyML development  [fig. 1] [§II]
# Input:  raw sensor streams; per-device SRAM, flash and latency budget
# Output: C code for the microcontroller

def develop_tinyml_model(raw_streams, device_budget):
    # Collect, label and clean sensor data  [§III] [fig. 1]
    dataset = engineer_data(raw_streams)
    # Optionally reduce the input dimension first  [§IV] [fig. 1]
    dataset = project_features(dataset)
    # Pick a candidate from a model zoo or NAS search space  [§VI] [§VII]
    candidate = search_candidate(dataset, device_budget)
    # Estimate hardware cost by proxy or on the device  [§VII.E] [fig. 1]
    metrics = profile_hardware(candidate, device_budget)
    # Repeat search and profiling until the budget is met  [fig. 1 caption]
    while not meets_budget(metrics, device_budget):
        candidate = search_candidate(dataset, device_budget, metrics)
        metrics = profile_hardware(candidate, device_budget)
    # Compress the model: pruning, quantization, encoding  [§V] [fig. 1]
    compressed = compress(candidate)
    # Port to a software suite and generate C code  [§VIII] [fig. 1]
    c_model = generate_c_code(compressed)
    # Optionally fine-tune on the device as data shifts  [§IX] [fig. 1]
    return c_model

Stage details, each with the review's own citation:

1. Data engineering §III. Tools range from big-data stacks to TinyML data tools such as Edge Impulse, MSWC, SensiML DCL, Qeexo AutoML and Plumerai Data. The review compares them on data type, collection mode, labelling, augmentation, visualisation and cleaning tab. III.
2. Feature projection §IV. Linear methods (SVD, NMF, ICA, LDA, PCA), nonlinear methods (kernel PCA, t-SNE, UMAP, autoencoders) and hand-designed domain features are described. The review says nonlinear methods are usually too costly for real-time microcontroller use §IV.B.
3. Compression §V. Quantization (post-training and quantization-aware), pruning, structured sparsity and encoding are described. Compression frameworks are compared in tab. IV. Deep Compression is cited for pruning, quantization and Huffman coding together §V.
4. Lightweight ML blocks §VI. Sparse projection to prototypes (Bonsai, ProtoNN), fire modules, depthwise separable convolution, low-rank or quantized recurrent cells (FastRNN, FastGRNN), temporal convolutions, and small attention or transformer variants are described fig. 2 §VI.A–§VI.F.
5. Neural architecture search §VII. The review classifies search spaces as layerwise, cellwise or hierarchical §VII.A. Strategies are reinforcement learning, gradient-based, evolutionary with weight sharing, Bayesian optimisation and training-free methods §VII.B. Cost functions and hardware profiling methods are described §VII.C–§VII.E. NAS frameworks for microcontrollers are listed in tab. V.
6. Software suites §VIII. Operator fusion, in-place operators, loop transformations and data reuse are described fig. 3 §VIII.A. Frameworks are compared in tab. VIII; TensorFlow Lite Micro is described with its arena allocator fig. 4 §VIII.B.
7. On-device learning §IX. Last-layer transfer learning, specialised operators and continual-learning techniques are described. Frameworks are compared in tab. IX.
8. Federated learning §IX. Strategy, communication stack, scalability, privacy and client hardware are compared in tab. X.

Applications are summarised in tab. XI (image recognition), tab. XII (visual wake words), tab. XIII (keyword spotting and speech), tab. XIV (anomaly detection), tab. XV (activity and gesture), tab. XVI (mHealth) and tab. XVII (face detection) §X.

Hyperparameters

This review has no training hyperparameters. The table lists the platform and benchmark constants that the review uses as its reference.

The benchmark rows are copied from the review's table and are not verified against the MLPerf Tiny release.

Results

Author-reported in the sense that the review collects these from cited sources. No experiments are run by the authors. Setup, dataset and baseline are given per row.

The paper's Table XIX has more rows than these; the average figures quoted in §XI.B are the ones reported in the text.

Paper vs code

No official code. The paper is a review and publishes no repository, and the arXiv record carries no code link. A GitHub repository search for the title's keywords and the authors' names returned no results.

Pitfalls

• Mixed metrics across tables. The review puts top-1 accuracy, top-5 accuracy, mAP, F1, AUC and mean AUC side by side in the same tables tab. XI–XIV. Compare numbers only within one row or one table, never across tables.
• Numbers come from different hardware, datasets and software. Most entries are from cited papers that used different microcontrollers and toolchains, and the review does not normalise them. The speedups and memory savings in §V–§VIII are not measured on one device.
• Pruning and quantization averages hide spread. The 13.6×, 3.9× and 4.9% vs 0.4% figures are averages over the models in tab. XIX. The text states that pruning gives less latency gain than quantization on microcontrollers §XI.B.3, so averages on size do not transfer to speed.
• Support limits the choice. Many NAS frameworks, online-learning frameworks and suites support only CNN backbones or only neural networks §XI.C tab. VIII tab. IX. An engineer who picks a technique may find the toolchain cannot run it.
• Search cost is not priced out. The NAS comparison table tab. VI gives search costs in GPU hours for ImageNet, not for microcontroller targets. Hardware-in-the-loop profiling is described as slowest but most accurate tab. VII.
• Quantization support on hardware. The review notes that a microcontroller's SIMD width may not suit very low bitwidths, and that some compilers do not support sub-byte bitwidths §V.B.3 §XI.B.3.
• Security and privacy claims are not tested. Adversarial and privacy threats are described in outline §XII.E; no measured results are given.
• Figures are not reproduced. Figures 1–4 are described in words above; the card does not include them.

Lineage

• Builds on: earlier TinyML surveys cited in the introduction, including the qualitative reviews by Sanchez-Iborra and Skarmeta [3], Dutta and Bharali [4], Ray [6] and Shafique et al. [7]; the review itself says these lack quantitative comparison §I.
• Compared against: the same surveys, plus the MLPerf Tiny benchmark suite [9] that defines the benchmark rows in tab. II.
• Superseded by: not known from content on this wiki.