A Machine Learning-oriented Survey on Tiny Machine Learning

In plain words

Tiny machine learning means running learned models on very small, low-power chips, such as microcontrollers. These chips have very little memory and power, so a model that works well on a server often does not fit. This paper is a map of the field. It looks at how people build such systems, which techniques shrink or redesign models to fit, how the learning itself can be adapted to small devices, and which chips and software tools exist. It also counts how often each technique appears in the literature and lists open problems, such as the lack of a shared benchmark.

TL;DR

A systematic review of TinyML from the machine-learning side, covering papers from January 2018 to September 2023 §3. It sorts the work into three development workflows §4, a taxonomy of model optimization, model design and learning algorithms §5, fig. 5, and the hardware and software tools in use §6. It is useful as a reading map and a list of research gaps. It is not a source of implementable numbers, because it reports no experiments of its own.

Problem

TinyML models must run on devices with memory in the kilobyte range and power in the milliwatt range, often with real-time deadlines §1. Modern networks grow to billions of parameters, and the energy needed to run them grows with their size §1. The authors state that the literature is spread across hardware, software and learning topics, and that no earlier systematic review covers the ML-oriented side §1 §3.

Method

This is a literature survey. It has no algorithm of its own, so the card lists what the paper does and the procedure it follows.

Selection procedure. The authors follow the PRISMA guidelines for systematic reviews §3. They searched five sources, including Google Scholar and IEEE Xplore §3, with seven keyword phrases built around "TinyML", "efficient machine deep learning", and "embedded / edge / MCU machine deep learning" §3. Only English-language papers in peer-reviewed venues were kept, with software tools as the stated exception §3. Steps and counts are in the pseudocode below §3 fig. 3.

# PRISMA-style selection  [§3] [fig. 3]
# Input:  keyword-search records from five databases, Jan 2018 - Sep 2023
# Output: eligible papers used in the survey

def select_papers(records):
    # Remove duplicate records  [§3]
    unique = remove_duplicates(records)
    # Remove records marked ineligible by the automation tool  [§3]
    tool_eligible = drop_tool_ineligible(unique)
    # Remove records that are not accessible, e.g. paid access  [§3]
    accessible = drop_inaccessible(tool_eligible)
    # Title and abstract screening  [§3]
    shortlist = screen_title_abstract(accessible)
    # Papers not retrievable are dropped; the rest are eligible  [§3]
    return retrieve_full_text(shortlist)

Workflows. The paper splits a TinyML build into three workflows §4 fig. 4. In the ML-oriented workflow, the hardware is fixed and ML experts go through model design, model optimization, on-host evaluation, target deployment and target evaluation §4. The HW-oriented workflow starts from hardware design and loops through target deployment (mostly in simulation) and target evaluation on physical devices §4. The co-design workflow has two groups, ML and hardware specialists, who define the requirements together, analyse the options, co-design the system and then evaluate it §4.

Taxonomy. The learning-side taxonomy has three areas §5 fig. 5:

• Model optimization §5.1: pruning (weight, neuron and structured variants, applied during or after training) §5.1.1; quantization, either quantization-aware training (QAT) or post-training quantization (PTQ), with dynamic-range and full-integer variants of PTQ §5.1.2; knowledge distillation with response-, feature- and relation-based knowledge and offline, online and self-distillation schemes §5.1.3; and hyperparameter optimization by grid, random and Bayesian search §5.1.4.
• Model design §5.2: neural architecture search (reinforcement learning, evolutionary and gradient-based search, with microcontroller-specific variants) §5.2.1; rational activation functions, which are trainable activations §5.2.2; depthwise-separable convolution §5.2.3; and attention §5.2.4.
• Learning algorithms §5.3: supervised §5.3.1, unsupervised and anomaly detection §5.3.2, self-supervised §5.3.3, deep reinforcement learning §5.3.4, weakly supervised §5.3.5, meta-learning split into meta-representation, meta-optimizer and meta-objective §5.3.6, and continual learning with regularization-based, replay-based and dynamic-architecture methods §5.3.7.

Hardware and software. The paper compares CPUs, GPUs, FPGAs and TPUs by advantages and disadvantages §6.1–6.4 tab. 4. It then lists seven software frameworks: TensorFlow Lite Micro, uTensor, Edge Impulse, the Microsoft Embedded Learning Library, X-CUBE-AI, uTVM and MinUn §6.5. The paper gives a one-paragraph description of each.

Discussion. Section 7 gives the authors' reading of the counts: model optimization has the most work, NAS dominates model design, unsupervised and continual learning dominate learning algorithms, self-supervised and weakly supervised learning are underexplored, and co-design is expected to lead future work §7. Open issues are benchmarking, memory limits, data quality, the lack of an accepted TinyML baseline model, and the lack of public datasets §7.

Hyperparameters

Not applicable. The paper has no experiments of its own §1 §8. The only numbers it gives are the selection counts in §3 and the popularity counts in the table below.

Results

Author-reported. The paper gives no accuracy, latency or energy results. Its quantitative content is the review counts and the popularity labels. The popularity labels (H, M, L) are the authors' own and are not defined in the paper (see Pitfalls).

Selection counts, from §3:

Technique popularity. The paper gives a popularity label and a table of cited works per technique and period. The "cited works" counts below were counted by us from the table source, not stated in the paper.

Hardware comparison, in the paper's terms tab. 4 §6:

Paper vs code

No official code. The paper is a survey and has no code. Its abstract, comments and LaTeX source contain no repository link. A GitHub repository search for "capogrosso tinyml" returned no results.

Pitfalls

• The PRISMA counts do not add up. Duplicates (3413), tool exclusions (271) and inaccessible papers (549) sum to 4233, which leaves nothing for the step that reports 3341 excluded at title and abstract screening §3. The 189 selected, the 17 not found and the 172 eligible are consistent with each other. Do not treat the exclusion counts as a reproducible audit trail.
• Search details are incomplete. The paper gives the period (January 2018 to September 2023) §3, the five databases and the keyword phrases §3. It does not give the exact query strings per database, the search date, or the screening criteria beyond language and venue type §3. Google Scholar is one of the sources, so the search cannot be repeated exactly from the paper.
• Popularity labels have no stated definition. H, M and L are used in tab. 1, tab. 2 and tab. 3, but the paper does not say how they are assigned. The "cited works" counts in the table are not a measure of TinyML work either.
• Referenced works are not all TinyML papers. The text describes rational activation functions and attention as general ML techniques §5.2.2 §5.2.4. Section 7 says no RAF work was found in TinyML §7, yet tab. 2 lists three RAF papers. The table counts cited works, so it mixes general and TinyML sources.
• Depthwise-separable order is described two ways. Section 5.2.3 lists the point-wise step first and the spatial step second §5.2.3. The Xception sentence in the same subsection describes a depthwise convolution followed by a pointwise one §5.2.3. The paper does not say which order the cited models (MobileNet, MobileNetV2, Tiny-Sepformer) use. Check the cited papers before implementing.
• Quantization naming is the authors'. The paper defines dynamic-range quantization (weights only, 8-bit) and full-integer quantization (inputs and activations) §5.1.2. Use the paper's definitions as written, and say which definition an implementation follows.
• The "first systematic review" claim is the authors' own. They say none of the 172 reviewed papers is an ML-oriented survey on TinyML §3. The paper does not describe how this was checked beyond the same search.
• Table and figure numbers are assigned by us. The LaTeX source has no printed table numbers. tab. 1 to tab. 3 are the model-optimization, model-design and learning-algorithm tables, and tab. 4 is the hardware table, numbered in order of appearance. Figures are numbered fig. 1 to fig. 5 in the same way.
• The discussion is opinion, not measurement. The statements that co-design is the main future direction §7 and that unsupervised and continual learning are the most active areas §7 follow from the authors' reading of the counts.
• Benchmarks and datasets are missing. The authors say there is no accepted TinyML benchmark or public dataset comparable to ImageNet §7. Any result from this survey must be compared against a benchmark the reader picks.

Lineage

• Builds on: the TinyML definition the paper quotes from Warden §1.
• Compared against: the existing surveys reviewed in Section 2 §1 §2. This card does not summarize them.
• Superseded by: none known from this wiki.