TinyML: Tools, Applications, Challenges, and Future Research Directions

In plain words

Most machine learning runs on large computers or in the cloud. TinyML is the idea of running small models directly on tiny, battery-powered chips such as sensors or microcontrollers. The data then never has to leave the device, the device can react without a network connection, and it can run for a long time on a small battery.

This paper is a survey. It does not add a new method. It explains the basics, lists the software tools and hardware boards people use for TinyML, goes through example applications in speech, vision, health and other fields, and ends with the problems that remain open, such as power, memory, hardware variety and missing shared benchmarks. Read it as a map of the field and its gaps.

TL;DR

• A survey of TinyML: ML on small, power-limited embedded devices. It proposes no method and runs no experiments of its own §I §V.
• It compares the software frameworks and hardware boards that support TinyML §II tab. II.
• It reviews example applications, grouped by domain §III-A to §III-J.
• It lists open challenges, including battery life, clock speed, memory and the lack of shared benchmarks and models §IV-A §IV-B tab. III.

Problem

The survey argues that cloud-based ML needs power, bandwidth and a connection, and raises privacy and latency concerns, so ML should run on edge devices instead §I. It identifies three open questions: the requirements and applications of TinyML, whether DNNs can run at the edge, and the trade-off between power use and accuracy §I-A. It notes that progress is limited by the lack of widely accepted TinyML benchmarks §I-A.

Method

This is a survey, so there is no algorithm or training procedure to implement. The paper's contribution is organised as follows:

1. Background (§II). TinyML is described as on-device analysis of sensing data (vision, audio, speech) at very low power §II. The paper treats TinyML as a mix of hardware, software and algorithms. Its algorithmic requirement is models with very small memory footprints §II. It gives the fundamental requirements as scalability to very many cheap devices and a small code budget in on-device RAM, described as a few KB §II.
2. Tools and platforms (§II, tab. II). Frameworks and their supported boards are compared (see Results below). The paper also reports a practitioner survey: about 59 % of participants use TensorFlow, 17 % PyTorch and 24 % other frameworks; the most common data types are vision, motion and sound; the most used boards are Raspberry Pi, Arduino Nano 33 BLE Sense, ESP32, Raspberry Pi Pico and NVIDIA Jetson Nano §II. No methodology for that survey is given (see Pitfalls).
3. Applications (§III). Each subsection describes one or more cited works and their reported results (see Results). The subsections are speech (speech communication §III-A1, hearing aids §III-A2), vision (§III-B), data pattern classification and compression (§III-C), health diagnosis (§III-D), edge computing (§III-E), brain-computer interfaces (§III-F), autonomous vehicles (§III-G), phenomics and ecology (§III-H), anomaly detection (§III-I) and predictive maintenance (§III-J) fig. 1.
4. Challenges and directions (§IV). The paper lists four resource constraints for edge devices: battery power, clock speed (10–1000 MHz), flash memory (under 1 MB on existing platforms) and cost at scale §IV-A. The "lessons learnt" section adds energy harvesting, reliability (process variation, soft errors, ageing), federated and transfer learning for model design, and standard benchmarks and datasets §IV-B tab. III.
5. Conclusion (§V). A summary, with no new results §V.

The taxonomy of the survey is shown in fig. 1. Section numbering follows the compiled paper: Sections I to V, with subsections such as §I-A, §III-A and §IV-B.

The deployment pipeline the paper describes in §II, as pseudocode §II:

# TinyML deployment pipeline as described in the survey  [§II]
# Input:  a dataset and a target microcontroller board
# Output: a compact model flashed onto the device

def deploy_tinyml(dataset, board):
    # 1. Train the model on a much larger machine  [§II]
    model = train(dataset)

    # 2. Shrink it so it fits the board's memory  [§II] [tab. III]
    model = quantize(model)              # e.g. float to integer weights

    # 3. Convert the model into a binary image for the device  [§II]
    binary = compile_for(model, board)

    # 4. Flash the binary onto the board  [§II]
    return flash(binary, board)

Hyperparameters

Not applicable. The paper proposes no model and trains none §I §V. It states no hyperparameters for the works it reviews §III. Individual cited works may state their own settings, which are not reproduced here.

Results

Author-reported, and this survey adds no experiments of its own. The figures below are the numbers the survey cites from other works, as it states them. Check the cited work before relying on any of them.

Framework and board comparison, as summarised in tab. II. The table is condensed here; the hardware lists are shortened.

The table reports the supported platforms as the paper states them. The survey gives no benchmark numbers for these frameworks.

Paper vs code

No official code. The paper is a survey and has no implementation. The LaTeX source, the abstract and the arXiv metadata (raw/arxiv/2303.13569.json) link no code repository. The GitHub search was not run, because a survey has no method to reproduce and the awesome-list repositories that surveys link are not code.

Pitfalls

• Survey numbers have unclear sources. The practitioner percentages (59 %, 17 %, 24 %) and the list of most-used boards are attributed to "our survey" §II. The paper gives no sample size, questionnaire or citation for them. Treat them as unverified.
• Undefined or hard-to-read sentences. The paper says that most TinyML frameworks are publicly available, and names two systems that are not, but it does not say whether that list is complete §II. Read this as the paper's claim, not as a verified inventory.
• Scale ranges differ by source. Requirements are stated as "a few KB" of on-device RAM §II, and existing platforms as "lesser than 1 MB" of flash §IV-A. Both are stated; they describe different resources (RAM against flash).
• Missing unit. The PhiNets sentence reads "85–90-" with no unit or completed phrase §II. The percentage reading is an inference from context.
• Battery arithmetic. A 2 Ah battery at under 12 µA is said to last "more than ten years" §IV-A. The aggregate 176.4 mA draw is said to give about 11 h 20 min §IV-A. Both figures are consistent with the stated capacity; the 176.4 mA draw is far above the 12 µA case, which means the battery claim holds only for the idle or ultra-low-power case. The paper does not state the duty cycle behind either number.
• Undefined abbreviations. SR (speech recognition) is used without definition §III-A1. The paper also refers to the surveyed works by citation keys only, so readers must check each reference to find the claimed result.
• Reported figures lack setup. The survey gives speedups and savings (e.g. the mini-vehicle work §III-G) without the hardware, workload or baseline details. Check the original paper before quoting any of them.
• Minor typos in the text and tables, e.g. "TensorFlow Lie" and "oppportunities" §II §IV-B. These do not change the content.
• Board mismatch. The TinyOL work is said to use an Arduino Nano 33 BLE Sense board, but the citation is to TensorFlow Lite §III-C. Check the original work for the board used.

Lineage

• Builds on: none stated. The paper compares itself with earlier TinyML surveys in tab. I. It names eight of them (e.g. Ray 2022, Alajlan and Ibrahim 2022, Banbury et al. 2020) and claims to cover tools and applications more fully than they do. The comparison is the paper's own and is not checked here.
• Compared against: the eight surveys listed in tab. I. Their arXiv ids are not given in the source.
• Superseded by: not known from this wiki.