---
kind: concept
slug: keyword-spotting
title: Keyword spotting
summary: >-
  Detecting a few target words in audio that is mostly silence or noise, on-device under tight compute and energy limits. Covers the Speech Commands benchmark, its Top-One and streaming protocols, building a similar corpus, and a spike-coded version of the same audio. Two cards; no detector comparison yet.
tags: [speech, on-device, benchmark, neuromorphic]
papers: ["1804.03209", "1910.07407"]
related: [dataset-curation, spiking-neural-network, benchmark-validity]
status: draft
updated_at: 2026-11-01
---

## Overview

Keyword spotting (KWS) decides whether a short audio segment contains one of a
small set of target words, and labels everything else as unknown or silence.
Most of the input is silence, background noise or unrelated speech, so a
detector must reject non-target audio as well as recognise targets
[arxiv:1804.03209 §4]. The detector runs on the device, so compute, energy and
latency budgets are tight [arxiv:1804.03209 §4].

The unit of recognition is one word or a short phrase, not a sentence
[arxiv:1804.03209 §4]. Sentence-aligned corpora such as Common Voice and
LibriSpeech therefore do not fit the task directly [arxiv:1804.03209 §3].

**Coverage today.** Two cards list this concept. One is the Speech Commands
dataset and its evaluation protocols [arxiv:1804.03209]. The other converts
speech datasets, one of them derived from Speech Commands, into spike trains
for spiking networks [arxiv:1910.07407 §2.1.2]. The two cards use different
settings: 12-class Top-One on audio, and 35-class top-one on spike-coded audio.
Their numbers are not comparable (see Pitfalls). No card compares streaming
detectors, so this page does not compare detector architectures for the
streaming task.

## Variants

These are four ways to use the two cards' material, not four competing model
families.

### Fixed-clip classification (Top-One protocol)

Each test clip is one second of 16 kHz audio. The model picks one of 12
classes: ten command words, "Unknown Word" and "Silence" [arxiv:1804.03209
§7.1]. The test set is balanced, with about 8.3% of examples per class
[arxiv:1804.03209 §7.1]. The reference TensorFlow `conv` baseline reaches
88.2% when trained and tested on Speech Commands v2 [arxiv:1804.03209 §7.3]
[arxiv:1804.03209 fig. 2]. Card: [Speech Commands: A Dataset for
Limited-Vocabulary Speech Recognition](/papers/1804.03209).

### Continuous-stream detection (streaming protocol)

The model runs over a long recording with known word times. Each detection is
matched to a ground-truth word within a time tolerance, and the score counts
matched, wrong-label and false-positive detections [arxiv:1804.03209 §7.2].
The default tolerance is 750 ms [arxiv:1804.03209 §7.2]. On the V2-trained
baseline, the author reports 49.0% matched, 46.0% correct, 3.0% wrong and 0.0%
false positives [arxiv:1804.03209 §7.2]. Card: [Speech Commands: A Dataset for
Limited-Vocabulary Speech Recognition](/papers/1804.03209).

### Building a word corpus

The collection pipeline is a recipe for making your own labelled word
corpus. Volunteers record single words through a browser on phone or laptop
microphones [arxiv:1804.03209 §5.3]. Clips are then filtered by file size,
loudness and a human check, and trimmed to the loudest second
[arxiv:1804.03209 §5.4–§5.6]. Splits are assigned by a hash of the file name,
so they stay stable across releases [arxiv:1804.03209 §7]. Card: [Speech
Commands: A Dataset for Limited-Vocabulary Speech Recognition](/papers/1804.03209).

### Spike-coded input (Spiking Speech Commands)

The spiking card converts Speech Commands version 0.02 clips into spike trains
with a fixed, cochlea-inspired model with 700 channels [arxiv:1910.07407
§2.1.2] [arxiv:1910.07407 §2.2]. The conversion is published, so a modeller
does not choose an encoder [arxiv:1910.07407 §2.2]. The paper describes the
resulting files as HDF5 with spike times, units and labels [arxiv:1910.07407
§2.3]. Author-reported test accuracies on this set, over 35 classes:

- Linear SVM on spike counts: 21.6% [arxiv:1910.07407 tab. 1].
- LSTM on 10 ms bins: 73.0% in the table [arxiv:1910.07407 tab. 1], 75.0% in
  the text [arxiv:1910.07407 §3.1].
- CNN on 10 ms bins: 77.7% [arxiv:1910.07407 tab. 1].
- Feed-forward SNNs of 1 to 3 layers: 32.5% to 41.0% [arxiv:1910.07407 tab. 1].
- Recurrent SNN with max-over-time readout: 50.9% [arxiv:1910.07407 tab. 1].

Card: [The Heidelberg spiking datasets for the systematic evaluation of spiking
neural networks](/papers/1910.07407).

## When to use which

| Situation | Use | Why |
|---|---|---|
| You need a freely redistributable word-level training and test set | Speech Commands v2 | Built for this task; the paper was released under CC BY 4.0 [arxiv:1804.03209 §2] [arxiv:1804.03209 §6] |
| You compare classifiers on isolated one-second clips | Top-One protocol | Fixed 12-class balanced test set, scored against folder labels [arxiv:1804.03209 §7.1] |
| You need detection behaviour over time (false alarms, missed words) | Streaming protocol | Measures matches, wrong labels and false positives on a continuous file [arxiv:1804.03209 §7.2] |
| Your target words are not in the released vocabulary | Building a word corpus | The released set covers 35 words; you collect your own [arxiv:1804.03209 §5.2] |
| Your model is a spiking network and needs spike-coded input | Spiking Speech Commands | Fixed published encoder, so no per-study encoder choice [arxiv:1910.07407 §2.2] |
| You want a spiking baseline on one shared input | The paper's table | The baselines are run on the same spike input, so their rows compare with each other [arxiv:1910.07407 tab. 1] [arxiv:1910.07407 app. B] |
| You plan to count spikes only | Avoid count-only readouts | Linear SVMs on spike counts reach 21.6% on SSC, against 73.0% to 77.7% for LSTMs and CNNs on binned spikes [arxiv:1910.07407 tab. 1] |
| Your task is full-sentence speech recognition | Not this dataset | Speech Commands is single-word and one-second [arxiv:1804.03209 §4] |
| The language is not English | Not covered | The paper is English-only by design [arxiv:1804.03209 §5.1] |

Constraints the cards do not cover: model size, latency and energy figures for
any detector. The paper names energy as a comparison target [arxiv:1804.03209
§2] [arxiv:1804.03209 §4], but neither card reports energy or latency numbers.
Do not infer them. The spiking card reports accuracy only.

## Implementation sketch

The two protocols as described in the cards. Labels come from the folder or
test-list entry. Thresholds and window settings are not stated in the paper;
see the card for the code defaults it lists [arxiv:1804.03209 §7.2].

```text
# Top-One accuracy over the balanced 12-class test set [arxiv:1804.03209 §7.1]
for clip in test_set:
    scores = model(clip)                # one-second, 16 kHz input
    prediction = argmax(scores)         # one of 12 classes
    correct += (prediction == clip.label)
accuracy = correct / len(test_set)
```

```text
# Streaming evaluation on a long recording [arxiv:1804.03209 §7.2]
detections = []
for window in sliding_windows(recording, stride):
    scores = model(window)
    if max(scores) > threshold:         # threshold is a setting, not stated
        detections.append((label, time))
detections = suppress_repeats(detections, span)   # same word within span
for word in ground_truth:
    match if a detection has the same label within the time tolerance
report matched %, wrong-label %, false-positive %
```

## Contradictions

Within the Speech Commands card, the paper and its own tooling disagree in
several places. The card lists each one and does not resolve it. Read the
card's "Paper vs code" section before reproducing any result [arxiv:1804.03209
§5.5] [§7.1].

- **Loudness filter order.** The paper measures loudness before trimming
  [arxiv:1804.03209 §5.5]. The released tool trims first, then measures
  as the card's "Paper vs code" section reports [arxiv:1804.03209 §5.5].
- **Unknown-word sampling.** The paper says unknown examples come from classes
  "that are part of the target set" [arxiv:1804.03209 §7.1], which contradicts
  the purpose of the class. The tutorial code uses the complement.
- **Accuracy set.** The text calls the 88.2% figure a training-set result
  [arxiv:1804.03209 §7.3], while the figure labels it as the V2 test set
  [arxiv:1804.03209 fig. 2].
- **Streaming file length.** The text says the result comes from a 10-minute
  clip [arxiv:1804.03209 §7.2]. The released streaming file is one hour long
  [arxiv:1804.03209 §7.2].

Within the spiking card, the paper's table and text disagree on the LSTM result
on SSC, 73.0% against 75.0% [arxiv:1910.07407 tab. 1] [arxiv:1910.07407 §3.1].
Its surrogate steepness also differs between text (40) and table (100), which
affects every SNN row on SSC [arxiv:1910.07407 §3.1] [arxiv:1910.07407 tab. 2].
The card reports both and does not choose.

Across the two cards there is no direct conflict. The two cards use different
class sets and inputs, so their numbers answer different questions (see
Pitfalls).

## Pitfalls

- **Version and split.** V1 and V2 differ in word count and clip count
  [arxiv:1804.03209 §5.2] [arxiv:1804.03209 §7.3]. Numbers from different
  versions are not comparable; state which one you report [arxiv:1804.03209
  fig. 2]. Use the released validation and test lists, or the same hash rule
  [arxiv:1804.03209 §7]. A random split leaks speakers.
- **Hashing by file name.** Speaker grouping depends on the file-name prefix
  before the no-hash marker. Renaming files moves them between splits (code
  only; see the card's "Paper vs code" section).
- **Which test set.** The canonical Top-One number uses the released balanced
  12-class test archive [arxiv:1804.03209 §7.1]. The training script's own
  final test uses a different sampling of unknown and silence, so its classes
  are not balanced. Say which one you used.
- **Class count differs across cards.** The Top-One protocol uses 12 classes
  [arxiv:1804.03209 §7.1]. The spiking card reports top-one over 35 classes
  [arxiv:1910.07407 §2.1.2]. The 88.2% and the 77.7% CNN figure are not
  comparable.
- **Input differs across cards.** The Top-One numbers use audio. The spiking
  numbers use spikes from one conversion chain [arxiv:1910.07407 §2.2]. Results
  depend on that chain. Its code differs from the paper in several parameters,
  so reproduce the chain only after reading the spiking card's "Paper vs code"
  section.
- **Unlisted word.** A word called "Visual" appears in the counts but in
  neither word list [arxiv:1804.03209 §5.2] [arxiv:1804.03209 fig. 1]. Build the
  label set from folder names, not from the paper's lists.
- **Class imbalance.** Command words have roughly twice as many clips as the
  auxiliary words and the V2 additions [arxiv:1804.03209 fig. 1]. Weight or
  sample when training on all 35 words as separate classes.
- **Label noise.** Each clip was checked by one crowd worker, and the alignment
  assumes the word is the loudest part [arxiv:1804.03209 §5.5] [arxiv:1804.03209
  §5.6]. Expect some mislabelled or cut-off clips.
- **Generated noise.** The noise generator casts to int16 without clipping, so
  large samples wrap around [arxiv:1804.03209 §5.8]. Clip before casting if you
  regenerate noise.
- **Recording bias.** No iOS devices, English only, speakers recruited through
  social media [arxiv:1804.03209 §5.1] [arxiv:1804.03209 §5.3]. The metadata has
  no demographic labels, so accent or gender bias cannot be measured from it
  [arxiv:1804.03209 §5.1].
- **Streaming settings.** Report the tolerance, suppression and threshold with
  any streaming result [arxiv:1804.03209 §7.2]. Results depend on them.
- **Lower bounds.** The spiking paper's authors expect the datasets may not
  prove final and describe their accuracies as lower bounds [arxiv:1910.07407
  §4] [arxiv:1910.07407 §3].

## Papers

- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](/papers/1804.03209):
  a word-level dataset of one-second clips, with the Top-One and streaming
  evaluation protocols for small-footprint keyword spotters.
- [The Heidelberg spiking datasets for the systematic evaluation of spiking neural networks](/papers/1910.07407):
  Spiking Speech Commands, a spike-coded version of Speech Commands 0.02 with
  35-class top-one results for spiking and non-spiking classifiers.