---
kind: concept
slug: semantic-change-detection
title: Semantic change detection
summary: >-
  Semantic change detection finds words whose meaning shifts between time periods by comparing word vectors trained on each period's corpus. Vectors from separate runs must be aligned, or trained incrementally or jointly, before comparison. Variants differ in model family, global or local measure and time slicing. The only paper here is a 2018 survey with no new experiments.
tags: [nlp, semantic-change, representation-learning]
papers: ["1806.03537"]
related: [word-embedding, skip-gram-negative-sampling, topic-modelling, debiasing-word-embeddings]
status: draft
updated_at: 2026-11-03
---

## Overview

The problem is to find words whose meaning differs between periods, or to rank
words by how much they changed, given one corpus per period [arxiv:1806.03537 §3].
The shared core is to learn a vector space per period, then compare each word's
vectors across periods. The invariant every variant must handle is that vectors
from independent training runs live in different coordinate systems, so a raw
cosine between them means nothing until they are aligned or trained together
[arxiv:1806.03537 §3.3].

## Variants

The survey organises the field along four axes. A method is a choice on each.

### Model family

Options run from frequency counts, through sparse count matrices weighted by
mutual information and dense SVD of PPMI matrices, to prediction-based skip-gram
with negative sampling (SGNS) and CBOW [arxiv:1806.03537 §3.2]. The survey
reports that frequency methods detect shifts less precisely than distributional
models [arxiv:1806.03537 §3.2]. Card: [Diachronic word embeddings and semantic shifts: a survey](/papers/1806.03537).

### Making periods comparable

- **Alignment:** train each period separately, then map them into one space with
  linear transformations [arxiv:1806.03537 §3.3].
- **Incremental training:** update one model from one period to the next, so no
  alignment step is needed [arxiv:1806.03537 §3.3].
- **Joint training:** train all periods together so they share one space
  [arxiv:1806.03537 §3.3].

The survey lists these three options but does not rank them. Card: [Diachronic word embeddings and semantic shifts: a survey](/papers/1806.03537).

### Comparison measure

- **Global measures** compare a word's similarity to the whole vocabulary, using
  second-order embeddings [arxiv:1806.03537 §3.3].
- **Local measures** compare nearest-neighbour lists [arxiv:1806.03537 §3.3].

Card: [Diachronic word embeddings and semantic shifts: a survey](/papers/1806.03537).

## When to use which

| Situation | Use | Why |
|---|---|---|
| The shift you want is gradual, regular linguistic drift | Global measure | The survey says global measures track linguistic drift [arxiv:1806.03537 §3.3] |
| The shift you want is a sudden cultural change | Local measure | The survey says local measures suit cultural shifts [arxiv:1806.03537 §3.3] |
| The corpora are small, as in historical or digital-humanities work | SVD of PPMI as a baseline, or SGNS | The survey says low-rank SVD can match SGNS, especially on smaller data, and that small data is an open need [arxiv:1806.03537 §3.2] [arxiv:1806.03537 §7] |
| You want to avoid an alignment step | Incremental or joint training | Both remove the need to align independently trained spaces [arxiv:1806.03537 §3.3] |
| You want to check whether a result is only a frequency effect | Frequency counts as a control | Shifts do not always come with frequency change, and the proposed laws may follow from frequency effects [arxiv:1806.03537 §3.2] [arxiv:1806.03537 §4] |
| The language is not English | Check first | Almost all surveyed work is on English, and transfer to other languages has not been shown [arxiv:1806.03537 §7] |
| You need sense-level shifts | Few options | Most surveyed algorithms are not sense-aware; one cited method (Biemann et al. 2014) handles senses [arxiv:1806.03537 §3.2] [arxiv:1806.03537 §7] |

The survey reports no runnable method and gives no default settings
[arxiv:1806.03537 §3] [arxiv:1806.03537 §8]. The time granularity of the cited
corpora ranges from one month to ten years [arxiv:1806.03537 §3.1.1]. Test sets
are scarce, so validate on your own labelled examples, or on the test-set types
the survey names: word epoch disambiguation, cross-time alignment, event
prediction and synthetic merged-word tasks [arxiv:1806.03537 §3.1.2].

## Implementation sketch

A generic pipeline that combines the survey's steps. It is an illustration, not a
method given in the paper.

```pseudocode
# Generic pipeline assembled from the survey  [§3] [§3.3]
# Input:  corpora, a list of (period, texts) ordered oldest first
#         words, the target words to score
# Output: score per word; higher means a larger change

def detect_semantic_change(corpora, words):
    # 1. Learn one vector space per period; incremental or joint training
    #    yields spaces that are already related  [§3.3]
    period_spaces = learn_period_spaces(corpora)
    # 2. Map independently trained spaces into one coordinate system with
    #    linear transformations; skip this after joint or incremental training
    #    [§3.3]
    aligned = align_spaces(period_spaces)
    # 3. Score each word by comparing its vectors across periods  [§3.3]
    #    a global measure suits regular drift; a local one suits cultural shifts
    scores = {}
    for word in words:
        scores[word] = compare_across_periods(aligned, word)
    return scores
```

## Contradictions

- **Laws of semantic change.** The survey lists three proposed laws: frequent
  words change more slowly (conformity), polysemous words change faster at
  matched frequency (innovation), and change correlates with distance to the
  cluster centroid (prototypicality) [arxiv:1806.03537 §4]. It also reports
  Dubossarsky et al. (2017), who found the same patterns in shuffled and
  synchronous control data, which suggests the laws may be artefacts
  [arxiv:1806.03537 §4]. Both sides are reported; the survey does not resolve this.
- **SGNS and PPMI.** The survey says SVD-factorised PPMI and SGNS are equivalent
  from a mathematical point of view [arxiv:1806.03537 §3.2]. It also reports an
  advantage for SGNS in shift analysis, which low-rank SVD can match, especially on
  smaller data [arxiv:1806.03537 §3.2].

## Papers

- [Diachronic word embeddings and semantic shifts: a survey](/papers/1806.03537): surveys data, model families, alignment and comparison measures for tracing word meaning change, and reviews the proposed laws. Reports no new experiments.