---
kind: concept
slug: topic-modelling
title: Topic modelling
summary: >-
  Finding the main topics in a text collection with non-probabilistic (LSA, NMF) or probabilistic (PLSA, LDA) models, and with extensions that put text on graph edges or nodes. Plain topic models have no hyperparameters or results on these cards; the graph variants report author-reported numbers.
tags: [topic-modelling, nlp, review]
papers: ["1903.00114", "2301.08826"]
related: [feature-selection, sentiment-analysis, stochastic-block-model]
status: draft
updated_at: 2026-10-19
---

## Overview

Topic modelling finds the main topics in a set of texts. A review of NLP for
student feedback lists it among the techniques for that task, next to feature
extraction and feature selection [arxiv:2301.08826 §2].

The same idea reappears in network models. A review of stochastic block models
covers topic models as their own section and as an extension of graph
clustering, where the text sits on the edges or nodes of a network
[arxiv:1903.00114 §10] [arxiv:1903.00114 §11].

Two families of plain topic model appear in the cards. Non-probabilistic models
include Latent Semantic Analysis (LSA) and Non-negative Matrix Factorisation
(NMF) [arxiv:2301.08826 §2.3.1]. Probabilistic models include Probabilistic LSA
(PLSA) and Latent Dirichlet Allocation (LDA) [arxiv:2301.08826 §2.3.2]
[arxiv:1903.00114 §10].

The cards give no hyperparameter values, no measured comparison between plain
topic models, and no results for them. The graph-based numbers below are the
review's summaries of other studies, not measurements made by the review.

## Variants

### Non-probabilistic models

The student-feedback review places LSA and NMF in this family
[arxiv:2301.08826 §2.3.1]. It gives no procedure, cost or quality comparison
for either. Card: [Shaik et al., 2023](/papers/2301.08826).

### Latent Dirichlet allocation (probabilistic)

The block-model review calls LDA the reference topic model [arxiv:1903.00114
§10]. In its description, each document has a topic mixture drawn from a
symmetric Dirichlet, each topic is a Dirichlet distribution over the
vocabulary, and each word draws its own topic [arxiv:1903.00114 §10]. Fitting
is mostly Bayesian, with collapsed Gibbs sampling or variational methods
[arxiv:1903.00114 §10.1]. The student-feedback review says LDA is the most used
topic model in the works it reviews [arxiv:2301.08826 §2.3.2]
[arxiv:2301.08826 §4]. That describes usage, not a measured advantage.

Cards: [Shaik et al., 2023](/papers/2301.08826) and
[Lee and Wilkinson, 2019](/papers/1903.00114).

### Choosing the number of topics

Plain LDA takes the number of topics as an input. The block-model review lists
three options: fix it, choose it by perplexity or marginal likelihood, or let a
Dirichlet process (HDP) or Indian buffet process (IBP) generate it
[arxiv:1903.00114 §10.2] [arxiv:1903.00114 §10.2.1]. It does not say which
option gives the best topics. Its comparison table lists the number of topics
per study, for example fixed at 20 in one study and set by a Dirichlet process
in others [arxiv:1903.00114 tab. 3].

### Dynamic topic models

These let topic distributions change over time, with a state-space model, or
with a Dirichlet process that evolves over time [arxiv:1903.00114 §10.3].

### Topic models on networks

When the text sits on the edges of a network, for example emails between
people, the review lists author-recipient topic models, group-based topic
models, and the stochastic topic block model (STBM), which gives each pair of
groups its own topic mixture [arxiv:1903.00114 §11.1.1].

When the text sits on the nodes, for example linked documents, it lists topic-link
LDA, a relational topic model, and a hierarchical tree model that mixes word
distributions along a path [arxiv:1903.00114 §11.1.2].

On the Enron email network, one reviewed study selects 10 groups and 5 topics
with the STBM, against 8 groups from a block model alone [arxiv:1903.00114
§12.1.2]. This is one study's result as summarised by the review. Card:
[Lee and Wilkinson, 2019](/papers/1903.00114). Its clustering setup is covered
on the [stochastic block model](/concepts/stochastic-block-model) page.

## When to use which

The cards give no rule that links a constraint to a topic model. The table
records what they support and what they leave open.

| Situation | Use | Why |
|---|---|---|
| No basis yet for choosing a plain topic model | LDA | The student-feedback review says it is the most used [arxiv:2301.08826 §2.3.2] [arxiv:2301.08826 §4]. The block-model review calls it the reference model [arxiv:1903.00114 §10]. Neither compares it with other models. |
| You want a non-probabilistic baseline | LSA or NMF | Grouped as non-probabilistic [arxiv:2301.08826 §2.3.1]. No comparison with probabilistic models is given. |
| The number of topics is unknown | HDP or IBP, or choose by perplexity or marginal likelihood | The review lists these options [arxiv:1903.00114 §10.2] [arxiv:1903.00114 §10.2.1]. It does not rank them. |
| Text on network edges | STBM or other group-based topic models | Listed for this setting [arxiv:1903.00114 §11.1.1]. One Enron study reports K = 10 groups and 5 topics [arxiv:1903.00114 §12.1.2]. |
| Text on network nodes | Topic-link LDA or a relational topic model | Listed for this setting [arxiv:1903.00114 §11.1.2]. No comparison is given. |
| Topics must change over time | Dynamic topic models | Listed [arxiv:1903.00114 §10.3]. One Enron study matches its period boundaries to events [arxiv:1903.00114 §12.1.3]. |
| Any other constraint (data size, latency, memory, model access) | Not stated | No card gives constraint-based guidance for topic models [arxiv:2301.08826 §2.3] [arxiv:1903.00114 §10]. |

For a real choice, read the original papers on each model. They are not on this
wiki yet, so no card supports a claim about them.

## Implementation sketch

The cards give no fitting algorithm, equation set or code for any topic model.
The sketch below is only the generative story of LDA, as the block-model
review describes it [arxiv:1903.00114 §10]. It does not say how to infer the
topics. The review names collapsed Gibbs sampling and variational methods for
that [arxiv:1903.00114 §10.1]. Hyperparameter values are not stated.

```pseudocode
# Sample documents from LDA  [arxiv:1903.00114 §10]
# Input:  K topics, V vocabulary size, alpha, beta (symmetric concentrations)
#         doc_lengths (list of ints)
# Output: documents as lists of word ids

def sample_lda(K, V, alpha, beta, doc_lengths):
    # Each topic is a distribution over the vocabulary  [arxiv:1903.00114 §10]
    topic_words = [draw_dirichlet(beta * ones(V)) for k in range(K)]
    docs = []
    for length in doc_lengths:
        # Each document has its own topic mixture  [arxiv:1903.00114 §10]
        mixture = draw_dirichlet(alpha * ones(K))
        words = []
        for _ in range(length):
            # Each word draws its own topic, then a word from that topic
            z = draw_categorical(mixture)  # [arxiv:1903.00114 §10]
            words.append(draw_categorical(topic_words[z]))
        docs.append(words)
    return docs
```

## Contradictions

Neither card disagrees with the other on a measured claim. They describe
different settings. The student-feedback review calls LDA the most used model
[arxiv:2301.08826 §2.3.2], and the block-model review calls it the reference
model [arxiv:1903.00114 §10]. Each is a claim about the field, not a test.

Within the student-feedback review, two source inconsistencies affect this
page:

- Table 2's Topic Modelling row lists Wang et al. and Fowler et al. The text
  does not discuss either. In the TeX source, their paragraphs are commented out
  [arxiv:2301.08826 §2.3.2] [tab. 2].
- Table 2's Feature Extraction row lists LSA and LDA. The text places both under
  topic modelling [arxiv:2301.08826 §2.3.1] [arxiv:2301.08826 §2.3.2]
  [tab. 2].

## Papers

- [Lee and Wilkinson, 2019](/papers/1903.00114): a review of stochastic block models. It covers LDA and the topic-model extensions to network models, and compares models in four tables. It runs no experiments of its own.
- [Shaik et al., 2023](/papers/2301.08826): a review of NLP for student feedback. It names two families of topic model and four models, but runs no experiments and gives no results tied to a specific model.