---
kind: concept
slug: linguistic-steganography
title: Linguistic steganography
summary: >-
  Hiding a secret inside text that reads as ordinary language. The known approach lets a word-level LSTM generate the cover text, with a shared key that maps secret bits to vocabulary bins. Capacity and perplexity rise together. Only one paper so far, with author-reported numbers on Twitter and no tests against human readers or detectors.
tags: [steganography, text-generation, privacy, lstm]
papers: ["1705.10742"]
related: [text-watermarking, ai-generated-text-detection, long-short-term-memory, recurrent-neural-network, constrained-decoding]
status: draft
updated_at: 2026-11-15
---

## Overview

Linguistic steganography hides a message inside text that looks like normal
writing, so an observer sees an ordinary tweet or email rather than encrypted
noise [arxiv:1705.10742 §1]. Earlier systems start from an existing cover text
and swap words for synonyms or paraphrases. Their capacity is bounded by the
number of possible swaps, and the result can read unnaturally [arxiv:1705.10742
§2.1].

The variant described below generates the whole text with a language model.
The invariant that all variants share is a key. Sender and receiver agree on a
split of the vocabulary into disjoint bins, and each block of secret bits names
one bin. The sender emits a word from that bin, and the receiver looks up which
bin each word belongs to to read the bits back [arxiv:1705.10742 §2.2]. Decoding
needs the key and the token list, not the language model [arxiv:1705.10742
§2.2].

The evidence base is one paper that calls its experiments preliminary
[arxiv:1705.10742 §1] [§4]. Its quantitative results cover Twitter only. Its
email results are sample passages [arxiv:1705.10742 tab. 5]. The authors say
they aim to open-source the system, and no official code is known [arxiv:1705.10742
§5].

## Variants

### Bin-restricted LSTM

A word-level LSTM generates the cover text. Each b-bit block of the secret
selects one of 2^b bins. At that step the model's output is restricted to the
words in that bin, and every other word gets probability zero, so the softmax
can only choose within the bin [arxiv:1705.10742 §2.2] [§3.1]. The number of
secret bits per output word equals b when there are no common tokens
[arxiv:1705.10742 §3.2]. The paper's comparison uses 4 bins, which gives 2 bits
per word [arxiv:1705.10742 §4.4]. Card:
[Generating Steganographic Text with LSTMs](/papers/1705.10742).

On the Twitter model, the author-reported capacity rises with the bin count
(1, 2 and 3 bits per word for 2, 4 and 8 bins), and so does perplexity (190.84,
381.2 and 833.11 against 134.73 with one bin) [arxiv:1705.10742 tab. 3]. The
authors estimate 32 bits per tweet for the 4-bin setting by multiplying 2 bits
per word by an assumed tweet length of 16.04 words [arxiv:1705.10742 §4.4]. The
figure is an estimate, not a measured stegotext length.

### With common tokens

A set C of frequent tokens is added to every bin. A common token carries no
bits. When the model picks one, the sender draws again from the bin until it
picks a token that is not common, and the decoder removes all common tokens
before the bin lookup [arxiv:1705.10742 §2.2]. This makes the output length
differ from the number of blocks [arxiv:1705.10742 §2.2] [§3.2].

On the Twitter model at 4 bins, common tokens lower capacity from 2 to 1.17
bits per word and lower perplexity from 381.2 to 277.55 [arxiv:1705.10742
tab. 3]. In a small manual review the authors report better grammar and
coherence, mostly in the weakest samples [arxiv:1705.10742 tab. 4].

## When to use which

Only one paper is covered here, so this table compares the settings that paper
tested. It does not compare with other methods.

| Situation | Use | Why |
|---|---|---|
| The cover text must be generated, not edited, and you can train a word-level LSTM on in-domain text | Bin-restricted LSTM | The paper generated tweets this way. Its Twitter model took about 10 hours on one GPU [arxiv:1705.10742 §4.2] |
| Readability matters more than capacity | Bin-restricted LSTM with common tokens | Perplexity falls and capacity falls too (4 bins: 381.2 to 277.55; 2 to 1.17 bits per word) [arxiv:1705.10742 tab. 3] |
| You want the most bits per word and accept higher perplexity | Bin-restricted LSTM with more bins | Capacity rises with bin count, but so does perplexity at every step the paper reports [arxiv:1705.10742 tab. 3] |
| The receiver has no model or training budget | Any variant above | Decoding uses only the key and the token list [arxiv:1705.10742 §2.2] |
| You need resistance to human readers or to steganalysis | None established | The paper leaves formal human and steganalysis tests to future work [arxiv:1705.10742 §4.4] [§5] |
| Your domain is email | Not characterised | The email results are sample passages with no capacity or perplexity [arxiv:1705.10742 tab. 5] |

## Implementation sketch

The sketch follows the common core of the bin-restricted LSTM. The paper does
not state the sampling rule, so `pick_within` is left open (see Contradictions).

```pseudocode
# Sender: emit one word per block of secret bits  [arxiv:1705.10742 §2.2]
def embed(bits, b, bins, common, lm):
    text = []
    # Cut the secret into b-bit blocks; a remainder is not sent  [§2.2]
    for block in chunks(bits, b):
        # The shared key maps each block to one bin  [§2.2]
        bin_words = bins[int(block, 2)]
        # The model may pick only from the bin, plus common tokens  [§3.1]
        word = lm.pick_within(bin_words | common, text)
        # Common words carry no bits: redraw from the bin  [§2.2]
        while word in common:
            text.append(word)
            word = lm.pick_within(bin_words, text)
        text.append(word)
    return text
```

```pseudocode
# Receiver: recover bits from the words  [arxiv:1705.10742 §2.2]
def decode(text, bins, common, b):
    bits = []
    for word in text:
        # Remove common tokens before the lookup  [§2.2]
        if word in common:
            continue
        # Each other word sits in exactly one bin  [§2.2]
        index = bin_index_of(word, bins)
        bits.append(format(index, "0{}b".format(b)))
    return "".join(bits)
```

## Contradictions

None known between papers, since only one paper is covered. Within that paper,
two points are unclear and remain open. First, §3.1 describes the choice as the
highest-probability word, and also as a multinomial softmax, so it is not clear
whether the sender samples or picks greedily [arxiv:1705.10742 §3.1]. Second,
the perplexity estimator averages over all bit-block values, and the exact
computation is not given. The 1-bin value is the only comparison the paper's
numbers support with confidence [arxiv:1705.10742 §3.2] [tab. 3].

## Papers

- [Generating Steganographic Text with LSTMs](/papers/1705.10742): a restricted-softmax LSTM that embeds bits through vocabulary bins, with Twitter and email experiments.
