wikillm

Concept · 1 paper · updated 2026-11-15

Linguistic steganography

Hiding a secret inside text that reads as ordinary language. The known approach lets a word-level LSTM generate the cover text, with a shared key that maps secret bits to vocabulary bins. Capacity and perplexity rise together. Only one paper so far, with author-reported numbers on Twitter and no tests against human readers or detectors.

steganographytext-generationprivacylstm

Markdown4 min read

AI-compiled overview, updated 2026-11-15, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

Overview

Linguistic steganography hides a message inside text that looks like normal writing, so an observer sees an ordinary tweet or email rather than encrypted noise 1705.10742 §1. Earlier systems start from an existing cover text and swap words for synonyms or paraphrases. Their capacity is bounded by the number of possible swaps, and the result can read unnaturally [arxiv:1705.10742 §2.1].

The variant described below generates the whole text with a language model. The invariant that all variants share is a key. Sender and receiver agree on a split of the vocabulary into disjoint bins, and each block of secret bits names one bin. The sender emits a word from that bin, and the receiver looks up which bin each word belongs to to read the bits back 1705.10742 §2.2. Decoding needs the key and the token list, not the language model [arxiv:1705.10742 §2.2].

The evidence base is one paper that calls its experiments preliminary 1705.10742 §1 §4. Its quantitative results cover Twitter only. Its email results are sample passages 1705.10742 tab. 5. The authors say they aim to open-source the system, and no official code is known [arxiv:1705.10742 §5].

Variants

Bin-restricted LSTM

A word-level LSTM generates the cover text. Each b-bit block of the secret selects one of 2^b bins. At that step the model's output is restricted to the words in that bin, and every other word gets probability zero, so the softmax can only choose within the bin 1705.10742 §2.2 §3.1. The number of secret bits per output word equals b when there are no common tokens 1705.10742 §3.2. The paper's comparison uses 4 bins, which gives 2 bits per word 1705.10742 §4.4. Card: Generating Steganographic Text with LSTMs.

On the Twitter model, the author-reported capacity rises with the bin count (1, 2 and 3 bits per word for 2, 4 and 8 bins), and so does perplexity (190.84, 381.2 and 833.11 against 134.73 with one bin) 1705.10742 tab. 3. The authors estimate 32 bits per tweet for the 4-bin setting by multiplying 2 bits per word by an assumed tweet length of 16.04 words 1705.10742 §4.4. The figure is an estimate, not a measured stegotext length.

With common tokens

A set C of frequent tokens is added to every bin. A common token carries no bits. When the model picks one, the sender draws again from the bin until it picks a token that is not common, and the decoder removes all common tokens before the bin lookup 1705.10742 §2.2. This makes the output length differ from the number of blocks 1705.10742 §2.2 §3.2.

On the Twitter model at 4 bins, common tokens lower capacity from 2 to 1.17 bits per word and lower perplexity from 381.2 to 277.55 [arxiv:1705.10742 tab. 3]. In a small manual review the authors report better grammar and coherence, mostly in the weakest samples 1705.10742 tab. 4.

When to use which

Only one paper is covered here, so this table compares the settings that paper tested. It does not compare with other methods.

SituationUseWhy
The cover text must be generated, not edited, and you can train a word-level LSTM on in-domain textBin-restricted LSTMThe paper generated tweets this way. Its Twitter model took about 10 hours on one GPU 1705.10742 §4.2
Readability matters more than capacityBin-restricted LSTM with common tokensPerplexity falls and capacity falls too (4 bins: 381.2 to 277.55; 2 to 1.17 bits per word) 1705.10742 tab. 3
You want the most bits per word and accept higher perplexityBin-restricted LSTM with more binsCapacity rises with bin count, but so does perplexity at every step the paper reports 1705.10742 tab. 3
The receiver has no model or training budgetAny variant aboveDecoding uses only the key and the token list 1705.10742 §2.2
You need resistance to human readers or to steganalysisNone establishedThe paper leaves formal human and steganalysis tests to future work 1705.10742 §4.4 §5
Your domain is emailNot characterisedThe email results are sample passages with no capacity or perplexity 1705.10742 tab. 5

Implementation sketch

The sketch follows the common core of the bin-restricted LSTM. The paper does not state the sampling rule, so pick_within is left open (see Contradictions).

Pseudocode
# Sender: emit one word per block of secret bits  [arxiv:1705.10742 §2.2]
def embed(bits, b, bins, common, lm):
    text = []
    # Cut the secret into b-bit blocks; a remainder is not sent  [§2.2]
    for block in chunks(bits, b):
        # The shared key maps each block to one bin  [§2.2]
        bin_words = bins[int(block, 2)]
        # The model may pick only from the bin, plus common tokens  [§3.1]
        word = lm.pick_within(bin_words | common, text)
        # Common words carry no bits: redraw from the bin  [§2.2]
        while word in common:
            text.append(word)
            word = lm.pick_within(bin_words, text)
        text.append(word)
    return text
Pseudocode
# Receiver: recover bits from the words  [arxiv:1705.10742 §2.2]
def decode(text, bins, common, b):
    bits = []
    for word in text:
        # Remove common tokens before the lookup  [§2.2]
        if word in common:
            continue
        # Each other word sits in exactly one bin  [§2.2]
        index = bin_index_of(word, bins)
        bits.append(format(index, "0{}b".format(b)))
    return "".join(bits)

Contradictions

None known between papers, since only one paper is covered. Within that paper, two points are unclear and remain open. First, §3.1 describes the choice as the highest-probability word, and also as a multinomial softmax, so it is not clear whether the sender samples or picks greedily 1705.10742 §3.1. Second, the perplexity estimator averages over all bit-block values, and the exact computation is not given. The 1-bin value is the only comparison the paper's numbers support with confidence 1705.10742 §3.2 tab. 3.

Papers

Source and license

Compiled from the papers listed on this page, all published under CC BY, CC BY-SA or CC0, with citations. See each paper's page for its authors and license.