Named Entity Recognition and Classification on Historical Documents: A Survey

In plain words

Named entity recognition means finding the names of people, organisations and places in a text and labelling each one. Libraries have scanned very large collections of old newspapers, books and records, and historians want to search them by the people and places they mention. This survey asks how well that task works on old documents. Its answer is that it works much worse than on modern news. The text was produced by machines that misread letters, spelling and naming habits have changed, and there are few labelled examples to learn from.

The survey sorts the work done so far into three generations. The first generation used hand-written rules and lists of names. The second learned from labelled examples with classic statistical models. The third uses neural networks that start from language models trained on large amounts of text. The survey's summary of the literature is that the best neural systems reach about 80% on historical text.

The authors group the problems into four themes. They list which datasets and pretrained models exist, and which fixes people have tried. They end with five priorities for the field: systems that carry over across time periods and document types, robustness to noise, fair comparison through shared tests, finer-grained entity types, and sharing of data. It is a literature review, not a new system.

TL;DR

A literature survey, not a new method. It names four challenges for entity recognition on historical text, inventories 17 annotated corpora and the historical embeddings and language models that exist, and reviews 45 works grouped as rule-based, classic machine learning and deep learning tab. 5. Use it as a map of the field and a source list. Do not treat its numbers as results of this paper, and do not compare them across rows without checking the settings.

Problem

Digitised historical text is hard for named entity recognition (NER). The authors list four challenges and argue that they compound each other §4:

• Document variety: historical collections span many document types, languages and periods, and the survey argues that generalisation to them has not been studied for historical text §4.1.
• Noisy input: OCR and layout errors create out-of-vocabulary tokens, broken words and mixed text segments §4.2 fig. 1.
• Dynamics of language: spelling norms, naming conventions and the entities themselves change over time §4.3.
• Lack of resources: typologies often do not fit historical content, and annotated corpora and historical language models are few §4.4.

The survey's research questions are: what the challenges are, which resources exist and what they cover, and which strategies worked §2.1. It defines a historical document as mainly textual and produced or published up to 1979. The authors call this cut-off a choice §2.2.

Method

The survey has no algorithm of its own. Its method is literature compilation and a structured review §2.2:

1. Compilation: scanning the archives of named journals and conference series, keyword searches on Google Scholar and Semantic Scholar, and citation chaining §2.2 tab. 1.
2. Inclusion: with a few exceptions, only publications with a formal evaluation were kept §2.2. The number of papers screened and included is not stated.
3. Organisation: the works are grouped by approach family, rule-based, traditional machine learning and deep learning, and ordered by year §6 tab. 5.
4. Challenges and strategies: each challenge is paired with strategies that address it, summarised in tab. 7 and §7 §7.1 §7.2 §7.3.
5. Resources: typologies and guidelines, annotated corpora and language representations are inventoried §5.1 §5.2 §5.3 tab. 3 tab. 4.

The survey separates three kinds of approach §6:

• Rule-based systems use gazetteers and hand-written rules. They need no training data, are easy to read, and need time and expertise to build §6.1.
• Traditional machine learning trains statistical taggers on annotated data. The Stanford CRF classifier is the most used of these §6.2.
• Deep learning uses neural encoders. Most models are BiLSTM or self-attention networks with a CRF output layer. Pretrained embeddings or language models supply the input representation §6.3. Table 6 groups the reviewed experiments under three themes: input representation, transfer learning, and neural architecture, which covers the comparison with CRFs §6.3 tab. 6.

The survey's own summary of strategies for noisy input is that correcting the text can help, and that character-aware or contextual embeddings and transformers can cope with noise with less effort §7.1. For language change it recommends loosened gazetteer matching, spelling normalisation, and embeddings trained on text close in time to the target §7.2. For scarce data it describes typology adaptation, active learning and data augmentation, and names transfer learning as the most used strategy §7.3.

The pipeline below is the deep learning recipe the survey describes. It is reconstructed from the text, not taken from code. The survey gives no exact dimensions, layer sizes or training settings.

# Neural historical NER, as described across the deep learning works  [§6.3]
# Input:  OCR text of one document (string)
# Output: one entity label per token, in IOB form  [§3.2] [tab. 2]

def tag_document(text, embeddings, encoder, decoder):
    # 1. Rejoin hyphenated words and split the text into sentences  [§6.3]
    #    The survey reports both as key pre-processing steps  [§7.1]
    sentences = split_sentences(rejoin_hyphens(text))
    labelled = []
    for sentence in sentences:
        # 2. Build one vector per token by stacking embeddings  [§6.3] [tab. 6]
        #    Stacks may mix character, sub-word, static and contextual vectors
        vectors = embeddings.stack(sentence)
        # 3. Encode the token sequence with a bidirectional LSTM or  [§6.3]
        #    self-attention network
        states = encoder(vectors)
        # 4. Decode the most likely label sequence with a CRF layer  [§6.3]
        labelled.extend(decoder.decode(states))
    return labelled

Hyperparameters

Not applicable. The survey runs no experiments and gives no training settings of its own. Settings of the reviewed systems are not reproduced here. Where the survey mentions one, for example a few extra epochs of domain adaptation §6.3, the cited work is the source.

Results

Author-reported figures, as the survey quotes them from the cited work. Each row keeps the setting the survey gives. Scores are F-scores unless marked otherwise. These numbers are not directly comparable across rows.

The survey does not state how the "60-70% on average" figure was computed from the work it reviews. Treat it as the authors' summary, not a recomputed result §8.

Paper vs code

No official code. A survey releases no method code. The repositories the paper links are corpora and embedding downloads, such as the Europeana NER corpora and the HIPE data §5.2 §5.3, not implementations. A GitHub search for "historical named entity recognition" returned 28 results; the first 16 were read. None was owned by an author or the authors' labs, and none was identified as the survey's own code. The remaining 12 results were not read.

Pitfalls

• Mixed settings. Scores use partial match in some rows and exact or strict match in others. Rule-based and deep learning rows often use different corpora and periods, so they cannot be ranked against each other §6.1 §6.2 §6.3 tab. 5. The survey itself notes that few results are comparable, mainly the German historical set and the HIPE campaign §6.3.
• Corpus licences vary and several are restrictive. The survey lists licences in tab. 3 as stated by the survey. Entries include CC BY-NC-SA (HIPE, De Gasperi, Czech), CC BY-NC-ND (BDCamões), GPL-3.0 (BIOfid, Latin NER) and LGPL-LR (GeoNER). The HIPE-2020 auxiliary fastText and flair embeddings are under CC BY-NC 4.0 §5.3. Check each source before reuse; the survey does not verify these licences.
• Time coverage. The newest entries in the survey's tables are from 2021 tab. 3 tab. 4. The survey's search end date is not stated §2.2. Later work is not covered, so check the newer literature before relying on the rankings.
• Unmeasured statistics. The introduction's figures on query logs, such as 80% of queries to one national library portal containing a proper name, are quoted from other studies, not measured here §1.
• Unknown noise effects. The survey calls for a better assessment of which OCR noise types are most harmful §8. Do not assume an error-rate threshold holds for a different collection without a test; §7.1 reports one threshold from one study.
• Evaluation details lost in summary. Some rows depend on details the survey does not give. For example, the evaluation setting of Grover et al. is not specified, and the settings for Moncla et al. are "not entirely clear" §6.1.
• Scope of "historical". The cut-off of 1979 is an acknowledged choice, and the definition is subjective §2.2. Systems tuned on 19th-century newspapers may not transfer to other periods, which the survey lists as an open problem §8.

Lineage

• Builds on: earlier surveys named in the paper: Nadeau et al. (2007), Yadav et al. (2018), Li et al. (2020), Nasar et al. (2021) §2.3. Only their names are recorded here; their content is not used.
• Compared against: not applicable; the survey compares no systems of its own.
• Superseded by: not stated.