Anaphora resolution

Overview

A reference, such as a pronoun, points back to an earlier item in the text. Resolving it means finding that item. The task is often mixed up with coreference resolution, which groups all mentions of one entity. The survey keeps the two apart: anaphora links a reference to an antecedent, coreference clusters mentions 1805.11824 §1. The survey covers English NLP only 1805.11824 §1.

The invariant all variants share: each mention either links to one earlier mention or starts a new entity. Systems differ in how they find mentions, how they score candidate antecedents, and what features or learned representations they use 1805.11824 §8.

This page rests on one survey card, Anaphora and Coreference Resolution: A Review. It describes the systems from their original papers, which have no cards here. Check those originals before implementing any of them.

Variants

Rule-based systems

Early systems apply discourse properties and syntactic and semantic constraints. The survey names the BFP centering algorithm as an early discourse-based method for pronouns 1805.11824 §8.1. It also names a multi-sieve deterministic system as the best rule-based system on CoNLL 2012 1805.11824 §8.1. The survey's table gives the rule-based row (Lee et al. 2013) a CoNLL F1 of 54.82 1805.11824 tab. 7. Card: Anaphora and Coreference Resolution: A Review.

Statistical and machine-learning systems

These use mention-pair or mention-ranking models trained on hand-built features. The survey also lists joint inference with integer linear programming and graph partitioning with conditional random fields 1805.11824 §8.2 1805.11824 tab. 5. The survey's CoNLL F1 for this group runs from 61.40 (Durrett and Klein 2013) to 63.02 (Clark and Manning 2015, entity-centric) 1805.11824 tab. 7. Card: Anaphora and Coreference Resolution: A Review.

Neural mention-ranking systems

A feed-forward or recurrent network scores each candidate antecedent. The survey groups Wiseman et al. 2015 and 2016, and Clark and Manning 2016 in their "improving" and "deep" forms 1805.11824 §8.3. The second Clark and Manning variant replaces the heuristic loss with a reinforcement-learning objective 1805.11824 §8.3. Their CoNLL F1 runs from 63.39 to 65.73 in the survey's table 1805.11824 tab. 7. Card: Anaphora and Coreference Resolution: A Review.

End-to-end neural systems

Lee et al. 2017 detects mentions and links them in one model. A bi-LSTM reads the words, every span up to width 10 is scored, the top-scoring spans are kept, and each is paired with the preceding 250 mentions 1805.11824 §8.3 1805.11824 tab. 6. The survey reports 68.80 CoNLL F1 for it, the best figure in its table 1805.11824 tab. 7. The survey also says this was the best system it knew of, as of its 2018 posting 1805.11824 §8.3. Card: Anaphora and Coreference Resolution: A Review.

When to use which

Training details are missing. The survey states no learning rate, optimiser, epoch count or batch size for any system 1805.11824 tab. 6. Do not implement a neural variant from this page alone. Read the original paper.

Implementation sketch

The common core of mention-ranking systems. Lines follow the survey's descriptions of how candidate antecedents are scored 1805.11824 §8.2 1805.11824 §8.3.

# Resolve each mention to an earlier mention or to "new entity"
# Input:  mentions in document order, score(m, c) for a pair, window size
# Output: dict mapping each mention to its antecedent, or None

def resolve(mentions, score, window):
    links = {}
    for i, m in enumerate(mentions):
        # Only look back a bounded number of mentions. Lee et al. 2017
        # used 250 [arxiv:1805.11824 §8.3]
        candidates = mentions[max(0, i - window):i]
        # Score each candidate and the "no antecedent" option. The survey
        # does not say how the null score is computed for every system
        best, best_score = None, score(m, None)
        for c in candidates:
            s = score(m, c)
            if s > best_score:
                best, best_score = c, s
        links[m] = best
    return links

End-to-end systems add mention detection and span pruning before this loop. The survey gives no pruning fraction for Lee et al. 2017 1805.11824 tab. 6.

Evaluation

Results are not comparable across papers unless the metric and the setup match. The survey says the papers use different metrics, splits and mention-detection setups 1805.11824 §5.8 1805.11824 §9.

• CoNLL F1 is the unweighted mean of MUC, B-cubed and CEAFe F1 1805.11824 §5.5 1805.11824 eq. 12. Most rows in the survey's table use this score.
• MUC is link-based and ignores singleton clusters 1805.11824 §5.2 1805.11824 §5.8.
• B-cubed is per-mention. The survey's formula is ambiguous about the weighting, so check the original definition 1805.11824 §5.1.
• CEAF first finds a one-to-one mapping between output and gold clusters, then scores it. The survey's table uses CEAFe 1805.11824 §5.3 1805.11824 tab. 7.
• LEA weights links by entity size, which the survey says is meant to avoid the mention-identification effect 1805.11824 §5.7.

Contradictions

• Clark and Manning 2016 (improving), CoNLL F1. The survey's text gives 65.39 1805.11824 §8.3. Its table gives 65.29 for the same system 1805.11824 tab. 7.
• Clark and Manning 2016 (deep), which score is which. The survey's text calls 65.73 and 63.40 the CEAF and B-cubed scores 1805.11824 §8.3. Its table lists 65.73 as the CoNLL score, with CEAFe 59.23 and B-cubed 63.40 1805.11824 tab. 7. The survey gives no way to settle this.
• Lee et al. 2017, score definition. The survey's text says the final score adds the two mention scores and the pair score 1805.11824 §8.3. Its equation gives only the pair term 1805.11824 eq. 28.

Papers

• Anaphora and Coreference Resolution: A Review: a survey of AR and CR. It defines the metrics, groups systems into rule-based, statistical and neural, and tabulates CoNLL 2012 English scores. It runs no experiments of its own.