Anaphora and Coreference Resolution: A Review

In plain words

Pronouns and other short words often point back to something said earlier: "Sara bought a bike. She rode it home." Working out which words refer to the same thing is needed for reading a text properly. The field splits this into two related jobs. Anaphora resolution links a reference back to an earlier item. Coreference resolution groups all the mentions that refer to one entity.

This review explains what counts as a reference, which rules and clues help find the right earlier item, and how the results are scored. It then walks through the main approaches, from hand-written rules to neural networks, and lists the datasets and public tools. It also looks at why these tools matter for sentiment analysis, where pronouns can point to a product or to one of its aspects.

The authors argue that the scores are hard to compare because the metrics differ, and that the field needs clearer standards and more evaluation. The paper is a map and comparison, not a new system.

TL;DR

• A survey, not a method. It covers anaphora resolution (AR) and coreference resolution (CR) in English NLP §1.
• It defines the common CR metrics (MUC, B-cubed, CEAF, ACE, BLANC, LEA) and the CoNLL score, which averages MUC, B-cubed and CEAF F1 §5 eq. 12.
• Systems are grouped into rule-based, statistical/ML and deep learning. The deep models are compared by their CoNLL 2012 F1 §8 tab. 7.
• The paper runs no experiments of its own. Every number is taken from cited work.
• Use it to pick a metric and to find which original paper to implement. Do not implement a method from this card alone.

Problem

Anaphora and coreference are central to discourse understanding, but progress in AR has been slow, and the two tasks are often confused §1. Many systems depend on hand-written features, so they do not transfer well across domains §1 §8.3. Results in the literature also cannot be compared directly, because the papers use different metrics, splits and mention-detection setups §5.8 §9.

Method

The paper is a survey. Its "method" is a taxonomy plus the formulas it reports.

1. Reference types and constraints §2 §3 §4.

1. The types covered are zero anaphora, one-anaphora, demonstratives, presuppositions, split antecedents, contextual disambiguation, pronominal (indefinite, definite, adjectival), cataphora and bridging §2.
2. Non-anaphoric pronouns (clefts, pleonastic "it", extraposition) are treated as a separate class from anaphoric ones §3.
3. Constraints used to filter candidate antecedents: gender, number, verb selectional preferences, person, grammatical role, recency, repeated mention, discourse structure and world knowledge §4.

2. Evaluation metrics §5. Each metric is a precision and recall over clusters or links. Formulas are cited below.

1. B-cubed: per-mention precision and recall combined as a weighted sum; the paper uses weights of 1/N §5.1 eq. 1 eq. 2.
2. MUC: link-based. It counts the link changes needed to turn the output into the gold clustering §5.2 eq. 3 eq. 4 eq. 5.
3. CEAF: first find the best one-to-one mapping between output and gold clusters, then score the mapped pairs with a similarity function (four variants, phi1 to phi4) §5.3 eq. 6 to eq. 11.
4. ACE value: similar to CEAF, but uses true and false positives and negatives and does not normalise §5.4.
5. CoNLL score: the unweighted mean of MUC F1, B-cubed F1 and CEAF F1, as used in the CoNLL 2012 shared task §5.5 eq. 12.
6. BLANC: a Rand-index style score over coreferent and non-coreferent links §5.6 eq. 13 to eq. 18.
7. LEA: link-based, weighted by entity size and resolution score, which the paper introduces to avoid the mention-identification effect §5.7 eq. 19 to eq. 22.

# CoNLL 2012 headline score  [§5.5] [eq. 12]
# Input:  F1 values for MUC, B-cubed and CEAF (same units)
# Output: CoNLL F1, the unweighted mean of the three

def conll_score(f1_muc, f1_b3, f1_ceaf):
    # Equal weight for each of the three metrics  [§5.5]
    return (f1_muc + f1_b3 + f1_ceaf) / 3

3. Algorithms §8.

1. Rule-based: the earliest systems use discourse properties and syntactic and semantic constraints. The paper names the BFP centering algorithm as an early discourse-based method for pronouns §8.1. It cites a multi-sieve deterministic system as the best rule-based CoNLL 2012 system §8.1 fig. 3 tab. 4 tab. 7.
2. Statistical/ML: mention-pair models, mention-ranking models, joint inference with integer linear programming, and graph partitioning with CRFs, trained on hand-built features §8.2 tab. 5.
3. Deep learning §8.3 tab. 6:
   • Wiseman et al. 2015: a feed-forward mention-ranking model. Mention and mention-pair features pass through tanh layers, and the model is trained with a slack-rescaled latent-variable loss. The loss is a sum over training examples of a mistake-specific cost times a margin term eq. 23 to eq. 26.
   • Wiseman et al. 2016: adds entity-level information from an RNN that reads each candidate cluster §8.3.
   • Clark and Manning 2016 (improving): a mention-pair encoder and a cluster-pair encoder that pools mention-pair vectors with max and average pooling. The mention-ranking model is pre-trained, and a cluster-ranking step then chooses between merge and pass actions §8.3 fig. 4.
   • Clark and Manning 2016 (deep): replaces the heuristic loss with a reinforcement-learning objective (REINFORCE with reward rescaling) §8.3.
   • Lee et al. 2017: end-to-end. It jointly detects mentions and links them. It uses a bi-LSTM over word vectors, scores every span up to width 10, keeps the top-scoring spans, and scores pairs among the preceding 250 mentions §8.3 fig. 5 fig. 6.

# Mention representation for span i  [§8.3] [eq. 27]
# Input:  BiLSTM states per token, start and end token indices of span i
# Output: vector g_i (one per span)

def mention_representation(states, start, end, extra_features):
    # Attention-weighted sum of the word states inside the span  [§8.3]
    head = attention_over_span(states, start, end)
    # Concatenate boundary states, head vector and extra features  [eq. 27]
    return concat(states[start], states[end], head, extra_features)

# Antecedent score for mentions i and j  [§8.3] [eq. 28]
# Input:  g_i, g_j (mention vectors), pair_features (speaker, distance)
# Output: scalar score s_a(i, j)

def pair_score(g_i, g_j, pair_features):
    # Both mentions, their elementwise product and the pair features  [§8.3]
    pair_input = concat(g_i, g_j, g_i * g_j, pair_features)
    # Feed-forward network, then a linear read-out  [eq. 28]
    return dot(w_a, ffnn_a(pair_input))

# Training objective: marginal log-likelihood over gold antecedents
# [§8.3] [eq. 29]
# Input:  scores[i] for each kept mention i and its candidate antecedents
#         gold_antecedents[i]: the gold antecedents of mention i
# Output: scalar objective to maximise

def marginal_log_likelihood(scores, gold_antecedents):
    total = 0
    for i in kept_mentions:
        # Conversion of scores to probabilities is not stated  [§8.3]
        probs = normalise(scores[i])
        # Probability mass on the gold antecedents of mention i  [eq. 29]
        total = total + log(sum(probs[j] for j in gold_antecedents[i]))
    return total

4. Sentiment analysis §11: resolving pronouns to product aspects can help extract opinions per aspect. One cited study reports an average gain of about 10% over baseline coreference modules on its sentiment task. Another reports reaching about half of the gain that perfect anaphora resolution would give §11.

5. Datasets and tools §7 §10. The English datasets are MUC-6 and MUC-7, ACE, CoNLL-2012 (OntoNotes 5.0), SemEval-2010, ARRAU, GUM and others, with sizes and languages in tab. 1 and tab. 2. Off-the-shelf tools are listed in tab. 8: GuiTAR, BART, ARKref, Reconcile and the Stanford CoreNLP deterministic, statistical and neural systems.

Hyperparameters

This survey has no training setup of its own. The table lists the values the paper states for the systems it describes.

Results

Author-reported. The paper does not run experiments. The numbers below are the CoNLL 2012 English test scores as tabulated by the survey. Setup: OntoNotes 5.0 with the CoNLL 2012 splits tab. 2 tab. 7. Each row's numbers come from the cited paper.

The survey's own statement is that the end-to-end model of Lee et al. 2017 was the best system it knew of. That claim is dated to the paper's 2018 posting and reflects results up to 2017 §8.3.

Paper vs code

No official code.

The paper is a survey and describes no implementation of its own. Searches run: (1) the paper's links are only CTAN package pages and a Hugging Face demo API used for a figure §11, which is a third-party demo, not this paper's code; (2) no project page is linked; (3) GitHub repository search for "anaphora coreference review" and "anaphora resolution survey" with the declare-lab organisation returned no repository by the authors or their lab; the broader "anaphora coreference" search returned 12 unrelated repositories; the declare-lab repository list was checked only on its first page (10 of 84 repositories). The Stanford, Berkeley and BART tools the survey discusses are other authors' work and are not used here.

Pitfalls

• Survey, not a method. Nothing here is a complete algorithm. To implement a system, read the original paper of the approach (Lee et al. 2017, Clark and Manning 2016 and so on). Those papers are not on this wiki, so this card cannot verify their details.
• Text and table disagree on Clark and Manning 2016 (improving). The text gives an F1 of 65.39 on the CoNLL English task §8.3. Table 7 lists 65.29 for the same system tab. 7.
• Text mislabels the deep RL result. The text says 65.73 and 63.4 are CEAF and B-cubed scores for Clark and Manning 2016 (deep) §8.3. Table 7 lists 65.73 as the CoNLL score, with CEAFe 59.23 and B-cubed 63.40 tab. 7. Use the table, and check the original.
• Lee et al. 2017 scoring. The text says the final score is a sum of the two mention scores and the pair score §8.3. Equation 28 gives only the pair term eq. 28. The mention-score terms are not written out.
• Lee et al. 2017 pruning and training. The fraction of spans kept, the FFNN sizes and the handling of mentions with no gold antecedent are not stated §8.3. The equation for the objective does not say how scores become probabilities eq. 29.
• Cost terms. The Wiseman et al. 2015 loss depends on cost terms whose values are not stated §8.3. The text says they were set after experiments and are dataset-specific.
• B-cubed definition is ambiguous. The formula sums over N entities with weights 1/N, but the text describes per-mention precision and recall §5.1 eq. 1 eq. 2. The standard definition sums over mentions. Check the original Bagga and Baldwin paper before implementing.
• MUC and singletons. The MUC formulas divide by |r| - 1 eq. 4 eq. 5. Singleton clusters give zero in that denominator. The paper does not say how they are handled, and notes that MUC ignores singletons §5.8.
• CEAF variant. The CoNLL formula eq. 12 says CEAF without naming the variant. The tables use CEAFe tab. 7. The paper does not say how the optimal one-to-one mapping is computed §5.3.
• BLANC formula typo. Equation 16 is labelled P_c, but it is the non-coreference precision and should be P_n eq. 16. Use the definition from the original paper.
• LEA undefined case. The LEA resolution score divides by the number of links of an entity eq. 20. A singleton has no links under a pair count, and the paper does not say how this case is handled §5.7.
• Results are dated and not reproduced here. Every number is from the cited paper, computed with that paper's preprocessing and mention setup. The survey does not rerun any system. The CoNLL 2012 column is the only one used for ranking.
• Equation numbers. Numbers in this card were counted from the TeX source in order of appearance. Check them against the PDF before citing externally.
• Figures. Figures are described in words and cited (for example fig. 3). The paper's own figures are not reproduced.

Lineage

• Builds on: the earlier AR and CR surveys the paper names (Mitkov 1999 for AR; Ng 2010 for CR) §1. Neither is on this wiki.
• Compared against: the metrics in §5 and the systems in tab. 7 and tab. 8.
• Superseded by: not stated.