---
kind: concept
slug: conversational-question-answering
title: Conversational question answering
summary: >-
  Answering the next question in a chat about a passage, using earlier turns. One paper (CoQA) gives a dataset and three baselines: a seq2seq copy model, a span reader, and a pipeline of both. Best reported test F1 is 65.4 against 88.8 for humans (author-reported). Trade-off: span readers cannot give free-form answers, and the best recipe is not in the released code.
tags: [question-answering, conversational-qa, reading-comprehension, dataset]
papers: ["1808.07042"]
related: [knowledge-base-question-answering, dialogue-generation, dialogue-evaluation, benchmark-validity, human-evaluation]
status: draft
updated_at: 2026-10-19
---

## Overview

A conversational question answerer reads a passage and a chat so far, then
answers the newest question. Questions often depend on earlier turns, so a
system that sees only the current question cannot answer them. Standard
reading-comprehension sets use standalone questions with span answers, which do
not test this [arxiv:1808.07042 §1, tab. 1]. The CoQA paper is built around that
gap: it collects a dataset of such chats and tests simple systems on it
[arxiv:1808.07042 §3, §5]. The shared core of the variants is that each answer is
conditioned on the passage and the previous question-answer pairs. They differ in
whether the answer is a span copied from the passage, a generated string, or a
span followed by generation [arxiv:1808.07042 §5]. Word overlap is a weak signal
here: only 29.8% of CoQA questions share a content word with their rationale
[arxiv:1808.07042 §4.2].

## Variants

### Sequence-to-sequence with copy (PGNet)

An LSTM encoder reads the passage, the history and the question. An LSTM decoder
with attention writes the answer and can copy words from the passage
[arxiv:1808.07042 §5.1]. It is the only family here that writes free-form text
without a span step. Test F1 is 44.1 overall, against 26.3 for a plain
seq2seq without copy [arxiv:1808.07042 tab. 7]. Card:
[CoQA: A Conversational Question Answering Challenge](/papers/1808.07042).

### Span reader (DrQA)

A reading-comprehension model predicts a start and an end token in the passage.
The previous questions and answers are prepended to the current question
[arxiv:1808.07042 §5.2]. It can only return text that appears in the passage.
Test F1 is 52.6 overall [arxiv:1808.07042 tab. 7]. Card:
[CoQA: A Conversational Question Answering Challenge](/papers/1808.07042).

### Span reader with yes/no tokens (Augmented DrQA)

Two tokens, "yes" and "no", are appended to the passage. When the gold answer is
one of them, the model points to that token [arxiv:1808.07042 §5.2]. This is the
best reported system: test F1 65.4 overall, and dev F1 67.2 [arxiv:1808.07042
tab. 7]. The released code does not contain this recipe at the paper's commit,
so treat it as a paper-only result until you reproduce it (see the card's Paper
vs code section).

### Pipeline (DrQA+PGNet)

The span reader first points to the evidence, and the seq2seq model then writes
the answer from the question and that span. When the answer is a substring of
the rationale, the span reader is trained to predict the answer directly;
otherwise it predicts the rationale [arxiv:1808.07042 §5.3]. Test F1 is 65.1
overall, close to Augmented DrQA [arxiv:1808.07042 tab. 7]. It also handles
answers that are not spans: 24.8 F1 on counting and 46.4 on multiple choice
[arxiv:1808.07042 tab. 10]. Augmented DrQA scores higher on Yes and No answers
(76.2 and 64.0) [arxiv:1808.07042 tab. 10].

### Dataset only

CoQA itself has 127k question-answer pairs in about 8.4k passages from seven
domains, and 15.2 turns per passage on average [arxiv:1808.07042 tab. 2]. Dev and
test questions have four gold answers; training questions have one
[arxiv:1808.07042 §3.3]. Use it as a benchmark or as training data for any of the
families above.

## When to use which

| Situation | Use | Why |
|---|---|---|
| You need a benchmark or training data, not a model | CoQA dataset | Multi-turn data with free-form answers and rationales [arxiv:1808.07042 §3, tab. 2]. |
| Answers can be spans of the passage and yes/no is not needed | Span reader (DrQA) | Test F1 52.6 [arxiv:1808.07042 tab. 7]. Span-only answers cannot express answers that are absent from the text [arxiv:1808.07042 §1]. |
| Yes/No answers matter (11.1% and 8.7% of questions) | Augmented DrQA, after reproduction | Best reported test F1 (65.4) [arxiv:1808.07042 tab. 4, tab. 7]. The recipe is not in the released code. |
| Answers must be free-form, including counts or choices | Pipeline (DrQA+PGNet) | Counting and multiple-choice scores (24.8 and 46.4) are reported for this system [arxiv:1808.07042 tab. 10]. It runs two models. |
| The system cannot see earlier turns | Do not use this setup as is | Human F1 falls from 86.4 with full history to 19.9 with none, on 300 questions [arxiv:1808.07042 §6.3]. |
| You need an upper bound for the task | Human results | Human test F1 is 88.8 overall [arxiv:1808.07042 tab. 7]. |

## Implementation sketch

The common core is one function: take the passage, the history and the new
question, and return an answer. The pipeline shows how the pieces fit.
The history size is a parameter, tuned on dev; one previous turn gave the best
DrQA and pipeline scores, and two gave the best Augmented DrQA score
[arxiv:1808.07042 tab. 9].

```pseudocode
# Answer one turn of a conversation  [arxiv:1808.07042 §5.3]
# Input:  passage; history of (question, answer) pairs; new question
# Output: answer string

def answer_turn(passage, history, question, reader, generator, k):
    # Keep only the last k turns; k is tuned on dev  [arxiv:1808.07042 tab. 9]
    recent = history[-k:] if k > 0 else []
    # The span reader sees the previous turns and the question  [§5.2]
    evidence = reader.predict_span(passage, recent, question)
    # The generator writes the answer from the question and the evidence  [§5.3]
    return generator.generate(question, evidence)
```

Notes on the variants:

- For the seq2seq variant, the encoder input is the passage, then each past
  turn tagged as question and answer, then the current question
  [arxiv:1808.07042 §5.1].
- For the span reader, training targets are the answer's position inside the
  rationale when it appears there, and otherwise the best-F1 span
  [arxiv:1808.07042 §5.2].
- Augmented DrQA adds the yes/no tokens as extra pointer targets next to the
  passage spans [arxiv:1808.07042 §5.2].

## Pitfalls

- **Gold history at test time.** The paper evaluates with gold answers of earlier
  turns, not the model's own answers [arxiv:1808.07042 §6.1]. A system that
  feeds its own answers back is measured in a setting the paper does not report.
- **Rationales are training-only.** Do not give rationales to a test-time model
  [arxiv:1808.07042 §2].
- **Word F1 undercounts paraphrase.** Different wordings of the same answer can
  both be correct, and word overlap does not credit that [arxiv:1808.07042 §6.3].
- **Dev and test scores depend on four gold answers.** Training data has one
  answer per question [arxiv:1808.07042 §3.3].
- **Unanswerable answers are rare.** About 1.3% of answers are unanswerable
  [arxiv:1808.07042 tab. 4], so this case barely moves the overall F1.
- **Passage licenses are mixed.** The paper's CC BY 4.0 licence covers the paper
  only. The dataset's passages come from sources with different licences, which
  the card lists from the project page (not from the paper). Check them before
  redistributing the data.

## Contradictions

None between papers: only one paper with a card covers this concept. The paper
and its released code disagree on several settings (optimizer, dropout,
embeddings, and the Augmented DrQA recipe). The card's Paper vs code section
lists them.

## Papers

- [CoQA: A Conversational Question Answering Challenge](/papers/1808.07042): the
  CoQA dataset, its evaluation metric, and three baseline families with their
  author-reported scores.