Dependency treebank (annotated sentences with word-to-word dependency links)

Overview

A dependency treebank is a corpus of sentences in which each word is linked to one head word and labelled with the grammatical relation. Universal Dependencies (UD) is a scheme that applies the same word, morphology and relation inventory to many languages, so that treebanks can be compared 2004.10643 §1 §2. The wiki has two cards for this slug. One describes version 2 of the UD guidelines and the v2.5 release of the collection 2004.10643 §3 §4. The other describes a biomedical treebank that the scispaCy authors converted to UD v1.0 from the GENIA corpus [arxiv:1902.07669 §3.1]. The core idea is that each sentence is a tree rooted in one word, normally the main predicate 2004.10643 §2.3.

Variants

Basic single-head tree

Each word has exactly one head and one relation label, taken from a fixed inventory of 37 universal relation types. Relations between predicates and their arguments or modifiers are preferred over relations that pass through function words. Subtypes such as nsubj:pass and aux:pass mark passive constructions in a language 2004.10643 §2.3. Card: Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection.

Enhanced graph

An optional second graph on top of the basic tree. It adds relations and null nodes to make implicit structure explicit. The paper names five enhancement types: null nodes for elided predicates, propagation of shared dependents across conjuncts, controlled and raised subjects, relative pronouns (ref), and case-augmented modifiers such as nmod:on 2004.10643 §3.4. Only 24 treebanks in v2.5 carry enhanced graphs, and few implement all five enhancements 2004.10643 §3.4. Card: Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection.

Legacy v1 labels

Treebanks annotated under v1 use some labels that v2 renamed, removed or re-attached. The main changes are that passive subjects use nsubj, oblique nominals at clause level use obl, and cc and punct inside a coordination attach to the following conjunct instead of the first 2004.10643 §3.3. Card: Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection.

Biomedical treebank converted from GENIA (UD v1.0)

The scispaCy authors took the GENIA 1.0 treebank of McClosky et al., converted it to UD v1.0 with the Stanford Dependency Converter, and aligned the trees to raw PubMed abstract text. They kept the PubMed metadata and released the converted trees as a separate dataset 1902.07669 §3.1. Their parser trained on this data reaches 90.60 UAS and 88.79 LAS on the converted GENIA test set, against 92.64 and 91.23 for a Biaffine parser trained on the same converted data 1902.07669 tab. 4. The labels are v1, so they follow the v1 rules above 2004.10643 §3.3. Card: ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing.

When to use which

Implementation sketch

The record below is a sketch of one word. Field names are ours, not the paper's. The layers follow the paper's order: segmentation, morphology, syntax.

# One record per syntactic word, not per orthographic token  [§2.1]
# Fields follow the layers in §2.2 and §2.3; names are ours
word = {
    "lemma": ...,       # canonical dictionary form  [§2.2]
    "upos": ...,        # one of the 17 universal POS tags  [§2.2]
    "feats": {...},     # name-value pairs, language extras allowed  [§2.2]
    "head": ...,        # id of the single head; the root has none  [§2.3]
    "deprel": ...,      # one of the 37 universal relations  [§2.3]
    "enhanced": [...],  # optional extra (head, relation) arcs  [§3.4]
}

Coordination needs care in a converter. Under v2, the first conjunct is the head of the others via conj, and each cc or punct attaches to the conjunct that immediately follows it 2004.10643 §3.3.

Contradictions

No direct conflict between the two cards. Their numbers do not compare: the scispaCy parser is scored on a UD v1.0 conversion of GENIA, and the v2 card reports corpus counts with no model results 1902.07669 tab. 4 2004.10643 §4. Within 2004.10643, the authors say the gapping analysis gives fewer non-projective arcs, but the paper gives no figure to check the claim §3.3. Within 1902.07669, the terms for the GENIA source data are not stated in the paper, so check them before reusing the released treebank §3.1.

Papers

• ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing: a biomedical parser trained on a GENIA-derived UD v1.0 treebank, with its accuracy and the released conversion.
• Universal Dependencies v2: An Evergrowing Multilingual Treebank Collection: the v2 guidelines, the v1 to v2 changes and the v2.5 collection of 157 treebanks in 90 languages.