Relational graph convolution

Overview

A relational graph layer updates each node from its neighbours, as a graph neural network does, but keeps a separate weight matrix for each type of edge. An edge's type is a label such as "the neighbour came earlier and spoke with the same speaker". Messages are grouped by type, each group is transformed by its own weight matrix and scaled, and the sums are combined with the node's own transformed features 1908.11540 §3.3.2 eq. 2. The authors present their first layer in the style of R-GCN (arXiv:1703.06103). That paper has no card on wikillm, so nothing from it is used here.

The invariant every variant shares: messages are grouped by relation, each group has its own weight, and the groups are combined into one update per node. The variants differ in how relations are defined, how messages are weighted and normalised, and whether a self term and an activation are included.

Only one card on wikillm uses this, so the variants below are design choices inside that single model, not a survey of the family.

Variants

Relation-specific layer with attention-weighted messages

Each incoming edge gets a weight from a softmax over its window, so the weights of a node's incoming edges sum to 1 1908.11540 eq. 1. Each message is multiplied by the weight matrix for its relation, scaled by its edge weight and by a per-relation constant c_{i,r}, and summed. A self term with its own weight matrix is added on top 1908.11540 eq. 2.

The paper's relation for an edge combines the speaker of each endpoint with whether the neighbour comes before the target. A self-edge falls in the "not before" relation of its speaker pair 1908.11540 tab. 1. This gives up to 2M² relation types for M speakers, which is 8 for two speakers 1908.11540 §3.3.2. The paper states that c_{i,r} is either the neighbour count for that relation or learned, and does not say which one it used 1908.11540 §3.3.2.

Relation-free neighbourhood layer

The second graph layer in the same model drops relation types. It applies one weight matrix to every neighbour and does not weight them by attention 1908.11540 eq. 3. It needs no relation labels, and it has one weight matrix rather than one per relation. The paper's ablations remove the whole speaker-level encoder, which contains both graph layers, so the card gives no separate number for this layer alone 1908.11540 tab. 5.

Coarser relation sets

The paper also runs the same layer with fewer relation types. On IEMOCAP, the table gives one weighted F1 per row 1908.11540 tab. 6:

• speaker pair and order (full set): 64.18;
• speaker pair only, no order (M^2 relations): 62.52;
• order only, no speaker pair (2 relations): 61.03;
• one relation for all edges: 60.11.

When to use which

Implementation sketch

Pseudocode of one relation-specific layer, following the shape of the paper's first graph layer. Edges exclude self-loops, which the self term covers.

# Relation-specific graph layer, the shape of the paper's first layer
# [arxiv:1908.11540 eq. 2]
# Input:  h (N, D_in) node features; edges [(i, j, r)] for neighbours j
#         of node i under relation r; weight[(i, j)] attention weights,
#         including weight[(i, i)] for the self term
#         W_rel[r] (D_out, D_in) per relation; W_self (D_out, D_in)
# Output: out (N, D_out)
def relational_layer(h, edges, weight, W_rel, W_self, sigma):
    # Self term, scaled by its own attention weight  [eq. 2]
    out = [weight[(i, i)] * (W_self @ h[i]) for i in range(N)]

    # Neighbour count per (target, relation) for c_{i,r}  [eq. 2]
    count = {}
    for (i, j, r) in edges:
        count[(i, r)] = count.get((i, r), 0) + 1

    # Relation-specific, attention-weighted messages  [eq. 2]
    for (i, j, r) in edges:
        c = count[(i, r)]  # fixed |N_i^r|; the paper also allows a learned c
        out[i] += weight[(i, j)] / c * (W_rel[r] @ h[j])

    # Activation sigma; the paper gives ReLU as an example  [§3.3.2]
    return [sigma(x) for x in out]

Where variants differ: how the relation id is computed from the two nodes, whether c_{i,r} is the neighbour count or learned, whether the self term is attention-weighted (the paper does this), and whether sigma is applied (the paper applies it; see Contradictions).

Contradictions

• Activation in graph layers. The paper applies a nonlinearity in both graph layers 1908.11540 eq. 2, eq. 3. The public code applies none in these layers, as recorded on the DialogueGCN card under Paper vs code.
• Edge weights. The paper scores each pair with a bilinear form in the two node states 1908.11540 eq. 1. The public code uses one learned vector per absolute position with a fixed maximum length. The card records this under Paper vs code.
• Reported numbers. The paper reports a mean over 5 runs of 64.18 weighted F1 on IEMOCAP 1908.11540 §5.1. The public README reports 64.67 and 64.44 on two machines, under different settings (LSTM encoder, other learning rate, dropout and L2). The public script also picks the best test epoch, so the paper's numbers may be optimistic. Both points are recorded on the DialogueGCN card.

Papers

• DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation: the only card here that uses a relation-specific layer with attention-weighted messages, and the source of the relation-set ablation above.