Relation network

Overview

A relation network scores a set of objects by comparing every pair of them. One learned function g takes two object embeddings and returns a relation vector. The relation vectors are summed over all pairs, and a second function f turns that sum into a score 1905.12506 eq. 2 1905.12506 §3. The pair sum is the shared core: any two objects can interact, and the model does not need to know in advance which pairs matter.

The only cited paper uses this design as a Wild Relation Network (WReN) for 3 × 3 Raven-style puzzles. Each of the eight context panels and each candidate answer is embedded, and each candidate is scored against the context 1905.12506 §3. The relation network itself comes from Santoro et al. 2017, which the paper cites 1905.12506 §3. No card exists for that paper, so its content is not used on this page.

Variants

The paper varies the input to the relation network rather than its form. The design fixed by eq. 2 stays the same across the variants below.

Learned from scratch

The panel embedding is a CNN trained together with the reasoner, with the same architecture as the disentanglement encoders 1905.12506 §4.2.1. The paper reports that these runs often stall. Some predict every answer with equal probability, and runs differ widely 1905.12506 §4.2.1. Card: Are Disentangled Representations Helpful for Abstract Visual Reasoning?.

Frozen pretrained representation

The reasoner receives a frozen pretrained representation instead of a learned one 1905.12506 §4.2.1. The card does not report a separate accuracy for this input. The paper's headline numbers, about 80% on dSprites and about 90% on 3dshapes, are averages over the models of the initial study, so they are not the accuracy of any one input or method 1905.12506 §4.2.1. Card: Are Disentangled Representations Helpful for Abstract Visual Reasoning?.

Ground-truth factor values

The reasoner receives the true factor values of each panel instead of an image embedding 1905.12506 §4.2.1. The paper uses this as a comparison point. The card gives no separate accuracy for it. Card: Are Disentangled Representations Helpful for Abstract Visual Reasoning?.

Disentangled encoder embeddings

The paper's main study. Each panel is embedded by an unsupervised disentangled encoder: beta-VAE, FactorVAE, beta-TCVAE, DIP-VAE-I or DIP-VAE-II 1905.12506 §2. The study trains 360 encoders 1905.12506 §4.1. Each one feeds 10 sampled relation-network configurations, which gives 3600 reasoners 1905.12506 §4.2.2. The paper then correlates each encoder's disentanglement scores with reasoning accuracy over training 1905.12506 §4.2.2. Card: Are Disentangled Representations Helpful for Abstract Visual Reasoning?.

When to use which

Implementation sketch

The shared core is the pair sum in eq. 2. The paper leaves some details open. The pseudocode marks them.

# Score one candidate answer against the context  [arxiv:1905.12506 eq. 2]
# context: 8 panels; answer: 1 panel; output: one scalar score
def relation_score(context, answer):
    # Embed all 9 panels with one shared encoder  [arxiv:1905.12506 §3]
    nodes = embed(concatenate(context, answer))   # (9, d)
    # Apply g to every ordered pair of nodes and sum the results
    # [arxiv:1905.12506 eq. 2]
    # Whether a node is paired with itself is not stated in the paper
    relations = sum(g(a, b) for a in nodes for b in nodes)
    # Turn the summed relations into one score  [arxiv:1905.12506 eq. 2]
    return f(relations)

The study does not fix one architecture. It samples the edge MLP g with 256 or 512 units and 2 to 4 hidden layers, and the graph MLP f with 128 or 256 units and 1 or 2 hidden layers 1905.12506 app. A 1905.12506 §4.2.2. Training uses batches of 32 fresh puzzles, Adam, and 100K steps 1905.12506 §4.2 1905.12506 app. A. The learning rate is sampled from 1e-2, 1e-3 or 1e-4 1905.12506 app. A. The card's Paper vs code section lists further differences between eq. 2 and the released code, which an implementer should read before copying the equation.

Contradictions

Only one card covers this concept, so no two cited papers disagree. Within the paper, the central claim concerns sample efficiency in the few-sample regime 1905.12506 §4.2.2 1905.12506 §5. At 100K steps, the paper reports no strong correlation with disentanglement scores, and reconstruction error correlates more 1905.12506 §4.2.2. Read the claim as limited to the early phase of training. The paper also finds that MIG and SAP correlate weakly in the few-sample regime 1905.12506 §4.2.2.

Papers

• Are Disentangled Representations Helpful for Abstract Visual Reasoning?: builds two Raven-style puzzle sets and trains a relation-network reasoner on 360 disentangled encoders, to test whether disentanglement helps reasoning.