Knowledge graph embedding

Overview

A knowledge graph stores facts as triples (head, relation, tail). Knowledge graph embedding places every entity and relation in a dense, low-dimensional space, defines a scoring function that rates how plausible a triple is, and learns the vectors by making observed triples score high 2303.13948 §5.1. The learned scores can then rank candidate triples that are not yet in the graph, which is the link prediction task used in most reported results 2303.13948 tab. 3.

The shared part across all variants is the triple score. The variants differ in how that score combines the head, relation and tail vectors, and the temporal variant adds a time argument. The survey gives the scoring functions for SME, ConvKB and R-GCN, but no loss, negative sampling or training settings for any of them 2303.13948 §5.1.3.

Five cards on wikillm cover this concept. The 2023 survey gives the static families below and runs no experiments. A 2024 survey on knowledge graph reasoning names the same embedding models, plus region-based and Gaussian embeddings, but reports no numbers 2412.10390 §3.2.1. A 2021 survey of graph learning treats knowledge-graph embedding as one application area among many, and names TransE, RESCAL and DistMult there without numbers 2105.00696 §3.5. A 2021 review of graph representation learning in biomedicine treats knowledge-graph variants as an extension of the node embedding similarity, and gives no equations or results for them 2104.04883 §2.1. The temporal variant comes from a paper on question answering over temporal graphs, which trains its own embeddings and reports results on its own dataset 2106.01515 §4 §6.

Variants

Tensor factorization

The graph is written as a binary tensor \mathcal{X} \in \mathbb{R}^{m \times m \times n} for m entities and n relations, one slice per relation. An entry is 1 when the triple is in the graph and 0 otherwise 2303.13948 §5.1.1. The tensor is then represented by embedding matrices for entities and relations. The survey does not specify the factorization itself 2303.13948 §5.1.1.

Models the survey files under this family, with Hits@10 on FB15K as reported by their original papers: ComplEx 84%, SimplE 83.8%, RotatE 88.4%, QuatE 90%, HolE 73.9%, RESCAL 44.1% 2303.13948 tab. 3. Card: Knowledge Graphs: Opportunities and Challenges.

Translation-based (TransE and extensions)

Each triple is embedded so that the head vector plus the relation vector lands near the tail vector, \mathbf{h} + \mathbf{r} \approx \mathbf{t} 2303.13948 §5.1.2. The survey names TransH and TransR as extensions, and lists TransD, TranSparse, STransE, TransA, KG2E and TransG as further members of the family 2303.13948 tab. 3.

The survey gives only the approximate equality above. It does not state the distance norm, the margin or the loss 2303.13948 §5.1.2. Reported Hits@10 on FB15K ranges from 47.1% for TransE to 88.2% for TransG 2303.13948 tab. 3; these come from different original papers.

Neural scoring (SME, ConvKB, R-GCN)

• SME 2303.13948 §5.1.3. The linear form projects the head and the tail, each combined with the relation, and takes their inner product 2303.13948 eq. 1. The bilinear form replaces the sum with an element-wise product 2303.13948 eq. 2. Reported Hits@10 on FB15K: 41.3% 2303.13948 tab. 3.
• ConvKB 2303.13948 §5.1.3. The three vectors of a triple are stacked into a matrix, convolved with a filter set, passed through a ReLU, and projected to a scalar 2303.13948 eq. 3. The filter count and shape are not stated. Reported Hits@10 on WN18RR: 52.5% 2303.13948 tab. 3.
• R-GCN 2303.13948 §5.1.3. A graph convolution in which each entity sums its neighbours' transformed vectors, normalised per relation, plus a self term 2303.13948 eq. 4. The survey calls it a relation-specific transformation. Reported Hits@10 on FB15K: 84.2% 2303.13948 tab. 3.

Walk-based and graph-neural methods on knowledge graphs

The 2021 graph learning survey covers two routes that apply to knowledge graphs without a triple score. Random walks may follow knowledge-graph paths through heterogeneous networks, and the walks are then fed to a word-embedding model 2105.00696 §2.3.3. Graph neural networks, in the form of GNN-based knowledge base completion, are a separate application 2105.00696 §3.5. The survey names both routes but gives no equations, training settings or results for either 2105.00696 §1.3. Card: Graph Learning: A Survey.

The biomedical review describes the same shallow idea: a node embedding is trained so that a graph similarity, such as co-occurrence on random walks, matches an embedding similarity. Knowledge-graph variants extend that graph similarity to relation types, and the vectors go to a separate classifier or regressor 2104.04883 §2.1. The review gives no training configuration or metric for it 2104.04883 §2.1. Card: Graph Representation Learning in Biomedicine.

Region-based and Gaussian embeddings

The 2024 survey lists region-based embeddings, which represent entities with regions such as boxes and spheres, and Gaussian embeddings as further members of the embedding group 2412.10390 §3.2.1. As a survey with no experiments, it gives no numbers for either group. The card does not give their scoring functions either, so take them from the original papers. Card: Neural-Symbolic Reasoning over Knowledge Graphs.

Temporal embeddings (TComplEx)

Each entity and relation is a complex vector, and each timestamp is a complex vector too 2106.01515 §4. The ComplEx score takes the real part of a product of the subject, relation and conjugated object vectors 2106.01515 eq. 1. TComplEx multiplies in a timestamp vector as well, so each fact is scored at a given time 2106.01515 eq. 2 §4.2. TNTComplEx mixes a time-sensitive and a time-insensitive relation vector 2106.01515 §4.2. The embedding rank is not stated 2106.01515 §4.2.

The paper uses TComplEx as the embedding inside a question-answering model, CronKGQA. Swapping ComplEx for TComplEx in that model raises time-answer Hits@1 from 0.057 to 0.549 and overall Hits@1 from 0.288 to 0.647 2106.01515 tab. 7. These are the authors' numbers on their own dataset and split, and they measure question answering, not link prediction. Card: Question Answering Over Temporal Knowledge Graphs.

When to use which

The survey does not give latency, memory or implementation-effort figures for any method, so the table only uses constraints that its text or tables address. The temporal row uses the authors' question-answering results.

Implementation sketch

The common core is a score over triples and a loop over observed facts. The survey gives the score functions but not the loss or the sampling, so the loss is left as a parameter. The temporal scorer adds a fourth argument. The pseudocode is ours.

# Build the binary KG tensor used by factorization methods  [arxiv:2303.13948 §5.1.1]
def build_kg_tensor(triples, num_entities, num_relations):
    # one m x m slice per relation; zero means the triple is absent  [§5.1.1]
    tensor = zeros(num_entities, num_entities, num_relations)
    for head, relation, tail in triples:
        tensor[head, tail, relation] = 1
    return tensor

# Generic training loop: maximise plausibility of observed triples  [§5.1]
# score(h, r, t) is one of the variants below; loss is not stated by the survey
def train(triples, entities, relations, score, loss, opt, steps):
    for step in range(steps):
        for h, r, t in sample_batch(triples):
            # plausibility of the observed triple  [§5.1]
            pos = score(E[h], R[r], E[t])
            # corrupted triple, sampling scheme not stated  [§5.1]
            neg = score(E[h], R[r], E[sample_entity(entities)])
            opt.step(loss(pos, neg))

# SME linear score of one triple  [arxiv:2303.13948 eq. 1]
def sme_linear_score(h, r, t, W_h1, W_h2, b_h, W_t1, W_t2, b_t):
    # project head and tail, each combined with the relation  [eq. 1]
    left = W_h1 @ h + W_h2 @ r + b_h      # (d,)
    right = W_t1 @ t + W_t2 @ r + b_t     # (d,)
    # inner product of the two projections  [eq. 1]
    return dot(left, right)

# TransE-style residual; the survey gives no norm  [§5.1.2]
def translation_score(h, r, t):
    # smaller distance means more plausible; L1 or L2 not stated  [§5.1.2]
    return -norm(h + r - t)

# TComplEx score of one timed triple  [arxiv:2106.01515 eq. 2]
def tcomplex_score(s, r, o, t):
    # complex vectors of one rank; the rank is not stated  [§4.2]
    # sum the elementwise product, with the object conjugated  [eq. 2]
    return real(sum(s * r * conj(o) * t))

Contradictions

• Ranking of the best models (within one survey). The survey names QuatE (90%), RMNN (89.9%) and KBGAN (89.2%) as the only strong results 2303.13948 §5.1.4. The table reports RMNN as triplet classification accuracy on WN11, while QuatE and KBGAN are Hits@10 on FB15K and WN18 2303.13948 tab. 3. The three numbers do not measure the same thing. The survey does not resolve this.
• Taxonomy differs between the two surveys. The 2023 survey files ComplEx and RESCAL under tensor factorization and TransE, TransH and TransR under translation 2303.13948 §5.1.1 §5.1.2. The 2024 survey puts TransE, DistMult, ComplEx, RotatE, TransH, TransR, RESCAL, QuatE and HolE in one embedding group 2412.10390 §3.2.1. Neither survey justifies its grouping here. This page follows the 2023 grouping, because it carries the numbers. The 2021 graph learning survey does not treat these models as embedding methods at all: it lists TransE, RESCAL and DistMult under knowledge-graph applications 2105.00696 §3.5. Its categories are different, so do not map its section numbers onto the 2023 families. The biomedical review places knowledge-graph variants inside shallow node embeddings with relation types added to the similarity [arxiv:2104.04883 §2.1], not in a triple-scoring family. These are different framings of one idea, and this page does not reconcile them.
• Mixed protocols in the literature table. The table mixes FB15K, WN18, WN18RR and WN11, and Hits@10 with accuracy. The values come from the original papers under their own protocols 2303.13948 tab. 3.
• Equation and text on R-GCN. The text calls R-GCN a relation-specific transformation 2303.13948 §5.1.3, but the printed update uses one weight W_i^{(l)} per neighbour with no relation index 2303.13948 eq. 4. Not resolved here.
• Not comparable across cards. The survey's Hits@10 values are link prediction on FB15K-style benchmarks 2303.13948 tab. 3. The CronKGQA values are Hits@1 for question answering on the authors' dataset 2106.01515 tab. 5. Do not rank one against the other.

Papers

• Knowledge Graphs: Opportunities and Challenges: a 2023 survey of knowledge graphs. It covers the embedding families above, with a literature table of reported link prediction and classification results, and proposes no new method.
• Question Answering Over Temporal Knowledge Graphs: adds timestamps to the complex-valued embedding (TComplEx) and uses it in a question-answering model, CronKGQA. It also builds a large temporal question dataset, CronQuestions.
• Neural-Symbolic Reasoning over Knowledge Graphs: A Survey from a Query Perspective: a 2024 survey of knowledge graph reasoning organised by query type. It covers the embedding group, complex logical query solvers and neural-symbolic methods, and reports no experiments.
• Graph Learning: A Survey: a 2021 survey of graph learning in four families, with a section on knowledge-graph applications. It reports no experiments and proposes no new method.
• Graph Representation Learning in Biomedicine: a 2021 review of graph representation learning for biomedical networks, including the knowledge-graph variant of shallow node embeddings. It runs no experiments and reports no benchmark numbers.