Paper · 2019-05-26 · cs.LG, cs.SI, stat.ML
Graph Attention Auto-Encoders
GATE is an unsupervised graph auto-encoder: attention-weighted encoder layers over the graph, mirrored by decoder layers that rebuild node features, plus a term that pulls neighbouring nodes' embeddings together. Author-reported node classification on Cora, Citeseer and Pubmed beats unsupervised baselines and, on Cora and Pubmed, the supervised GAT.
Graph neural networkAttention mechanismAutoencoderAttributed network embeddinggraph-representation-learningunsupervised-learningnode-classificationauto-encoder
arXivPDFCodeMarkdown15 min read
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Many datasets are graphs: papers that cite each other, users who follow each other, web pages that link. Each node usually also carries features, such as the words in a paper. Most ways of turning nodes into vectors use only the links. The stronger methods that also use the features need labels, and labels are expensive to get.
GATE is an auto-encoder for such graphs. The encoder takes each node's features and combines them with its neighbours' features, giving more weight to the neighbours it finds more relevant. Several such layers stack up, so information spreads further through the graph. A decoder runs the same steps in reverse and tries to rebuild the original features from the final vectors. A second training signal asks connected nodes to end up with similar vectors, so the links are rebuilt too. Nothing in this training uses labels. The paper says the model does not need the whole graph in advance, so it can also embed nodes it never saw during training.
TL;DR
- GATE is an unsupervised graph auto-encoder built from self-attention layers over a node's neighbours, with a mirror-image decoder §4.
- Training minimises a feature reconstruction error plus a weighted term that makes neighbouring nodes' embeddings similar §4.3.
- Author-reported node classification on Cora, Citeseer and Pubmed: best unsupervised result on Cora and Pubmed, and beats the supervised GAT on Cora and Pubmed in the transductive setup §5.4, tab. 3a.
- The official code is a Python 2 and TensorFlow 1.13 implementation of one variant only. Its loss and matrix form differ from the paper in several places (see Paper vs code).
Problem
Most graph embedding methods use only the graph structure and ignore node attributes §1. The most successful attributed methods, GCN and GAT, need labels, which many applications do not have §1. Earlier unsupervised graph auto-encoders reconstruct either the structure or the attributes, not both §1, §2. Many unsupervised methods also cannot handle unseen nodes (inductive learning) §1. The paper's claim is that GATE does not need the graph structure upfront, so it can be applied to inductive learning §1, §4. The paper's setting is an undirected, unweighted graph with self-loops in the adjacency matrix §3. Its notation is summarised in tab. 1.
Method
GATE has an encoder of L layers and a decoder of L layers, with the same L. Node features are the initial representations, h_i^(0) = x_i §4.1. The paper's Figure 1 shows a two-layer example for one node's features fig. 1. The paper uses more layers both to add capacity and to propagate representations further through the graph §4.1. The experiments use L = 2 and width 512 for each layer, and the activation σ is the identity §5.3.
Encoder layer k §4.1: for each edge (i, j) the layer scores how relevant neighbour j is to node i eq. 1. The scores are normalised with a softmax over each node's neighbourhood, which includes the node itself eq. 2. The new representation of node i is the attention-weighted sum of its neighbours' transformed representations eq. 3.
Decoder layer k §4.2: it mirrors encoder layer k. It has its own weights and attention, and it maps the representations of layer k back to layer k−1 eq. 4–6. After the last decoder layer, the output is the reconstructed feature matrix §4.2.
Tied variant §5.3: the experiments use a variant with half the trainable parameters, where the decoder weights are the transposed encoder weights and the decoder attention matrix is the encoder's §5.3. The untied form above is the general architecture §4.2. The official code implements only the tied variant (see Paper vs code).
Loss §4.3: the feature term is the sum over nodes of the L2 norm of the reconstruction error eq. 7. The structure term is a negative log-sigmoid of the dot product between each node and each of its neighbours, including itself eq. 8. The total is the feature term plus λ times the structure term eq. 9. The paper has no negative pairs.
Matrix form §4.4: the paper gives an equivalent matrix form for sparse operations, with an attention matrix C^(k) that is zero outside the edges of A eq. 10–13. Algorithm 1 shows the forward pass in that form alg. 1. Its parameter-update line is commented out in the source, so the algorithm shows only the forward pass alg. 1. The text names Adam as the optimiser §5.3.
Complexity §4.5: the paper states one training iteration costs O(NFD + ED), where N and E are the numbers of nodes and edges, F the feature count and D the largest layer width eq. 18. It states that space is linear in N and E, given sparse operations §4.5.
Pseudocode of the tied variant used in the experiments. Rows are nodes, as in the official code (see Paper vs code). Each line is cited to its equation or section.
# GATE loss, tied variant used in the experiments [§5.3] [alg. 1]
# Input: X (N, F) node features; A (N, N) adjacency with self-loops [§3]
# Output: H (N, d) node embeddings; loss (scalar)
def gate_loss(X, A, weights, src_vecs, dst_vecs, lambda_):
# Encoder: attention over neighbours, layer by layer [§4.1]
H = X
attentions = []
for k in range(len(weights)):
# Attention matrix, zero outside the edges of A [eq. 1-2] [eq. 10]
C = attention(H @ weights[k], A, src_vecs[k], dst_vecs[k])
attentions.append(C)
# Weighted sum of transformed neighbours [eq. 3] [eq. 13]
H = C @ (H @ weights[k]) # (N, d_k)
# Decoder: mirror layers, transposed weights, same attention [§5.3]
X_hat = H
for k in reversed(range(len(weights))):
# Reuse the encoder attention matrix of layer k [eq. 6] [§5.3]
X_hat = attentions[k] @ (X_hat @ weights[k].T) # (N, d_{k-1})
# Feature term: sum over nodes of the L2 error norm [eq. 7]
feature_term = sum(norm(X[i] - X_hat[i]) for i in range(N))
# Structure term: -log sigmoid(h_i . h_j) over edges, self-loops
# included [eq. 8] [§4.3]
structure_term = 0
for i, j in edges_of(A):
structure_term += -log(sigmoid(dot(H[i], H[j])))
# Total objective [eq. 9]
return feature_term + lambda_ * structure_term, H
def attention(Z, A, src_vec, dst_vec):
# Score per edge i <- j: sigmoid(v_s . z_i + v_r . z_j) [eq. 1]
scores = sigmoid(
A * (Z @ src_vec)[:, None] + A * (Z @ dst_vec)[None, :])
# Softmax over each row's neighbours; entries outside A are zero
# [eq. 2] [§4.4]
return softmax_over_edges(scores, A)Training §5.3: Adam with initial learning rate 10^-4. The code runs one full-batch step per epoch (see Paper vs code). The number of epochs depends on the dataset §5.3.
Key equations (as typeset in the paper; the matrix-form equations are internally inconsistent, see Pitfalls):
e_{ij}^{(k)} = \text{Sigmoid}\left({\mathbf{v}_s^{(k)}}^T \sigma\left(\mathbf{W}^{(k)} \mathbf{h}_i^{(k-1)}\right) + {\mathbf{v}_r^{(k)}}^T \sigma\left(\mathbf{W}^{(k)} \mathbf{h}_j^{(k-1)}\right)\right) eq. 1
\alpha_{ij}^{(k)} = \frac{\exp\left(e_{ij}^{(k)}\right)}{\sum_{l \in \mathcal{N}_i} \exp\left(e_{il}^{(k)}\right)} eq. 2
\mathbf{h}_i^{(k)} = \sum_{j \in \mathcal{N}_i} \alpha_{ij}^{(k)}\, \sigma\left(\mathbf{W}^{(k)} \mathbf{h}_j^{(k-1)}\right) eq. 3
\widehat{\alpha}_{ij}^{(k)} = \frac{\exp\left(\widehat{e}_{ij}^{(k)}\right)}{\sum_{l \in \mathcal{N}_i} \exp\left(\widehat{e}_{il}^{(k)}\right)} eq. 4
\widehat{e}_{ij}^{(k)} = \text{Sigmoid}\left({\widehat{\mathbf{v}}_s^{(k)}}^T \sigma\left(\widehat{\mathbf{W}}^{(k)} \widehat{\mathbf{h}}_i^{(k)}\right) + {\widehat{\mathbf{v}}_r^{(k)}}^T \sigma\left(\widehat{\mathbf{W}}^{(k)} \widehat{\mathbf{h}}_j^{(k)}\right)\right) eq. 5
\widehat{\mathbf{h}}_i^{(k-1)} = \sum_{j \in \mathcal{N}_i} \widehat{\alpha}_{ij}^{(k)}\, \sigma\left(\widehat{\mathbf{W}}^{(k)} \widehat{\mathbf{h}}_j^{(k)}\right) eq. 6
\sum_{i=1}^{N} \left\lVert \mathbf{x}_i - \widehat{\mathbf{x}}_i \right\rVert_2 eq. 7
-\sum_{i=1}^{N} \sum_{j \in \mathcal{N}_i} \log\left(\frac{1}{1 + \exp(-\mathbf{h}_i^T \mathbf{h}_j)}\right) eq. 8
\text{Loss} = \sum_{i=1}^{N} \left\lVert \mathbf{x}_i - \widehat{\mathbf{x}}_i \right\rVert_2 - \lambda \sum_{j \in \mathcal{N}_i} \log\left(\frac{1}{1 + \exp(-\mathbf{h}_i^T \mathbf{h}_j)}\right) eq. 9
\mathbf{C}^{(k)} = \text{Softmax}\left(\text{Sigmoid}\left(\mathbf{M}_s^{(k)} + \mathbf{M}_r^{(k)}\right)\right) eq. 10
\mathbf{M}_s^{(k)} = \mathbf{A} \odot \left({\mathbf{v}_s^{(k)}}^T \sigma\left(\mathbf{W}^{(k)} \mathbf{H}^{(k-1)}\right)\right) \qquad \mathbf{M}_r^{(k)} = \mathbf{A} \odot \left({\mathbf{v}_r^{(k)}}^T \sigma\left(\mathbf{W}^{(k)} \mathbf{H}^{(k-1)}\right)\right)^T eq. 11–12
\mathbf{H}^{(k)} = \sigma\left(\mathbf{W}^{(k)} \mathbf{H}^{(k-1)}\right) \mathbf{C}^{(k)} eq. 13
\widehat{\mathbf{C}}^{(k)} = \text{Softmax}\left(\text{Sigmoid}\left(\widehat{\mathbf{M}}_s^{(k)} + \widehat{\mathbf{M}}_r^{(k)}\right)\right) eq. 14
\widehat{\mathbf{M}}_s^{(k)} = \mathbf{A} \odot \left({\widehat{\mathbf{v}}_s^{(k)}}^T \sigma\left(\widehat{\mathbf{W}}^{(k)} \widehat{\mathbf{H}}^{(k)}\right)\right) \qquad \widehat{\mathbf{M}}_r^{(k)} = \mathbf{A} \odot \left({\widehat{\mathbf{v}}_r^{(k)}}^T \sigma\left(\widehat{\mathbf{W}}^{(k)} \widehat{\mathbf{H}}^{(k)}\right)\right)^T eq. 15–16
\widehat{\mathbf{H}}^{(k-1)} = \sigma\left(\widehat{\mathbf{W}}^{(k)} \widehat{\mathbf{H}}^{(k)}\right) \widehat{\mathbf{C}}^{(k)} eq. 17
O(NFD + ED) eq. 18
Hyperparameters
| Name | Value | Where | Notes |
|---|---|---|---|
| optimiser | Adam | §5.3 | betas and epsilon not stated; code uses TensorFlow's Adam defaults trainer.py |
| learning rate (initial) | 1e-4 | §5.3 | code default 0.0001 transductive_classifier.py; no schedule in code |
| layers L | 2 | §5.3 fig. 1 | code: hidden dims [F, 512, 512] give two layers transductive_classifier.py |
| width d(1) = d(2) | 512 | §5.3 | same in the official code's default transductive_classifier.py |
| activation σ | identity | §5.3 | paper says identity works better than other choices; the code has no activation gate.py |
| epochs | 100 (Cora, Citeseer); 500 (Pubmed) | §5.3 | code default 200 for all datasets, no per-dataset setting transductive_classifier.py |
| λ (structure weight) | 0.5 (Cora, Pubmed); 20 (Citeseer) | §5.3 | code default 1 transductive_classifier.py; no other setting in code |
| weight tying | Ŵ = Wᵀ, Ĉ = C | §5.3 | code always ties; decoder reuses encoder attention gate.py |
| self-loops in A | included | §3 | code adds the identity and binarises A process.py |
| gradient clipping | not stated in paper | §5.3 absent | code clips the global norm to 5.0 trainer.py; default in transductive_classifier.py |
| dropout (GATE) | not stated | §5.3 absent | code has a dropout flag, default 0.0, which is never used transductive_classifier.py |
| dropout (baselines) | sweep {0, 0.2, 0.5} | §5.3 | supervised baselines only |
| baseline learning-rate sweep | {1e-3…1e-7} unsupervised; {1e-2, 1e-3, 1e-4} supervised | §5.3 | GATE's own learning rate is 1e-4 |
| baseline epoch sweep | four values, from 50 to 300, for VGAE and GAE | §5.3 | |
| baseline width | 512 | §5.3 | |
| train/val/test split | 20 labelled nodes per class; 500 validation; 1000 test | §5.1 tab. 2 | Planetoid split §5.1 |
| inductive training graph | test nodes and their features and edges unseen | §5.1 | code also removes validation nodes from training inductive_process.py |
| initialisation | not stated | — | code uses tf.get_variable with no initialiser gate.py |
| batch | not stated | — | code is full-batch trainer.py |
| feature normalisation | not stated | — | code uses raw features; normalisers exist but are not called process.py |
| classifier | logistic regression | §5.4 | regularisation not stated; code uses library defaults classifier.py |
| runs | 100 training runs | §5.4 | code runs once with seed 0 classifier.py |
| hardware | not stated | — | |
| framework | TensorFlow | §5.3 | code: TensorFlow 1.13.1, Python 2, NumPy 1.16.2, scikit-learn 0.21.2 requirements.txt |
Results
Author-reported. Setup: node classification on Cora, Citeseer and Pubmed tab. 2. Transductive: the whole graph and all node features are visible in training §5.1. Inductive: test nodes, their features and edges are unseen during training §5.1. A logistic regression is trained on the embeddings §5.4. Accuracy in percent, mean ± standard deviation over 100 training runs §5.4. The paper states that some baseline numbers are copied from earlier papers rather than rerun §5.4. Table 3 is reproduced in full below; "—" means the paper gives no value.
Transductive tab. 3a:
| Inputs used | Model | Cora | Citeseer | Pubmed |
|---|---|---|---|---|
| X | Raw features | 47.9 ± 0.4 | 49.4 ± 0.2 | 69.1 ± 0.3 |
| A | DeepWalk | 67.2 | 43.2 | 65.3 |
| A, Y | LP | 68.0 | 45.3 | 63.0 |
| X, A | DeepWalk + features | 70.7 ± 0.6 | 51.4 ± 0.5 | 74.3 ± 0.9 |
| X, A | VGAE | 72.4 ± 0.2 | 55.7 ± 0.2 | 71.6 ± 0.4 |
| X, A | GAE | 81.8 ± 0.1 | 69.2 ± 0.9 | 78.2 ± 0.1 |
| X, A | DGI | 82.3 ± 0.6 | 71.8 ± 0.7 | 76.8 ± 0.6 |
| X, A | GATE (ours) | 83.2 ± 0.6 | 71.8 ± 0.8 | 80.9 ± 0.3 |
| X, A, Y | Planetoid | 75.7 | 62.9 | 75.7 |
| X, A, Y | Chebyshev | 81.2 | 69.8 | 74.4 |
| X, A, Y | Monet | 81.7 ± 0.5 | — | 78.0 ± 0.3 |
| X, A, Y | GCN | 81.5 | 70.3 | 79.0 |
| X, A, Y | StoGCN | 82.0 ± 0.8 | 70.9 ± 0.2 | 79.0 ± 0.4 |
| X, A, Y | GAT | 83.0 ± 0.7 | 72.5 ± 0.7 | 79.0 ± 0.3 |
Inductive tab. 3b:
| Inputs used | Model | Cora | Citeseer | Pubmed |
|---|---|---|---|---|
| X, A | GraphSAGE-LSTM | 50.1 ± 0.2 | 40.3 ± 0.2 | 77.1 ± 0.1 |
| X, A | GraphSAGE-pool | 57.5 ± 0.2 | 45.9 ± 0.2 | 79.9 ± 0.1 |
| X, A | VGAE | 58.4 ± 0.4 | 55.4 ± 0.2 | 71.1 ± 0.2 |
| X, A | GraphSAGE-mean | 67.0 ± 0.2 | 52.8 ± 0.1 | 79.3 ± 0.1 |
| X, A | GraphSAGE-GCN | 74.3 ± 0.1 | 54.5 ± 0.1 | 77.5 ± 0.1 |
| X, A | GAE | 80.5 ± 0.1 | 69.1 ± 0.9 | 78.1 ± 0.2 |
| X, A | GATE (ours) | 82.5 ± 0.5 | 71.5 ± 0.7 | 80.8 ± 0.3 |
| X, A, Y | Planetoid | 61.2 | 64.7 | 77.2 |
| X, A, Y | GAT | 76.4 ± 0.2 | 66.4 ± 0.2 | 77.7 ± 0.03 |
The inductive table has no DGI row tab. 3b. Both tables are the paper's as typeset; the paper's text on the comparison is in §5.4.
Headline claims, as the paper states them §5.4:
- Transductive: GATE beats all supervised and unsupervised baselines on Cora and Pubmed. On Pubmed it is 1.9 points above GAT and 2.7 above the best unsupervised baseline. On Citeseer it equals DGI and is below GAT tab. 3a.
- Inductive: GATE beats all baselines on all three datasets, by 2.4, 2.0 and 0.9 points on Citeseer, Cora and Pubmed over the best baseline in each case tab. 3b.
- The gap between GATE's transductive and inductive accuracy is 0.1, 0.3 and 0.7 points on Pubmed, Citeseer and Cora tab. 3a, tab. 3b.
Ablation fig. 2, values read from the plotted data in the source. Transductive, Cora / Pubmed / Citeseer: GATE 83.2 / 80.9 / 71.8; GATE/F (no feature reconstruction) 82.1 / 77.4 / 69.5; GATE/S (no structure reconstruction) 81.6 / 74.9 / 71.4; GATE/A (no attention, uniform weights) 76.8 / 74.4 / 67.5. Inductive: GATE 82.5 / 80.8 / 71.5; GATE/F 81.7 / 74.6 / 68.8; GATE/S 81.3 / 74.8 / 71.4; GATE/A 75.8 / 74.2 / 66.9. The paper's reading: attention contributes most, and GATE/S beats GATE/F on Citeseer §5.5. The t-SNE plots of Cora nodes and attention-weighted edges are in fig. 3 and are described in §5.6.
Paper vs code
Official code: https://github.com/amin-salehi/GATE at commit 30af474257a0461580546e0980b2e2c2f3ff3e96. The paper's LaTeX source contains this link only as a commented-out line §5.3. The owner name matches the first author, Amin Salehi, and the repository's description matches the title. The repository has no LICENSE file, so no code is quoted here; code is cited by path and line only. Code license: none.
Differences, each with both citations:
- Feature loss: the paper sums per-node L2 error norms eq. 7. The code takes one square root of the sum of all squared errors, a global Frobenius norm gate.py.
- Structure loss: the paper's form eq. 8 matches the code, which sums −log σ(h_s · h_r) over every row/column pair of the adjacency with self-loops gate.py process.py.
- Attention scores: the paper's matrix form puts v_s on the neighbour index j in M_s eq. 11. Eq. 1 puts v_s on node i, and the code does too:
v[0](v_s) is applied to row i andv[1]to column j gate.py gate.py. Follow eq. 1, not eq. 11. - Aggregation orientation: the paper's matrix form H^(k) = σ(WH)C eq. 13 with C_ij = α_ij gives column i of H as a sum over the entries C_ji, which is the transpose of eq. 3. The code treats nodes as rows and computes C·H, which matches eq. 3 directly gate.py.
- Decoder: the code implements only the tied variant. It applies the transposed encoder weights and reuses the encoder attention gate.py. The untied decoder of §4.2 eq. 4–6, 14–17 is not implemented.
- Epochs: one epoch is one full-batch Adam step on the whole graph trainer.py. The paper gives epoch counts without defining the step §5.3.
- Epoch count and λ defaults: the code defaults are 200 epochs and λ = 1 transductive_classifier.py. The paper's per-dataset values (100 / 500, and 0.5 / 20) are in §5.3 and are not set anywhere in the code or README.
- Gradient clipping: the code clips the global norm to 5.0 trainer.py. The paper does not mention clipping §5.3.
- Dropout: the code has a dropout flag, but nothing reads it transductive_classifier.py gate.py. The paper does not use dropout for GATE §5.3.
- Input features: the code uses raw features. Its normalisation helpers are defined but not called process.py process.py. The paper does not say which features it uses §5.1.
- Evaluation: the code trains once with seed 0 and prints micro- and macro-F1 classifier.py transductive_classifier.py. The paper reports accuracy over 100 runs §5.4. For single-label data, micro-F1 equals accuracy.
- Inductive inference: the code trains on a graph with validation and test nodes removed inductive_process.py. It then embeds all nodes on the full graph, including the test nodes' features and edges inductive_classifier.py. The paper says test nodes are unseen during training §5.1, which holds, but the inference step uses them.
- Self-loops and binarisation: the code adds self-loops and sets all nonzero adjacency entries to 1 process.py, as §3 assumes §3.
- Activation: the code has no activation function, which matches σ = identity gate.py §5.3.
- Language and dependencies: Python 2 code (
xrangeandprintstatements) pinned to TensorFlow 1.13.1, NumPy 1.16.2 and scikit-learn 0.21.2 gate.py transductive_classifier.py requirements.txt. - No run instructions: the README gives the paper and citation only, with no command or per-dataset settings ReadMe.md.
Not verified here: the table numbers were not reproduced, because the code needs Python 2 and TensorFlow 1.13.1, which were not set up. The inductive path also builds the feature matrix and the adjacency in different node orders. The features come from a set, and the adjacency comes from a networkx subgraph inductive_process.py. Whether row i of each matches node i was not checked.
Pitfalls
- Matrix form is inconsistent with eq. 1–3. Eq. 11 puts v_s on the neighbour index, eq. 13 uses the transpose of the attention orientation, and the code follows eq. 1–3. Implement from eq. 1–3 and the code's convention, not from the matrix equations. Because Sigmoid is applied after the sum, it matters which node the v_s term depends on: the term does not cancel in the softmax.
- Softmax is not defined in the paper's matrix form. Eq. 10 and 14 use Softmax, but its definition was commented out in the source §4.4. The code applies a softmax over each row's nonzero entries gate.py. That is a row softmax over N_i, as eq. 2 says, provided A has self-loops process.py.
- Eq. 9 has an ambiguous sum. As typeset, the structure sum runs over j in N_i with no outer sum over i. Read it as eq. 8 times λ, and expect the sign convention to match eq. 8.
- Feature loss differs between paper and code. Per-node norms eq. 7 give a different scale and gradient than one global norm gate.py. The paper does not say which form its λ values were used with, so expect λ to need checking if you implement eq. 7 literally.
- The structure term has only positive pairs. Each edge contributes −log σ of a dot product, and there are no negative samples gate.py eq. 8. Nothing in the paper or code bounds the embedding norm, so the term can be lowered by growing dot products. The paper does not discuss this. Watch for norm growth at λ = 20 (Citeseer) §5.3.
- Loss scale depends on graph size. The structure term sums over all edges and self-loops, and the feature term is a global norm gate.py. λ's effect therefore varies with N and E. The paper gives no normalisation.
- Gradient clipping is hidden. Without the 5.0 global-norm clip trainer.py, training may differ, and the paper does not report it.
- Feature preprocessing is hidden. Features are raw, unnormalised values process.py. The paper does not state this.
- "Epoch" means one full-batch step. On Pubmed, 500 epochs is 500 gradient steps over the whole graph trainer.py.
- Feature memory is not counted by the space claim. The paper says space is linear in N and E §4.5. The code feeds a dense N × F feature placeholder trainer.py process.py, so feature memory is O(NF) on top of that.
- Inference uses the full graph. The inductive claim holds at training time only. At inference, the code embeds test nodes with their real edges inductive_classifier.py. Do not implement inductive inference as if only training-time nodes were available.
- Single seed in code. The code gives one number per dataset, not the mean and deviation of 100 runs classifier.py. Compare to the paper only after multi-seed runs.
- Ablation variants are not in the code. GATE/A, GATE/S and GATE/F have no switches in the repository gate.py, so fig. 2 cannot be reproduced from it.
- Degree figures disagree. §5.4 gives average degree 2.25 for Pubmed §5.4. §5.5 gives 2.5 for Pubmed §5.5. The edge and node counts in tab. 2 give about 2.25, so 2.5 looks like a typo.
- Tensor framework limit. The paper says TensorFlow's lack of rank-3 matrix products limits batching, so training uses the whole graph at once §6.
- No DGI in the inductive table. The inductive table omits DGI (its line is commented out tab. 3b). Do not infer a DGI inductive result from the paper.
Lineage
- Builds on: graph attention networks (Veličković et al., 2018), whose attention form the encoder reuses §4.1. The paper says the encoder is reminiscent of GAT but uses unsupervised training §4.2. No card exists yet.
- Compared against: DeepWalk, enhanced DeepWalk, label propagation, Planetoid, Chebyshev, GCN, GAE, VGAE, GraphSAGE (four variants), Monet, DGI, StoGCN and GAT §5.2.
- Superseded by: not stated in the paper.
Source and license
Adapted from “Graph Attention Auto-Encoders” by Amin Salehi, Hasan Davulcu (arXiv:1905.10715), used under CC0 1.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/amin-salehi/GATE at commit 30af474257a0 (none (no LICENSE file in the repository)). Any quoted code is shown under that license.