wikillm

Paper · 2021-08-12 · cs.CV, cs.MM

Attention-driven Graph Clustering Network

AGCN clusters nodes by jointly training an autoencoder and a GCN. Attention gates fuse the two streams in each layer (AGCN-H) and weight all layers' features (AGCN-S); a KL self-training loss sharpens the assignments. Author-reported ACC: USPS 80.98, HHAR 88.11, Reuters 79.30, ACM 90.59, CiteSeer 68.79, DBLP 73.26.

Deep clusteringGraph neural networkAttention mechanismMulti-scale feature fusiondeep-clusteringgraph-convolutionattention-fusionmulti-scale-features

arXivPDFCodeMarkdown16 min read

AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Clustering means sorting items into groups without labels. Here the items are nodes in a network, such as papers linked by citations or authors linked by co-authorship. Each node also has a list of features, such as the words in a paper. A good clustering has to use both the features and the links.

The method runs two kinds of processing side by side. One is an autoencoder, which compresses each node's features into a short code. The other is a graph network, which mixes each node's features with those of its neighbours. At every stage, the method decides how much weight to give the feature-based view and how much the link-based view. It also decides how much weight to give each stage of processing when it makes the final grouping.

The training pushes the grouping to become more confident, and the compressed codes are trained to agree with it. The authors report that it beats most of the earlier methods they compare with, across six benchmark datasets. These numbers are their own, measured under their own setup.

TL;DR

  • AGCN is a deep clustering network that trains an autoencoder and a graph convolutional network (GCN) together, without labels §3.
  • AGCN-H fuses the GCN and autoencoder features of each layer with learned attention weights; AGCN-S weights and concatenates the features of all layers before the final prediction §3.1 §3.2.
  • Training minimises a reconstruction loss plus two KL terms that pull the soft assignment and the autoencoder distribution toward a sharpened target distribution §3.3 eq. 13.
  • Author-reported accuracy beats the eight compared methods on most datasets, including ACC 80.98 on USPS and 88.11 on HHAR tab. 3.
  • The official code is public and has no licence file. It differs from the paper in the loss weighting, graph normalisation, batch handling, and the way it picks the reported epoch. Read "Paper vs code" before implementing.

Problem

Existing deep clustering methods learn from node features, or from a GCN over the graph, but not both in a balanced way. The authors say that prior works treat the graph feature and the attribute feature as equally important in every case, and use only the deepest layer's features §1 §2. They also argue that the fusion between graph and attribute features is too weak §2.

The method assumes an undirected graph over the samples. For non-graph data such as images or records, the graph is built as a k-nearest-neighbour graph, and the value k' is not given in the text §4.1.

Method

AGCN has an autoencoder (AE), a stack of four GCN layers, two attention fusion modules, a prediction layer and a self-training loss fig. 1. Notation: X is the N×d input, A the adjacency matrix, I the identity, D the degree matrix. Layer count l = 4, and the widths are 500, 500, 2000 and 10 for both the AE encoder and the GCN stack §4.3 alg. 1.

  1. Autoencoder. The encoder and decoder are fully connected stacks. The latent code H is the last encoder layer. The reconstruction loss is the squared Frobenius norm of X minus its reconstruction §3.1 eq. 1.
  2. GCN stack. Z_1 is a GCN over the raw input (Z_0 = X). Each later layer applies the normalised adjacency to the previous fused feature and a weight matrix, followed by LeakyReLU §3.1 eq. 4.
  3. AGCN-H (heterogeneity-wise fusion). The GCN feature Z_i and AE feature H_i of a layer are concatenated. The paper does not give the range of layers; the code fuses layers 1 to 3 only AGCN.py. A weight matrix maps the concatenation to two scores, which pass through LeakyReLU, a softmax over the two, and an ℓ2 normalisation per node. The fused feature is the weighted sum of the two, one weight per node and branch §3.1 eq. 2 eq. 3 fig. 2a.
  4. AGCN-S (scale-wise fusion). The five features Z_1, Z_2, Z_3, Z_4 and the latent H_l are concatenated (with H_l taken as Z_{l+1}). A weight matrix maps this to five scores, with the same LeakyReLU, softmax and ℓ2 steps. Each block is scaled by its node-wise weight and the blocks are concatenated §3.2 eq. 5 eq. 6 eq. 7 fig. 2b.
  5. Prediction. The concatenated features pass through one more GCN layer with no activation, then a softmax gives the soft assignment Z. The hard label is the argmax of Z §3.2 eq. 8 eq. 9.
  6. Target and losses. A Student's t kernel with α = 1 gives the soft assignment Q of the latent H to k cluster centres. A sharpened target P is derived from Q §3.3 eq. 10 eq. 11. The KL loss aligns Z and Q with P eq. 12. The total loss adds the reconstruction term eq. 13.

The paper's equations, as numbered in the compiled paper:

\mathcal{L}_{R} = \left\| \mathbf{X} - \hat{\mathbf{X}} \right\|^2_F, \quad \mathbf{H}_i = \phi(\mathbf{W}^e_i \mathbf{H}_{i-1} + \mathbf{b}^e_i), \quad \hat{\mathbf{H}}_i = \phi(\mathbf{W}^d_i \hat{\mathbf{H}}_{i-1} + \mathbf{b}^d_i), \; i = 1..l eq. 1

\mathbf{M}_i = \ell_2\left(\mathrm{softmax}\left(\mathrm{LeakyReLU}\left([\mathbf{Z}_i \| \mathbf{H}_i]\, \mathbf{W}^a_i\right)\right)\right) eq. 2

\mathbf{Z}'_i = (\mathbf{m}_{i,1} \mathbf{1}_i) \odot \mathbf{Z}_i + (\mathbf{m}_{i,2} \mathbf{1}_i) \odot \mathbf{H}_i eq. 3

\mathbf{Z}_{i+1} = \mathrm{LeakyReLU}\left(\mathbf{D}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I})\mathbf{D}^{-\frac{1}{2}} \mathbf{Z}'_i \mathbf{W}_i\right) eq. 4

\mathbf{Z}' = [\mathbf{Z}_1 \| \cdots \| \mathbf{Z}_l \| \mathbf{Z}_{l+1}], \quad \mathbf{Z}_{l+1} = \mathbf{H}_l eq. 5

\mathbf{U} = \ell_2\left(\mathrm{softmax}\left(\mathrm{LeakyReLU}\left([\mathbf{Z}_1 \| \cdots \| \mathbf{Z}_{l+1}]\, \mathbf{W}^s\right)\right)\right) eq. 6

\mathbf{Z}' = [(\mathbf{u}_1 \mathbf{1}_1) \odot \mathbf{Z}_1 \| \cdots \| (\mathbf{u}_{l+1} \mathbf{1}_{l+1}) \odot \mathbf{Z}_{l+1}] eq. 7

\mathbf{Z} = \mathrm{softmax}\left(\mathbf{D}^{-\frac{1}{2}}(\mathbf{A}+\mathbf{I})\mathbf{D}^{-\frac{1}{2}} \mathbf{Z}' \mathbf{W}\right), \quad \textstyle\sum_j z_{ij} = 1 eq. 8

y_i = \arg\max_j z_{i,j} eq. 9

q_{ij} = \frac{(1+\|\mathbf{h}_i-\boldsymbol{\mu}_j\|^2/\alpha)^{-\frac{\alpha+1}{2}}}{\sum_{j'} (1+\|\mathbf{h}_i-\boldsymbol{\mu}_{j'}\|^2/\alpha)^{-\frac{\alpha+1}{2}}}, \quad \alpha = 1 eq. 10

p_{ij} = \frac{q_{ij}^2 / \sum_i q_{ij}}{\sum_{j'} q_{ij'}^2 / \sum_i q_{ij'}} eq. 11

\mathcal{L}_{KL} = \lambda_1\, KL(\mathbf{P}, \mathbf{Z}) + \lambda_2\, KL(\mathbf{P}, \mathbf{H}) eq. 12

\mathcal{L} = \mathcal{L}_{R} + \mathcal{L}_{KL} eq. 13

The training procedure is Algorithm 1 alg. 1. Written out in our own pseudocode, one iteration does the following. The code is in the paper's form, not the official code's (see "Paper vs code").

Pseudocode
# AGCN forward pass  [§3.1] [§3.2] [eq. 1]-[eq. 8]
# X: (N, d); A_norm: sparse (N, N), D^-1/2 (A+I) D^-1/2  [eq. 4]
# Layers l = 4; AE widths 500, 500, 2000, 10  [§4.3]

def forward(X, A_norm):
    # Autoencoder pass: reconstruction, per-layer AE features, latent  [eq. 1]
    X_hat, H_1, H_2, H_3, H = autoencoder(X)
    # Z_1 is a GCN over the raw input  [§3.1]
    Z_1 = gcn_layer_1(X, A_norm)
    # AGCN-H fuses GCN and AE features, layers 1 to 3 in the code  [eq. 2] [eq. 3]
    Z_fused_1 = fuse_heterogeneity(Z_1, H_1)
    # Next GCN layer uses the fused feature  [eq. 4]
    Z_2 = gcn_layer_2(Z_fused_1, A_norm)
    Z_fused_2 = fuse_heterogeneity(Z_2, H_2)
    Z_3 = gcn_layer_3(Z_fused_2, A_norm)
    Z_fused_3 = fuse_heterogeneity(Z_3, H_3)
    Z_4 = gcn_layer_4(Z_fused_3, A_norm)
    # AGCN-S: attention over the five scale features  [eq. 5] [eq. 6]
    scales = [Z_1, Z_2, Z_3, Z_4, H]
    weights = attention_l2(concat(scales) @ W_s)     # (N, 5)
    # Weight each scale and concatenate  [eq. 7]
    scaled = concat([weights[:, j:j+1] * scales[j]
                     for j in range(5)])              # (N, 3020)
    # Prediction layer with smoothing, then softmax  [eq. 8]
    Z_soft = softmax(A_norm @ (scaled @ W))          # (N, k)
    return X_hat, Z_soft, H


def fuse_heterogeneity(gcn_feature, ae_feature):
    # Scores from the concatenation, LeakyReLU, softmax, l2  [eq. 2]
    weights = attention_l2(concat(gcn_feature, ae_feature) @ W_a)  # (N, 2)
    # Weighted sum, one weight per node and branch  [eq. 3]
    return (weights[:, 0:1] * gcn_feature
            + weights[:, 1:2] * ae_feature)
Pseudocode
# One training iteration  [alg. 1] [eq. 10]-[eq. 13]
# k: number of clusters (6 to 10 in the paper's datasets)  [tab. 2]

def training_step(X, A_norm, k):
    # Forward pass gives reconstruction, soft assignment Z, latent H  [alg. 1]
    X_hat, Z_soft, H = forward(X, A_norm)
    # Cluster centres mu from K-means on H, redone each iteration  [alg. 1]
    centres = kmeans(H.detach(), k)
    # Student's t soft assignment of H to the centres, alpha = 1  [eq. 10]
    Q = student_t_assignment(H, centres, alpha=1.0)
    # Auxiliary target, held fixed for this step  [eq. 11]
    P = target_distribution(Q.detach())
    # Reconstruction term, then the two KL terms  [eq. 1] [eq. 12]
    loss_rec = squared_frobenius(X - X_hat)
    loss_kl = (lambda_1 * kl(P, Z_soft)
               + lambda_2 * kl(P, Q))
    # Total loss, followed by one backward pass and one update  [eq. 13]
    return loss_rec + loss_kl


def target_distribution(Q):
    # Square and divide by the soft cluster size f_j = sum_i q_ij  [eq. 11]
    weight = Q ** 2 / Q.sum(axis=0)                  # (N, k)
    # Normalise each row to sum to one  [eq. 11]
    return weight / weight.sum(axis=1, keepdims=True)

Per iteration the authors also count the cost. The AE costs O(n Σ d_{i-1} d_i); the sparse GCN costs O(|E| Σ d_{i-1} d_i); the Q term costs O(nk + n log n); the proposed modules add O(Σ d_i) plus O((Σ d_i)(l+1)) §3.4. The paper gives no runtime measurements.

Hyperparameters

NameValueWhereNotes
AE encoder widths500-500-2000-10§4.3 AGCN.pyLatent width 10
AE decoder widthsnot stated in paperAGCN.pyCode uses 2000-500-500
GCN widths500-500-2000-10§4.3 AGCN.pySame as AE encoder
Layers l4alg. 1
AE pre-training epochs30§4.3Pre-training code not in repository
AE pre-training learning rate0.001§4.3
Main training iterations200§4.3 alg. 1 AGCN.py
Learning rate, USPS, HHAR, ACM, DBLP0.001§4.3 AGCN.pyCode default
Learning rate, Reuters, CiteSeer0.0001§4.3 AGCN.py AGCN.py
λ1, λ2, USPS1000, 1000§4.3 AGCN.pyCode hard-codes these for every dataset
λ1, λ2, HHAR1, 0.1§4.3Not in released code
λ1, λ2, Reuters10, 10§4.3Not in released code
λ1, λ2, graph datasets0.1, 0.01§4.3Not in released code
Batch size256§4.3Code trains full batch; see "Paper vs code"
LeakyReLU negative slope, attention0.2§3.1Code uses PyTorch default in attention
LeakyReLU negative slope, GCNnot stated in paperGNN.pyCode uses 0.2
Student's t degrees α1eq. 10 AGCN.py
Neighbours k' (non-graph data)not stated in paper§4.1 AGCN.py AGCN.pyCode: USPS 3, HHAR 5, Reuters 3 (default)
Latent dimension10AGCN.pyMatches last width §4.3
Runs per dataset10§4.3 AGCN.pyMean ± standard deviation
Hardware and frameworkPyTorch, GeForce RTX 2080 Ti§4.3Code README: Python 3.6.12, PyTorch 1.7.1 README.md
Optimisernot stated in paperAGCN.pyCode uses Adam, default settings
K-means for centresnot stated in paperAGCN.pyCode uses n_init=20
Activation φ in AE"such as Tanh, ReLU"§3.1Code uses ReLU in hidden layers only
Weight init, GCNnot stated in paperGNN.pyCode: Xavier uniform

Datasets, as given in the paper tab. 2:

DatasetTypeSamplesClassesDimension
USPSImage929810256
HHARRecord102996561
ReutersText1000042000
ACMGraph302531870
CiteSeerGraph332763703
DBLPGraph40574334

Metrics are ACC, NMI, ARI and macro F1 §4.3. The paper calls ARI the "Average Rand Index" §4.3.

Results

Author-reported. Setup: six datasets, 10 runs, mean ± std §4.3. The paper says ARGA was run with the parameter settings of its original paper, and the results of the other baselines are cited from the SDCN paper §4.3.

Table 3, clustering performance (mean ± std), as printed in the paper tab. 3. The paper marks the best value in bold and the second-best underlined; the markings are not reproduced here.

DatasetMetricAEDECIDECGAEVGAEDAEGCARGASDCNAGCN
USPSACC71.04±0.0373.31±0.1776.22±0.1263.10±0.3356.19±0.7273.55±0.4066.80±0.7078.08±0.1980.98±0.28
USPSNMI67.53±0.0370.58±0.2575.56±0.0660.69±0.5851.08±0.3771.12±0.2461.60±0.3079.51±0.2779.64±0.32
USPSARI58.83±0.0563.70±0.2767.86±0.1250.30±0.5540.96±0.5963.33±0.3451.10±0.6071.84±0.2473.61±0.43
USPSF169.74±0.0371.82±0.2174.63±0.1061.84±0.4353.63±1.0572.45±0.4966.10±1.2076.98±0.1877.61±0.38
HHARACC68.69±0.3169.39±0.2571.05±0.3662.33±1.0171.30±0.3676.51±2.1963.30±0.8084.26±0.1788.11±0.43
HHARNMI71.42±0.9772.91±0.3974.19±0.3955.06±1.3962.95±0.3669.10±2.2857.10±1.4079.90±0.0982.44±0.62
HHARARI60.36±0.8861.25±0.5162.83±0.4542.63±1.6351.47±0.7360.38±2.1544.70±1.0072.84±0.0977.07±0.66
HHARF166.36±0.3467.29±0.2968.63±0.3362.64±0.9771.55±0.2976.89±2.1861.10±0.9082.58±0.0888.00±0.53
ReutersACC74.90±0.2173.58±0.1375.43±0.1454.40±0.2760.85±0.2365.50±0.1356.20±0.2077.15±0.2179.30±1.07
ReutersNMI49.69±0.2947.50±0.3450.28±0.1725.92±0.4125.51±0.2230.55±0.2928.70±0.3050.82±0.2157.83±1.01
ReutersARI49.55±0.3748.44±0.1451.26±0.2119.61±0.2226.18±0.3631.12±0.1824.50±0.4055.36±0.3760.55±1.78
ReutersF160.96±0.2264.25±0.2263.21±0.1243.53±0.4257.14±0.1761.82±0.1351.10±0.2065.48±0.0866.16±0.64
ACMACC81.83±0.0884.33±0.7685.12±0.5284.52±1.4484.13±0.2286.94±2.8386.10±1.2090.45±0.1890.59±0.15
ACMNMI49.30±0.1654.54±1.5156.61±1.1655.38±1.9253.20±0.5256.18±4.1555.70±1.4068.31±0.2568.38±0.45
ACMARI54.64±0.1660.64±1.8762.16±1.5059.46±3.1057.72±0.6759.35±3.8962.90±2.1073.91±0.4074.20±0.38
ACMF182.01±0.0884.51±0.7485.11±0.4884.65±1.3384.17±0.2387.07±2.7986.10±1.2090.42±0.1990.58±0.17
CiteSeerACC57.08±0.1355.89±0.2060.49±1.4261.35±0.8060.97±0.3664.54±1.3956.90±0.7065.96±0.3168.79±0.23
CiteSeerNMI27.64±0.0828.34±0.3027.17±2.4034.63±0.6532.69±0.2736.41±0.8634.50±0.8038.71±0.3241.54±0.30
CiteSeerARI29.31±0.1428.12±0.3625.70±2.6533.55±1.1833.13±0.5337.78±1.2433.40±1.5040.17±0.4343.79±0.31
CiteSeerF153.80±0.1152.62±0.1761.62±1.3957.36±0.8257.70±0.4962.20±1.3254.80±0.8063.62±0.2462.37±0.21
DBLPACC51.43±0.3558.16±0.5660.31±0.6261.21±1.2258.59±0.0662.05±0.4861.60±1.0068.05±1.8173.26±0.37
DBLPNMI25.40±0.1629.51±0.2831.17±0.5030.80±0.9126.92±0.0632.49±0.4526.80±1.0039.50±1.3439.68±0.42
DBLPARI12.21±0.4323.92±0.3925.37±0.6022.02±1.4017.92±0.0721.03±0.5222.70±0.3039.15±2.0142.49±0.31
DBLPF152.53±0.3659.38±0.5161.33±0.5661.41±2.2358.69±0.0761.75±0.6761.80±0.9067.71±1.5172.80±0.56

Headline claims in the text, as stated. The improvements on HHAR and DBLP are averages over the second-best method, and the text gives them as percentages (they are absolute differences, since 88.11 − 84.26 = 3.85) §4.4:

  • HHAR: +3.85 ACC, +2.54 NMI, +4.23 ARI, +5.42 F1 over the second-best method §4.4.
  • DBLP: +5.21 ACC, +0.18 NMI, +3.34 ARI, +5.09 F1 over the second-best method §4.4.
  • The paper says the method is "the best results in all six datasets" §4.4. Table 3 does not support this for CiteSeer F1, where SDCN is 63.62 and AGCN is 62.37 tab. 3.

Ablation, Table 4 tab. 4. The first row of each block has no AGCN component; its numbers equal the SDCN column of Table 3 on all six datasets. The paper does not say this explicitly. A tick marks a component that is on. The column "Scale uniform" is the paper's AGCN-S[S], the scale fusion with all weights set to 1. "Scale attention" is the paper's AGCN-S[A], the attention-based scale fusion. "Heterogeneity fusion" is AGCN-H.

DatasetScale attentionScale uniformHeterogeneity fusionACCNMIARIF1
USPS78.08±0.1979.51±0.2771.84±0.2476.98±0.18
USPS✓79.43±1.1079.13±0.5172.00±1.1676.71±0.78
USPS✓✓80.20±0.7579.38±0.2872.79±0.7477.10±0.50
USPS✓✓✓80.98±0.2879.64±0.3273.61±0.4377.61±0.38
HHAR84.26±0.1779.90±0.0972.84±0.0982.58±0.08
HHAR✓84.39±1.6180.63±0.6573.40±0.6482.67±2.43
HHAR✓✓82.85±1.6080.24±0.4372.41±0.6480.32±2.43
HHAR✓✓✓88.11±0.4382.44±0.6277.07±0.6688.00±0.53
Reuters77.15±0.2150.82±0.2155.36±0.3765.48±0.08
Reuters✓77.81±0.8953.94±1.0856.83±1.5365.10±0.65
Reuters✓✓78.15±0.6753.90±1.3156.53±1.9264.84±0.58
Reuters✓✓✓79.30±1.0757.83±1.0160.55±1.7866.16±0.64
ACM90.45±0.1868.31±0.2573.91±0.4090.42±0.19
ACM✓90.47±0.2468.42±0.6173.95±0.6090.48±0.26
ACM✓✓90.57±0.1168.43±0.4274.16±0.3090.56±0.11
ACM✓✓✓90.59±0.1568.38±0.4574.20±0.3890.58±0.17
CiteSeer65.96±0.3138.71±0.3240.17±0.4363.62±0.24
CiteSeer✓66.38±1.7239.07±1.5240.93±1.7860.91±0.81
CiteSeer✓✓68.34±0.3241.10±0.4343.27±0.5362.00±0.35
CiteSeer✓✓✓68.79±0.2341.54±0.3043.79±0.3162.37±0.21
DBLP68.05±1.8139.50±1.3439.15±2.0167.71±1.51
DBLP✓69.65±1.4335.37±1.5837.78±1.8568.69±1.65
DBLP✓✓71.49±0.5237.38±0.6539.91±0.7871.02±0.60
DBLP✓✓✓73.26±0.3739.68±0.4242.49±0.3172.80±0.56

The ablation does not show a gain on every metric. On CiteSeer, adding AGCN-H alone lowers F1 from 63.62 to 60.91, and on HHAR, adding only the multi-scale fusion lowers ACC and F1 tab. 4. The paper's text attributes the HHAR drop to over-smoothing in middle layers §4.5.

Figures, described only (no images are reproduced):

  • fig. 1 shows the full architecture: the autoencoder above, and AGCN-H and AGCN-S below. It also says the network is trained by minimising the KL divergence between H and Z fig. 1.
  • fig. 2 gives the two modules: (a) AGCN-H, with the fused feature passed to the next GCN layer, and (b) AGCN-S, with an attention step followed by a weighted concatenation fig. 2.
  • fig. 3 compares each single layer's feature and the fused feature on HHAR. The text says fusion is clearly better than any single layer §4.5. The bar values are not given in the text.
  • fig. 4 shows the k' sensitivity on USPS, HHAR and Reuters. The text says the method is not sensitive to k' §4.5. The values are only in the figure.
  • fig. 5 shows t-SNE plots on HHAR for raw data, SDCN and AGCN §4.6.

Paper vs code

The official repository is https://github.com/ZhihaoPENG-CityU/MM21---AGCN, at commit 439de9935eef837c8d770462e54b7005e42edce1. It belongs to the authors: the README gives the first author's CityU address, which matches the paper's affiliation README.md. The repository has no LICENSE file, so no code is quoted verbatim here; all code points are cited by path and line. The paper says code "will be publicly available upon acceptance" §4.3. The paper links no code. The repository was found by a GitHub repository search for "AGCN graph clustering attention" (one hit, owned by the first author).

Differences between paper and code, each with both citations:

  1. Equation 1 activation. Paper applies φ to every encoder and decoder layer, including the latent and the output eq. 1. Code leaves the latent and the reconstruction linear and uses ReLU only in the hidden layers AGCN.py.
  2. Attention slope. Paper sets the LeakyReLU slope in attention to 0.2 §3.1. Code calls F.leaky_relu with no slope, which is PyTorch's default of 0.01 AGCN.py AGCN.py AGCN.py AGCN.py. The GCN layers do use 0.2 GNN.py.
  3. Graph normalisation. Paper normalises with D^-1/2 (A+I) D^-1/2 in eq. 4 and eq. 8. Code row-normalises with D^-1 (A+I), which is a different matrix utils.py utils.py. The symmetrisation step keeps an edge if either direction exists utils.py.
  4. Which λ goes with which KL term. Paper gives λ1 to KL(P, Z) and λ2 to KL(P, H) eq. 12. Code gives the first weight, ld1, to the KL with the AE distribution Q (H), and the second, ld2, to the KL with the prediction Z AGCN.py. The two are swapped. This matters for the datasets with unequal λ (HHAR, Reuters, graph data).
  5. Loss normalisation. Paper sums over all nodes and entries eq. 12 eq. 1. Code uses batchmean for both KL terms (divide by N) and mse_loss for reconstruction (divide by N·d) AGCN.py. Up to a common factor N, the code's objective equals the paper's with the reconstruction term divided by d and the two KL weights swapped, as noted in item 4.
  6. Batch. Paper uses batch size 256 §4.3. Code trains on the full data matrix in every step. The DataLoader is imported but never used AGCN.py AGCN.py AGCN.py AGCN.py.
  7. K-means. Algorithm 1 recomputes the centres by K-means in every iteration alg. 1. Code runs K-means once, before the 200 epochs, and uses n_init=20 AGCN.py. Cluster centres are then learned as parameters.
  8. Attention range. Paper defines the AGCN-H fusion for a general layer index i eq. 3. Code applies it at layers 1, 2 and 3 only; the last GCN output is not fused AGCN.py.
  9. λ values in the released script. Paper gives λ per dataset §4.3. The script hard-codes λ1 = λ2 = 1000 for every dataset AGCN.py. The README says to edit them by hand for each dataset README.md. Hence the script as released reproduces only the USPS setting.
  10. Epoch selection. The paper reports results from the final training. The code computes ACC, NMI, ARI and F1 against the ground-truth labels after every epoch, takes the maximum over the 200 epochs for each run, and reports the mean of those maxima over runs AGCN.py AGCN.py. The paper does not say how the reported epoch is chosen. Whether the published numbers were produced this way is not stated.
  11. Metric definitions. Paper says "Average Rand Index" §4.3. Code uses sklearn's adjusted_rand_score evaluation.py. Code uses arithmetic-mean NMI evaluation.py. Paper does not state the NMI averaging. ACC is found by Hungarian matching (Munkres), and F1 is macro F1 after that matching evaluation.py.
  12. Pre-training. Paper trains the AE for 30 epochs at lr 0.001 §4.3. The pre-training script is not in the repository. Code loads data/<name>.pkl AGCN.py AGCN.py. The clone holds only the USPS weights usps.pkl.
  13. Data. Only the USPS files are in the repository. The README points to an external Google Drive folder for the other datasets README.md. The only graph-construction script is for Reuters, using 10 neighbours and an L1-normalised binary bag-of-words cosine calcu_graph.py calcu_graph.py. The neighbour count is encoded only in the file names, such as usps3_graph.txt.
  14. Dimensions and device. The code hard-codes dimensions: 500, 2000 and 10 in the broadcasts, and 3020 = 500+500+2000+10+10 in the fusion layer AGCN.py AGCN.py. It also calls .cuda() throughout, so it does not run on a CPU AGCN.py AGCN.py AGCN.py.
  15. Seeds. The code imports random and never seeds it AGCN.py. There is no fixed seed for the NumPy, PyTorch or K-means random state, so exact reruns are not expected.
  16. Ablation variants. The paper's ablations AGCN-S[S] and AGCN-S[A] are not in the released code, which contains only the full model AGCN.py.

Pitfalls

  • Reported numbers may depend on epoch selection. Each run reports the best of 200 evaluations against the labels AGCN.py. A reimplementation that follows the paper's last-epoch or fixed-epoch rule will likely score lower. Compare with the paper's numbers only after applying the same rule.
  • Baselines are not re-run. Other than ARGA, the baseline numbers are copied from the SDCN paper §4.3. Protocol differences in SDCN's paper carry over.
  • Fusion weights do not sum to one. The softmax is followed by an ℓ2 normalisation, so the two weights in each row have unit norm, not unit sum eq. 2 eq. 6. A reimplementation that assumes they sum to one will mis-scale the features.
  • Scale of the loss. Paper's KL and reconstruction terms are sums eq. 1 eq. 12. Code's are means (see Paper vs code item 5). Tuning the paper's λ values in a mean-based implementation will shift the balance between reconstruction and clustering.
  • K-means each step is costly. Algorithm 1 implies K-means on the full N×10 latent each iteration alg. 1. The paper gives no runtime, so the cost is unmeasured.
  • Dense graph multiplications. The adjacency is sparse in the code, but the paper's complexity term O(|E| Σ d_{i-1} d_i) assumes sparse products §3.4.
  • Unstated values. The text does not give the neighbour count k', the optimiser, the decoder widths, the GCN activation slope, or the weight initialisation §4.1 §4.3. The code uses Adam, decoder widths 2000-500-500, and Xavier initialisation AGCN.py AGCN.py GNN.py.
  • Only one dataset runs as released. Changing --name loads the matching files, but the λ values are fixed to USPS's AGCN.py, and the other datasets need the external download.
  • Evaluation edge case. cluster_acc overwrites predicted labels when the number of predicted classes differs from the truth, and returns 0 when the counts still differ evaluation.py evaluation.py.

Lineage

  • Builds on: the deep embedded clustering line of work (DEC) and its improved version (IDEC), which the paper uses as baselines §2 §4.2; and the GCN-plus-autoencoder clustering method SDCN, which it follows for settings and compares against §2 §4.3.
  • Compared against: AE, DEC, IDEC, GAE, VGAE, DAEGC, ARGA and SDCN, with eight methods in total §4.2 tab. 3.
  • Superseded by: not stated in the card's sources.

Source and license

Adapted from “Attention-driven Graph Clustering Network” by Zhihao Peng, Hui Liu, Yuheng Jia, Junhui Hou (arXiv:2108.05499), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/ZhihaoPENG-CityU/MM21---AGCN at commit 439de9935eef (none (no LICENSE file in the repository)). Any quoted code is shown under that license.