wikillm

Paper · 2018-04-22 · cs.CL

Semi-supervised User Geolocation via Graph Convolutional Networks

Predicts a Twitter user's location from a bag-of-words text view and an @-mention graph, using a graph convolutional network whose layers have highway gates. Author-reported, with 1% of users labelled: median error 609 km (GeoText), 311 km (Twitter-US) and 853 km (Twitter-World), the lowest of the three joint models tested.

Graph neural networkSemi-supervised learningGated skip connectionDeep canonical correlation analysisgeolocationsocial-mediagraph-convolutionsemi-supervisedhighway-networks

arXivPDFCodeMarkdown14 min read

AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.

In plain words

Knowing where a social media user lives helps with things like spotting local events, but most people do not say where they are. Typically only about 1% of posts carry a location, so the model has to learn from mostly unlabelled users.

The idea is to use two kinds of clues about each user: what they write, and who they mention. People who mention each other tend to live near each other, so a user's own words can be mixed with the words of the people around them. A graph convolutional network does this mixing layer by layer. Each extra layer reaches one step further through the mention network.

Reaching too far picks up noise from distant users. The fix is a gate on each layer that decides how much neighbour information passes through. The authors report that these gates matter a lot, and that the method still works well when only a small fraction of users have labels.

TL;DR

  • The model, GCN, combines a bag-of-words view of each user's tweets with a normalised @-mention graph. Each layer averages neighbours' features, and a highway gate controls how much of that average is kept §2.2 eq. 2 eq. 3.
  • Locations are discretised into classes with a k-d tree. A prediction is mapped to a class location and scored by haversine distance using median, mean and Acc@161 §3.1 §3.3 §3.4.
  • With 1% of users labelled, GCN has the lowest median error of the three joint models on all three datasets tab. 1.
  • Without highway gates, median error on GeoText's development set rises steeply as layers are added. With gates it stays low fig. 5.
  • The official code is by the first author. It has no licence file and differs from the paper in several places, including hop count, batch handling and a missing GCN-LP baseline. Read "Paper vs code" before implementing.

Problem

Labelled users are scarce. The paper notes that about 1% of tweets are geotagged, and that 1% is often the only supervision available §1. Text-only and network-only models ignore one view, and earlier joint models either ignore unlabelled data or use it in only one view §1 §5. Few earlier evaluations test the minimal-supervision case §5.

The model assumes location homophily: connected users tend to be near each other §5 §6. It ignores the context of each interaction §6.

Method

The network is transductive. Users are split into a labelled set and a held-out set, and the graph and text of all users are used during training §2.1.

  1. Labelled split. All users U are partitioned into labelled users U_S and held-out unlabelled users U_H. Predictions are made for U_H §2.1.
  2. Text view X. Each user's tweets are concatenated into one document §3.1. X is a bag of words with binary term frequency, inverse document frequency and L2 normalisation §3.2.
  3. Network view A. Users are linked if one @-mentions the other, or if they co-mention another user. The graph is then normalised to  with a self-loop weight λ, which the paper describes as "set to 1 in the original model" §3.2 §2.2 eq. 1.
  4. Location classes. Training coordinates are split with a k-d tree whose leaves hold at most a set number of users (the bucket size). Each leaf is one class §3.1 §3.3. The paper does not say which location represents a class; the code uses the median of the leaf's training points data.py.
  5. GCN layers. Each hidden layer is a highway graph convolution, with a neighbourhood average eq. 2 and a gate that mixes it with the layer input eq. 3 §2.2.1. The output layer is a graph convolution followed by softmax §3.3. The released code makes the first layer a text-only projection, which changes the hop count; see Paper vs code gcnmodel.py.
  6. Training. Cross-entropy is computed on labelled users only. The fraction of labelled users is counted over all users, including development and test users §4.2 gcnmain.py.
  7. MLP-TXT+NET baseline. One hidden ReLU layer takes the concatenation of a user's text row and network row §3.4.
  8. DCCA baseline. Two networks, one per view, are trained without labels to maximise the canonical correlation of their outputs §2.3 eq. 4 eq. 5. Linear CCA then reduces the outputs, and a supervised MLP is trained on the concatenation §2.3 §3.4.
  9. GCN-LP baseline. GCN with one-hot inputs instead of text, plus predicted labels fed back as inputs once training accuracy reaches 0.2 §3.4.

The paper's key equations, in its notation:

\hat{A} = \tilde{D}^{-1/2} (A + \lambda I) \tilde{D}^{-1/2} eq. 1

\vec{h}_i^{\,l+1} = \sigma\left( \sum_{j \in \mathrm{nhood}(i)} \hat{A}_{ij} \vec{h}_j^{\,l} W^l + b^l \right) eq. 2

T(\vec{h}^{\,l}) = \sigma\left( W_t^l \vec{h}^{\,l} + b_t^l \right), \quad \vec{h}^{\,l+1} = \tilde{\vec{h}}^{\,l+1} \circ T(\vec{h}^{\,l}) + \vec{h}^{\,l} \circ \left(1 - T(\vec{h}^{\,l})\right) eq. 3

\rho = \mathrm{corr}(f_1(X), f_2(\hat{A})) eq. 4

\max \ \mathrm{tr}(W_1^T \Sigma_{12} W_2) \quad \text{s.t.} \quad W_1^T \Sigma_{11} W_1 = W_2^T \Sigma_{22} W_2 = I eq. 5

The paper's own notation for eq. 2 uses σ as "an arbitrary nonlinearity" §2.2. The forward pass below follows the layer structure in the released code, which the paper's equations do not fully capture (see "Paper vs code").

Pseudocode
# Input:  X (N, V) binary-tf idf bag of words per user      [§3.2]
# Input:  A_hat (N, N) normalised adjacency                   [eq. 1]
# Output: probs (N, C) over location classes                  [§3.3]

def gcn_forward(X, A_hat, params, dropout_p, train):
    # Layer 1 mixes text only; no neighbours are used  [code: gcnmodel.py#L353]
    hidden = tanh(X @ params.W_first + params.b_first)    # (N, H)
    hidden = dropout(hidden, dropout_p, train)            # (N, H)
    # Each highway layer averages neighbours, then gates  [eq. 2] [eq. 3]
    for layer in params.highway_layers:                   # len(hid) - 1
        # Neighbourhood average of the layer input  [eq. 2]
        conv = tanh(A_hat @ (hidden @ layer.W_h) + layer.b_h)  # (N, H)
        # Gate uses the layer input, not the neighbours  [eq. 3]
        gate = sigmoid(hidden @ layer.W_t + layer.b_t)         # (N, H)
        hidden = gate * conv + (1 - gate) * hidden             # (N, H)
    # Output: one more graph convolution, then softmax  [§3.3]
    logits = A_hat @ (hidden @ params.W_out) + params.b_out  # (N, C)
    return softmax(logits)                                   # (N, C)

Training uses a full batch: each epoch makes one Adam step on all labelled users, with early stopping on the development loss gcnmodel.py. Pseudocode:

Pseudocode
# Training: one full-batch Adam step per epoch  [code: gcnmodel.py#L418]
# Input:  X, A_hat, labels (N,) ints, train_idx, dev_idx
# Output: parameters with the lowest development loss

def train_gcn(X, A_hat, labels, train_idx, dev_idx):
    best_dev_loss = infinity
    best_params = None
    epochs_since_best = 0
    for epoch in range(10000):    # maximum epochs  [code: gcnmain.py#L221]
        # Dropout is on in this pass  [code: gcnmodel.py#L375-L382]
        probs = gcn_forward(X, A_hat, params, 0.5, train=True)
        train_loss = cross_entropy(probs[train_idx], labels[train_idx])
        dev_loss = cross_entropy(probs[dev_idx], labels[dev_idx])
        # Adam step; dev_loss above was computed before it  [code: gcnmodel.py#L407-L410]
        adam_step(params, train_loss, learning_rate=2e-3)
        if dev_loss < best_dev_loss:
            # dev_loss is from before this step, params are after it
            # [code: gcnmodel.py#L434-L437]
            best_dev_loss = dev_loss
            best_params = copy(params)
            epochs_since_best = 0
        else:
            epochs_since_best += 1
        # Patience 10, and no stop before epoch 20  [code: gcnmodel.py#L445]
        if epochs_since_best > 10 and epoch > 20:
            break
    return best_params

Prediction maps the predicted class to its location, then computes the haversine distance to the user's true location gcnmain.py. The MLP-TXT+NET and DCCA baselines use the same evaluation.

Hyperparameters

Paper-stated values come from the text. Other rows are taken from the released code or its run scripts, and say so. "Not stated" means neither the paper text nor the code gives a value.

NameValueWhereNotes
GCN layers (hop count)3 for all datasets§3.3The first layer is not convolved; see Paper vs code
GCN hidden size300 (GeoText), 600 (Twitter-US), 900 (Twitter-World)§3.3With highway gates only the first size sets the width
Highway gateson§2.2.1 §3.3Gate is a sigmoid of the layer input eq. 3
Self-loop weight λ1§2.2 gcnmain.pyPaper says 1 is the value in Kipf and Welling; its own value is not stated in the text
Hidden nonlinearity"arbitrary σ" in paper; tanh in code§2.2 gcnmodel.py
Output layergraph convolution + softmax§3.3 gcnmodel.py
Gate weightsW_t orthogonal; W_h Glorot uniform (gain 1)gcnmodel.py gcnmodel.pyNot stated in paper
Gate bias b_t−4.0gcnmodel.pyCode comment suggests −2; not stated in paper
Optimiser (GCN)Adam, lr 2e-3, β1 0.9, β2 0.999, ε 1e-8gcnmodel.pyNot stated in paper
Batchfull batch; -batch flag ignoredgcnmodel.pyOne epoch is one update
Max epochs10000gcnmain.py
Early stoppingdev loss, patience 10, no stop before epoch 20gcnmodel.py gcnmain.pyNot stated in paper
Dropout (GCN)0.5, after the first layer onlycmu-gcn-fractions.sh gcnmodel.pyNot stated in paper
L1 and L2 regularisation (GCN)0.0 in run scripts (code default 1e-6)cmu-gcn-fractions.sh gcnmain.pyNot stated in paper
k-d tree bucket size50 (GeoText), 2400 (Twitter-US), 2400 (Twitter-World)§3.3 save_model.sh README.mdBucket size sets the number of classes
Number of classes129 (GeoText), 256 (Twitter-US), 930 (Twitter-World)§3.1
Celebrity threshold5 (GeoText), 15 (Twitter-US), 5 (Twitter-World)cmu-gcn-fractions.sh save_model.sh README.mdNon-target mentions with degree 1 or above threshold are dropped data.py. Not in paper
Min document frequency10gcnmain.pyNot in paper
Max document frequency0.2data.pyNot in paper
Stop wordsEnglishdata.pyNot in paper
Token patternkeeps hashtags, removes @mentionsgcnmain.py
Labelled fractionfraction of all users, capped at training size§4.2 gcnmain.pyFractions 0.01 to 0.6 in CMU script
Seedsdefault 77; runs use 1 to 5gcnmain.py cmu-gcn-fractions.shFigure 5 is averaged over 5 runs fig. 5
MLP-TXT+NET hidden300 (GeoText), 600 (Twitter-US), 600 (Twitter-World), ReLU§3.4
MLP-TXT+NET optimiserAdam, lr 3e-4 (default), batch 300, up to 1000 epochsdeepcca.py mlp.py deepcca.py cmu-concat-fractions.shNot stated in paper
MLP-TXT+NET regularisation0.0 in run script; dropout 0.5cmu-concat-fractions.shNot stated in paper
DCCA unsupervised networksone hidden layer of 1000 units, linear output of 500§3.4 cmu-dcca-fractions.shPaper says sigmoid; the run script sets ReLU
DCCA CCA output size500 per view (run script), 300 (code default)cmu-dcca-fractions.sh deepcca.py
DCCA regularisationL2 1e-4 on networks; 1e-4 covariance ridgedeepcca.py deepcca.pyNot stated in paper
DCCA optimiserAdam, lr 3e-4, 500 epochs, full batchdeepcca.py deepcca.pyNot stated in paper
DCCA supervised MLPone hidden layer of 300 (GeoText), 600 (Twitter-US), 1000 (Twitter-World)§3.4Tuned on development sets
GCN-LP inputsone-hot user identity instead of text§3.4Not implemented in the released code; see Paper vs code
MetricsMedian, Mean in km (haversine), Acc@161 (distance < 161 km)§3.4 gcnmain.py

Results

Author-reported. Setup: the three Twitter geolocation benchmarks GeoText, Twitter-US and Twitter-World §3.1. Metrics are Acc@161 (%), Mean and Median (km) §3.4. The table compares text-only, network-only and joint models tab. 1.

Rows with "1%" use 1% of all samples as labelled data tab. 1 caption. The paper does not say which split the main rows report.

ModelLabelledGeoText Acc@161GeoText MeanGeoText MedianTwitter-US Acc@161Twitter-US MeanTwitter-US MedianTwitter-World Acc@161Twitter-World MeanTwitter-World Median
Rahimi et al. (2017b)full3884438954554120341456415
Wing and Baldridge (2014)full———48686191311669509
Cha et al. (2015)full—581425——————
Rahimi et al. (2015a)full585866054705116452525279
GCN-LPfull585765653653126452357279
Do et al. (2017)full62532326643345531044118
Miura et al. (2017)full———6148165———
Rahimi et al. (2017b)full59578616151577531280104
MLP-TXT+NETfull5855458664205658103053
DCCAfull56627795851690212095913
GCNfull60546456248571541130108
MLP-TXT+NET1%8152112951414361411838652041
DCCA1%71425979388693481430141367
GCN1%6110360941788311212071853

Source: tab. 1. The table caption says Do et al. (2017) use timezone and Miura et al. (2017) use the description and location fields, so they are not like-for-like with the text-plus-network models.

What the paper reports, from its text and figures:

  • Full supervision. MLP-TXT+NET and GCN beat the text-only and network-only models. On Twitter-US and Twitter-World, MLP-TXT+NET beats GCN §4.4 tab. 1.
  • Labelled-data curve. Below about 10% labelled, GCN and DCCA beat MLP-TXT+NET. Above about 20%, GCN and MLP-TXT+NET beat DCCA. MLP-TXT+NET beats GCN once 95% (Twitter-US) and 98% (Twitter-World) of training samples are used §4.2 fig. 4. Figure 4 is on the development set.
  • Gates. With gates, median error on the GeoText development set stays roughly between 10 and 40 km from 1 to 10 layers. Without gates it rises to about 760 km at 10 layers. Values read from the plot fig. 5. The paper says performance peaks at 4 layers §4.3.
  • Shortest paths. Most shortest-path lengths in the largest GeoText component are 3 or 4. The printed shares are 3% (length 2), 32% (3), 47% (4), 13% (5) and 2% (6) fig. 6.
  • Error analysis. Errors cluster in some states and in less dense regions. Small states such as Rhode Island get high error, and misclassifications go towards large states §4.5 fig. 7.
  • Local terms. Under 1% supervision, GCN surfaces place-specific terms that only occur in unlabelled users, such as "#goseahawks" for Seattle §4.6 tab. 2.

Paper vs code

The paper's footnote points to github.com/afshinrahimi/geoconv §2. That URL returns 404. A search found afshinrahimi/geographconv with the exact title as its description and the first author as owner. We use that repository, at commit 52c6fd3f6e147efe3360394f0da6392bd67f2db2. It has no LICENSE file, so code snippets are not quoted.

  • Hop count. The paper says a 3-layer GCN has radius 4 once the softmax layer is counted §3.3. Its §2.2 says 3 layers reach 3 hops §2.2. With -hid 300 300 300 the code applies the adjacency three times: two highway layers and the output layer. The first layer is a text-only projection with no adjacency gcnmodel.py gcnmodel.py.
  • First layer. Eq. 2 with H⁰ = X would convolve the first layer §2.2 eq. 2. The code's first layer is SparseInputDenseLayer with no adjacency gcnmodel.py.
  • Layer widths. The paper gives sizes 300, 600 and 900 for the three datasets §3.3. With highway gates the code uses the first size for every layer, because highway layers copy their input width. Later sizes are ignored gcnmodel.py gcnmodel.py. The run scripts repeat one value, so the reported runs are unaffected.
  • Batch. The batch_size argument of the GCN fit routine is never used in the training loop. Every epoch is one full-batch Adam step gcnmodel.py.
  • Early stopping. The dev loss is computed in the dropout-on training pass, before the Adam step. The saved parameters are read after the step. The stored weights are therefore one update ahead of the loss that selected them gcnmodel.py gcnmodel.py.
  • GCN-LP. The paper describes GCN-LP with one-hot inputs and predicted labels fed back once accuracy reaches 0.2 §3.4. The released build_model ignores its use_text and use_labels arguments gcnmodel.py and never uses pseudolikelihood_thresh gcnmodel.py. GCN-LP cannot be reproduced from this code.
  • DCCA activation. The paper says sigmoid §3.4. The default is ReLU deepcca.py, and the CMU run script passes -dccanonlin relu cmu-dcca-fractions.sh.
  • DCCA weights. The CMU DCCA script loads pretrained DCCA weights from a file that is not in the repository cmu-dcca-fractions.sh. The DCCA pretraining cannot be reproduced from this code alone.
  • No-highway runs. The no-highway layer experiment calls resnet.py, which is not in the repository cmu_layers_nohighway.sh.
  • Self-loop. The code sets self-loop weight 1 explicitly gcnmain.py. The paper gives the value only as "set to 1 in the original model" §2.2.
  • Paper not stated but in code. Max document frequency 0.2, English stop words, celebrity pruning, the cluster-median class location, the full-batch training schedule and all optimiser settings are only in the code data.py data.py data.py data.py gcnmodel.py.

Pitfalls

  • Hop count is ambiguous in the paper. It gives both 3 hops and radius 4 for the same 3-layer model. The code applies the adjacency three times for -hid 300 300 300. Pick one and test it on a small graph before comparing numbers.
  • Early-stopping snapshot is off by one step. The dev loss is measured before the update, but the saved parameters are taken after it. Copy the snapshot before the step if you want the loss and the weights to match gcnmodel.py.
  • Dropout in the dev loss. The dev loss used for early stopping is computed with dropout on, so it is noisy. Evaluation uses the deterministic path gcnmodel.py gcnmodel.py.
  • Full batch, many steps. One epoch is one update, and training runs up to 10000 epochs. The README gives runtimes of about 3 minutes on a GPU for GeoText, about 15 hours on CPU for Twitter-US, and about 2.5 days on CPU for Twitter-World README.md.
  • Gate bias. The gate bias is set to −4 in code. The comment in the code suggests −2, and the paper gives no value gcnmodel.py.
  • Class location is a median, not a centroid. Predicted class locations are per-leaf medians of the training points. Dev and test users are assigned to the nearest median by haversine distance. Results depend on the bucket size, which the paper sets per dataset data.py §3.3.
  • Labelled fraction base. The fraction is taken over all users, including dev and test users, and capped at the training size. A "1%" run has about 1% of all users labelled, which is fewer than 1% of the training set §4.2 gcnmain.py.
  • Graph filtering. Mentioned accounts outside the target set are dropped if they have degree 1 or a degree above the celebrity threshold. The threshold changes the graph and the results. The paper does not mention this step data.py.
  • Vocabulary filtering. Words in fewer than 10 documents are dropped, words in over 20% of documents are dropped, and English stop words are removed. None of these appear in the paper's description of the text view gcnmain.py data.py data.py.
  • Dataset sizes conflict. §3.1 gives 9k, 449k and 1.3m users for GeoText, Twitter-US and Twitter-World. The Figure 4 caption gives 9k, 440k and 1.4m §3.1 fig. 4.
  • Data are not in the repository. The README says the raw dataset link is no longer available and points to a Google Drive folder with preprocessed dumps. We did not check that link README.md.
  • Python and library versions. The MLP and DCCA code uses Python 2 constructs (sys.maxint, xrange). The requirements pin Theano 1.0.3 and Lasagne 0.1 deepcca.py mlp.py requirements.txt. A port to a current framework will need care with the sparse layers.
  • Unknown splits and seeds. The text does not say which split Table 1 reports, or whether its numbers are averaged over seeds. Figure 5 is averaged over 5 runs fig. 5.
  • Not stated. The paper gives no optimiser, learning rate, dropout, epochs, early-stopping rule, gate bias, or regularisation value. All of these come from code or scripts.

Lineage

  • Builds on: Kipf and Welling (2017), the graph convolution in eq. 2 §2.2. Highway networks (Srivastava et al., 2015), whose gates the model copies §2.2.1. Deep canonical correlation analysis (Andrew et al., 2013), the DCCA baseline §2.3.
  • Compared against: text-only models (Wing and Baldridge, 2014; Rahimi et al., 2017b), network-only models (Rahimi et al., 2015a), and joint models (Do et al., 2017; Miura et al., 2017) §5 tab. 1. GCN-LP is the paper's own baseline §3.4.
  • Superseded by: not known from wiki content.

Source and license

Adapted from “Semi-supervised User Geolocation via Graph Convolutional Networks” by Afshin Rahimi, Trevor Cohn, Timothy Baldwin (arXiv:1804.08049), used under CC0 1.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/afshinrahimi/geographconv at commit 52c6fd3f6e14 (none (no LICENSE file in the repository)). Any quoted code is shown under that license.