Paper · 2018-04-22 · cs.CL
Semi-supervised User Geolocation via Graph Convolutional Networks
Predicts a Twitter user's location from a bag-of-words text view and an @-mention graph, using a graph convolutional network whose layers have highway gates. Author-reported, with 1% of users labelled: median error 609 km (GeoText), 311 km (Twitter-US) and 853 km (Twitter-World), the lowest of the three joint models tested.
Graph neural networkSemi-supervised learningGated skip connectionDeep canonical correlation analysisgeolocationsocial-mediagraph-convolutionsemi-supervisedhighway-networks
arXivPDFCodeMarkdown14 min read
AI-compiled by claude-haiku-5-5 on 2026-11-13, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
Knowing where a social media user lives helps with things like spotting local events, but most people do not say where they are. Typically only about 1% of posts carry a location, so the model has to learn from mostly unlabelled users.
The idea is to use two kinds of clues about each user: what they write, and who they mention. People who mention each other tend to live near each other, so a user's own words can be mixed with the words of the people around them. A graph convolutional network does this mixing layer by layer. Each extra layer reaches one step further through the mention network.
Reaching too far picks up noise from distant users. The fix is a gate on each layer that decides how much neighbour information passes through. The authors report that these gates matter a lot, and that the method still works well when only a small fraction of users have labels.
TL;DR
- The model, GCN, combines a bag-of-words view of each user's tweets with a normalised @-mention graph. Each layer averages neighbours' features, and a highway gate controls how much of that average is kept §2.2 eq. 2 eq. 3.
- Locations are discretised into classes with a k-d tree. A prediction is mapped to a class location and scored by haversine distance using median, mean and Acc@161 §3.1 §3.3 §3.4.
- With 1% of users labelled, GCN has the lowest median error of the three joint models on all three datasets tab. 1.
- Without highway gates, median error on GeoText's development set rises steeply as layers are added. With gates it stays low fig. 5.
- The official code is by the first author. It has no licence file and differs from the paper in several places, including hop count, batch handling and a missing GCN-LP baseline. Read "Paper vs code" before implementing.
Problem
Labelled users are scarce. The paper notes that about 1% of tweets are geotagged, and that 1% is often the only supervision available §1. Text-only and network-only models ignore one view, and earlier joint models either ignore unlabelled data or use it in only one view §1 §5. Few earlier evaluations test the minimal-supervision case §5.
The model assumes location homophily: connected users tend to be near each other §5 §6. It ignores the context of each interaction §6.
Method
The network is transductive. Users are split into a labelled set and a held-out set, and the graph and text of all users are used during training §2.1.
- Labelled split. All users U are partitioned into labelled users U_S and held-out unlabelled users U_H. Predictions are made for U_H §2.1.
- Text view X. Each user's tweets are concatenated into one document §3.1. X is a bag of words with binary term frequency, inverse document frequency and L2 normalisation §3.2.
- Network view A. Users are linked if one @-mentions the other, or if they co-mention another user. The graph is then normalised to  with a self-loop weight λ, which the paper describes as "set to 1 in the original model" §3.2 §2.2 eq. 1.
- Location classes. Training coordinates are split with a k-d tree whose leaves hold at most a set number of users (the bucket size). Each leaf is one class §3.1 §3.3. The paper does not say which location represents a class; the code uses the median of the leaf's training points data.py.
- GCN layers. Each hidden layer is a highway graph convolution, with a neighbourhood average eq. 2 and a gate that mixes it with the layer input eq. 3 §2.2.1. The output layer is a graph convolution followed by softmax §3.3. The released code makes the first layer a text-only projection, which changes the hop count; see Paper vs code gcnmodel.py.
- Training. Cross-entropy is computed on labelled users only. The fraction of labelled users is counted over all users, including development and test users §4.2 gcnmain.py.
- MLP-TXT+NET baseline. One hidden ReLU layer takes the concatenation of a user's text row and network row §3.4.
- DCCA baseline. Two networks, one per view, are trained without labels to maximise the canonical correlation of their outputs §2.3 eq. 4 eq. 5. Linear CCA then reduces the outputs, and a supervised MLP is trained on the concatenation §2.3 §3.4.
- GCN-LP baseline. GCN with one-hot inputs instead of text, plus predicted labels fed back as inputs once training accuracy reaches 0.2 §3.4.
The paper's key equations, in its notation:
\hat{A} = \tilde{D}^{-1/2} (A + \lambda I) \tilde{D}^{-1/2} eq. 1
\vec{h}_i^{\,l+1} = \sigma\left( \sum_{j \in \mathrm{nhood}(i)} \hat{A}_{ij} \vec{h}_j^{\,l} W^l + b^l \right) eq. 2
T(\vec{h}^{\,l}) = \sigma\left( W_t^l \vec{h}^{\,l} + b_t^l \right), \quad \vec{h}^{\,l+1} = \tilde{\vec{h}}^{\,l+1} \circ T(\vec{h}^{\,l}) + \vec{h}^{\,l} \circ \left(1 - T(\vec{h}^{\,l})\right) eq. 3
\rho = \mathrm{corr}(f_1(X), f_2(\hat{A})) eq. 4
\max \ \mathrm{tr}(W_1^T \Sigma_{12} W_2) \quad \text{s.t.} \quad W_1^T \Sigma_{11} W_1 = W_2^T \Sigma_{22} W_2 = I eq. 5
The paper's own notation for eq. 2 uses σ as "an arbitrary nonlinearity" §2.2. The forward pass below follows the layer structure in the released code, which the paper's equations do not fully capture (see "Paper vs code").
# Input: X (N, V) binary-tf idf bag of words per user [§3.2]
# Input: A_hat (N, N) normalised adjacency [eq. 1]
# Output: probs (N, C) over location classes [§3.3]
def gcn_forward(X, A_hat, params, dropout_p, train):
# Layer 1 mixes text only; no neighbours are used [code: gcnmodel.py#L353]
hidden = tanh(X @ params.W_first + params.b_first) # (N, H)
hidden = dropout(hidden, dropout_p, train) # (N, H)
# Each highway layer averages neighbours, then gates [eq. 2] [eq. 3]
for layer in params.highway_layers: # len(hid) - 1
# Neighbourhood average of the layer input [eq. 2]
conv = tanh(A_hat @ (hidden @ layer.W_h) + layer.b_h) # (N, H)
# Gate uses the layer input, not the neighbours [eq. 3]
gate = sigmoid(hidden @ layer.W_t + layer.b_t) # (N, H)
hidden = gate * conv + (1 - gate) * hidden # (N, H)
# Output: one more graph convolution, then softmax [§3.3]
logits = A_hat @ (hidden @ params.W_out) + params.b_out # (N, C)
return softmax(logits) # (N, C)Training uses a full batch: each epoch makes one Adam step on all labelled users, with early stopping on the development loss gcnmodel.py. Pseudocode:
# Training: one full-batch Adam step per epoch [code: gcnmodel.py#L418]
# Input: X, A_hat, labels (N,) ints, train_idx, dev_idx
# Output: parameters with the lowest development loss
def train_gcn(X, A_hat, labels, train_idx, dev_idx):
best_dev_loss = infinity
best_params = None
epochs_since_best = 0
for epoch in range(10000): # maximum epochs [code: gcnmain.py#L221]
# Dropout is on in this pass [code: gcnmodel.py#L375-L382]
probs = gcn_forward(X, A_hat, params, 0.5, train=True)
train_loss = cross_entropy(probs[train_idx], labels[train_idx])
dev_loss = cross_entropy(probs[dev_idx], labels[dev_idx])
# Adam step; dev_loss above was computed before it [code: gcnmodel.py#L407-L410]
adam_step(params, train_loss, learning_rate=2e-3)
if dev_loss < best_dev_loss:
# dev_loss is from before this step, params are after it
# [code: gcnmodel.py#L434-L437]
best_dev_loss = dev_loss
best_params = copy(params)
epochs_since_best = 0
else:
epochs_since_best += 1
# Patience 10, and no stop before epoch 20 [code: gcnmodel.py#L445]
if epochs_since_best > 10 and epoch > 20:
break
return best_paramsPrediction maps the predicted class to its location, then computes the haversine distance to the user's true location gcnmain.py. The MLP-TXT+NET and DCCA baselines use the same evaluation.
Hyperparameters
Paper-stated values come from the text. Other rows are taken from the released code or its run scripts, and say so. "Not stated" means neither the paper text nor the code gives a value.
| Name | Value | Where | Notes |
|---|---|---|---|
| GCN layers (hop count) | 3 for all datasets | §3.3 | The first layer is not convolved; see Paper vs code |
| GCN hidden size | 300 (GeoText), 600 (Twitter-US), 900 (Twitter-World) | §3.3 | With highway gates only the first size sets the width |
| Highway gates | on | §2.2.1 §3.3 | Gate is a sigmoid of the layer input eq. 3 |
| Self-loop weight λ | 1 | §2.2 gcnmain.py | Paper says 1 is the value in Kipf and Welling; its own value is not stated in the text |
| Hidden nonlinearity | "arbitrary σ" in paper; tanh in code | §2.2 gcnmodel.py | |
| Output layer | graph convolution + softmax | §3.3 gcnmodel.py | |
| Gate weights | W_t orthogonal; W_h Glorot uniform (gain 1) | gcnmodel.py gcnmodel.py | Not stated in paper |
| Gate bias b_t | −4.0 | gcnmodel.py | Code comment suggests −2; not stated in paper |
| Optimiser (GCN) | Adam, lr 2e-3, β1 0.9, β2 0.999, ε 1e-8 | gcnmodel.py | Not stated in paper |
| Batch | full batch; -batch flag ignored | gcnmodel.py | One epoch is one update |
| Max epochs | 10000 | gcnmain.py | |
| Early stopping | dev loss, patience 10, no stop before epoch 20 | gcnmodel.py gcnmain.py | Not stated in paper |
| Dropout (GCN) | 0.5, after the first layer only | cmu-gcn-fractions.sh gcnmodel.py | Not stated in paper |
| L1 and L2 regularisation (GCN) | 0.0 in run scripts (code default 1e-6) | cmu-gcn-fractions.sh gcnmain.py | Not stated in paper |
| k-d tree bucket size | 50 (GeoText), 2400 (Twitter-US), 2400 (Twitter-World) | §3.3 save_model.sh README.md | Bucket size sets the number of classes |
| Number of classes | 129 (GeoText), 256 (Twitter-US), 930 (Twitter-World) | §3.1 | |
| Celebrity threshold | 5 (GeoText), 15 (Twitter-US), 5 (Twitter-World) | cmu-gcn-fractions.sh save_model.sh README.md | Non-target mentions with degree 1 or above threshold are dropped data.py. Not in paper |
| Min document frequency | 10 | gcnmain.py | Not in paper |
| Max document frequency | 0.2 | data.py | Not in paper |
| Stop words | English | data.py | Not in paper |
| Token pattern | keeps hashtags, removes @mentions | gcnmain.py | |
| Labelled fraction | fraction of all users, capped at training size | §4.2 gcnmain.py | Fractions 0.01 to 0.6 in CMU script |
| Seeds | default 77; runs use 1 to 5 | gcnmain.py cmu-gcn-fractions.sh | Figure 5 is averaged over 5 runs fig. 5 |
| MLP-TXT+NET hidden | 300 (GeoText), 600 (Twitter-US), 600 (Twitter-World), ReLU | §3.4 | |
| MLP-TXT+NET optimiser | Adam, lr 3e-4 (default), batch 300, up to 1000 epochs | deepcca.py mlp.py deepcca.py cmu-concat-fractions.sh | Not stated in paper |
| MLP-TXT+NET regularisation | 0.0 in run script; dropout 0.5 | cmu-concat-fractions.sh | Not stated in paper |
| DCCA unsupervised networks | one hidden layer of 1000 units, linear output of 500 | §3.4 cmu-dcca-fractions.sh | Paper says sigmoid; the run script sets ReLU |
| DCCA CCA output size | 500 per view (run script), 300 (code default) | cmu-dcca-fractions.sh deepcca.py | |
| DCCA regularisation | L2 1e-4 on networks; 1e-4 covariance ridge | deepcca.py deepcca.py | Not stated in paper |
| DCCA optimiser | Adam, lr 3e-4, 500 epochs, full batch | deepcca.py deepcca.py | Not stated in paper |
| DCCA supervised MLP | one hidden layer of 300 (GeoText), 600 (Twitter-US), 1000 (Twitter-World) | §3.4 | Tuned on development sets |
| GCN-LP inputs | one-hot user identity instead of text | §3.4 | Not implemented in the released code; see Paper vs code |
| Metrics | Median, Mean in km (haversine), Acc@161 (distance < 161 km) | §3.4 gcnmain.py |
Results
Author-reported. Setup: the three Twitter geolocation benchmarks GeoText, Twitter-US and Twitter-World §3.1. Metrics are Acc@161 (%), Mean and Median (km) §3.4. The table compares text-only, network-only and joint models tab. 1.
Rows with "1%" use 1% of all samples as labelled data tab. 1 caption. The paper does not say which split the main rows report.
| Model | Labelled | GeoText Acc@161 | GeoText Mean | GeoText Median | Twitter-US Acc@161 | Twitter-US Mean | Twitter-US Median | Twitter-World Acc@161 | Twitter-World Mean | Twitter-World Median |
|---|---|---|---|---|---|---|---|---|---|---|
| Rahimi et al. (2017b) | full | 38 | 844 | 389 | 54 | 554 | 120 | 34 | 1456 | 415 |
| Wing and Baldridge (2014) | full | — | — | — | 48 | 686 | 191 | 31 | 1669 | 509 |
| Cha et al. (2015) | full | — | 581 | 425 | — | — | — | — | — | — |
| Rahimi et al. (2015a) | full | 58 | 586 | 60 | 54 | 705 | 116 | 45 | 2525 | 279 |
| GCN-LP | full | 58 | 576 | 56 | 53 | 653 | 126 | 45 | 2357 | 279 |
| Do et al. (2017) | full | 62 | 532 | 32 | 66 | 433 | 45 | 53 | 1044 | 118 |
| Miura et al. (2017) | full | — | — | — | 61 | 481 | 65 | — | — | — |
| Rahimi et al. (2017b) | full | 59 | 578 | 61 | 61 | 515 | 77 | 53 | 1280 | 104 |
| MLP-TXT+NET | full | 58 | 554 | 58 | 66 | 420 | 56 | 58 | 1030 | 53 |
| DCCA | full | 56 | 627 | 79 | 58 | 516 | 90 | 21 | 2095 | 913 |
| GCN | full | 60 | 546 | 45 | 62 | 485 | 71 | 54 | 1130 | 108 |
| MLP-TXT+NET | 1% | 8 | 1521 | 1295 | 14 | 1436 | 1411 | 8 | 3865 | 2041 |
| DCCA | 1% | 7 | 1425 | 979 | 38 | 869 | 348 | 14 | 3014 | 1367 |
| GCN | 1% | 6 | 1103 | 609 | 41 | 788 | 311 | 21 | 2071 | 853 |
Source: tab. 1. The table caption says Do et al. (2017) use timezone and Miura et al. (2017) use the description and location fields, so they are not like-for-like with the text-plus-network models.
What the paper reports, from its text and figures:
- Full supervision. MLP-TXT+NET and GCN beat the text-only and network-only models. On Twitter-US and Twitter-World, MLP-TXT+NET beats GCN §4.4 tab. 1.
- Labelled-data curve. Below about 10% labelled, GCN and DCCA beat MLP-TXT+NET. Above about 20%, GCN and MLP-TXT+NET beat DCCA. MLP-TXT+NET beats GCN once 95% (Twitter-US) and 98% (Twitter-World) of training samples are used §4.2 fig. 4. Figure 4 is on the development set.
- Gates. With gates, median error on the GeoText development set stays roughly between 10 and 40 km from 1 to 10 layers. Without gates it rises to about 760 km at 10 layers. Values read from the plot fig. 5. The paper says performance peaks at 4 layers §4.3.
- Shortest paths. Most shortest-path lengths in the largest GeoText component are 3 or 4. The printed shares are 3% (length 2), 32% (3), 47% (4), 13% (5) and 2% (6) fig. 6.
- Error analysis. Errors cluster in some states and in less dense regions. Small states such as Rhode Island get high error, and misclassifications go towards large states §4.5 fig. 7.
- Local terms. Under 1% supervision, GCN surfaces place-specific terms that only occur in unlabelled users, such as "#goseahawks" for Seattle §4.6 tab. 2.
Paper vs code
The paper's footnote points to github.com/afshinrahimi/geoconv §2. That URL returns 404. A search found afshinrahimi/geographconv with the exact title as its description and the first author as owner. We use that repository, at commit 52c6fd3f6e147efe3360394f0da6392bd67f2db2. It has no LICENSE file, so code snippets are not quoted.
- Hop count. The paper says a 3-layer GCN has radius 4 once the softmax layer is counted §3.3. Its §2.2 says 3 layers reach 3 hops §2.2. With
-hid 300 300 300the code applies the adjacency three times: two highway layers and the output layer. The first layer is a text-only projection with no adjacency gcnmodel.py gcnmodel.py. - First layer. Eq. 2 with H⁰ = X would convolve the first layer §2.2 eq. 2. The code's first layer is
SparseInputDenseLayerwith no adjacency gcnmodel.py. - Layer widths. The paper gives sizes 300, 600 and 900 for the three datasets §3.3. With highway gates the code uses the first size for every layer, because highway layers copy their input width. Later sizes are ignored gcnmodel.py gcnmodel.py. The run scripts repeat one value, so the reported runs are unaffected.
- Batch. The
batch_sizeargument of the GCN fit routine is never used in the training loop. Every epoch is one full-batch Adam step gcnmodel.py. - Early stopping. The dev loss is computed in the dropout-on training pass, before the Adam step. The saved parameters are read after the step. The stored weights are therefore one update ahead of the loss that selected them gcnmodel.py gcnmodel.py.
- GCN-LP. The paper describes GCN-LP with one-hot inputs and predicted labels fed back once accuracy reaches 0.2 §3.4. The released
build_modelignores itsuse_textanduse_labelsarguments gcnmodel.py and never usespseudolikelihood_threshgcnmodel.py. GCN-LP cannot be reproduced from this code. - DCCA activation. The paper says sigmoid §3.4. The default is ReLU deepcca.py, and the CMU run script passes
-dccanonlin relucmu-dcca-fractions.sh. - DCCA weights. The CMU DCCA script loads pretrained DCCA weights from a file that is not in the repository cmu-dcca-fractions.sh. The DCCA pretraining cannot be reproduced from this code alone.
- No-highway runs. The no-highway layer experiment calls
resnet.py, which is not in the repository cmu_layers_nohighway.sh. - Self-loop. The code sets self-loop weight 1 explicitly gcnmain.py. The paper gives the value only as "set to 1 in the original model" §2.2.
- Paper not stated but in code. Max document frequency 0.2, English stop words, celebrity pruning, the cluster-median class location, the full-batch training schedule and all optimiser settings are only in the code data.py data.py data.py data.py gcnmodel.py.
Pitfalls
- Hop count is ambiguous in the paper. It gives both 3 hops and radius 4 for the same 3-layer model. The code applies the adjacency three times for
-hid 300 300 300. Pick one and test it on a small graph before comparing numbers. - Early-stopping snapshot is off by one step. The dev loss is measured before the update, but the saved parameters are taken after it. Copy the snapshot before the step if you want the loss and the weights to match gcnmodel.py.
- Dropout in the dev loss. The dev loss used for early stopping is computed with dropout on, so it is noisy. Evaluation uses the deterministic path gcnmodel.py gcnmodel.py.
- Full batch, many steps. One epoch is one update, and training runs up to 10000 epochs. The README gives runtimes of about 3 minutes on a GPU for GeoText, about 15 hours on CPU for Twitter-US, and about 2.5 days on CPU for Twitter-World README.md.
- Gate bias. The gate bias is set to −4 in code. The comment in the code suggests −2, and the paper gives no value gcnmodel.py.
- Class location is a median, not a centroid. Predicted class locations are per-leaf medians of the training points. Dev and test users are assigned to the nearest median by haversine distance. Results depend on the bucket size, which the paper sets per dataset data.py §3.3.
- Labelled fraction base. The fraction is taken over all users, including dev and test users, and capped at the training size. A "1%" run has about 1% of all users labelled, which is fewer than 1% of the training set §4.2 gcnmain.py.
- Graph filtering. Mentioned accounts outside the target set are dropped if they have degree 1 or a degree above the celebrity threshold. The threshold changes the graph and the results. The paper does not mention this step data.py.
- Vocabulary filtering. Words in fewer than 10 documents are dropped, words in over 20% of documents are dropped, and English stop words are removed. None of these appear in the paper's description of the text view gcnmain.py data.py data.py.
- Dataset sizes conflict. §3.1 gives 9k, 449k and 1.3m users for GeoText, Twitter-US and Twitter-World. The Figure 4 caption gives 9k, 440k and 1.4m §3.1 fig. 4.
- Data are not in the repository. The README says the raw dataset link is no longer available and points to a Google Drive folder with preprocessed dumps. We did not check that link README.md.
- Python and library versions. The MLP and DCCA code uses Python 2 constructs (
sys.maxint,xrange). The requirements pin Theano 1.0.3 and Lasagne 0.1 deepcca.py mlp.py requirements.txt. A port to a current framework will need care with the sparse layers. - Unknown splits and seeds. The text does not say which split Table 1 reports, or whether its numbers are averaged over seeds. Figure 5 is averaged over 5 runs fig. 5.
- Not stated. The paper gives no optimiser, learning rate, dropout, epochs, early-stopping rule, gate bias, or regularisation value. All of these come from code or scripts.
Lineage
- Builds on: Kipf and Welling (2017), the graph convolution in eq. 2 §2.2. Highway networks (Srivastava et al., 2015), whose gates the model copies §2.2.1. Deep canonical correlation analysis (Andrew et al., 2013), the DCCA baseline §2.3.
- Compared against: text-only models (Wing and Baldridge, 2014; Rahimi et al., 2017b), network-only models (Rahimi et al., 2015a), and joint models (Do et al., 2017; Miura et al., 2017) §5 tab. 1. GCN-LP is the paper's own baseline §3.4.
- Superseded by: not known from wiki content.
Source and license
Adapted from “Semi-supervised User Geolocation via Graph Convolutional Networks” by Afshin Rahimi, Trevor Cohn, Timothy Baldwin (arXiv:1804.08049), used under CC0 1.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors. Code citations refer to https://github.com/afshinrahimi/geographconv at commit 52c6fd3f6e14 (none (no LICENSE file in the repository)). Any quoted code is shown under that license.