Location encoder

Overview

A location encoder turns a longitude and latitude pair into a vector that a downstream model can use in place of the raw numbers. Raw coordinates generalise poorly: a model built on them tends to work only where the evaluation area overlaps the training area 2311.17179 §1. Image-based alternatives avoid this but need a satellite image and a vision model for every query point 2311.17179 §2.

The idea shared by the learned variants is to pretrain the encoder so that places with similar ground appearance get similar embeddings, then freeze it and use its output as features. The invariant that holds across the variants is that prediction needs only the coordinates, not imagery 2311.17179 §2.

Only one paper with a card covers this concept so far, so the variants below all come from that paper. Other encoders it compares against are named only.

Variants

Raw coordinates

Feed longitude and latitude straight into the downstream model. The paper uses this as its "Identity" baseline 2311.17179 tab. 2. It needs no pretraining and no imagery.

Spherical harmonics with a SIREN MLP, pretrained against satellite tiles

The coordinates are expanded into real spherical-harmonic basis functions up to a degree limit L. A two-hidden-layer SIREN network with 512-unit hidden layers maps this to a 256-dimensional embedding 2311.17179 §2.2 2311.17179 app. C.1. Training matches each coordinate with the satellite tile at that place, using a CLIP-style contrastive loss against a frozen MoCo image encoder whose last linear projection is trained 2311.17179 §2.1 2311.17179 §2.2. Card: SatCLIP.

• L = 10. The authors report this setting as the better generaliser to a continent held out from training 2311.17179 §4.2.
• L = 40. A finer basis. The authors report it as stronger on in-distribution regional tasks and on interpolation 2311.17179 §4.1 2311.17179 §4.4.

The paper's released checkpoints are the L = 10 and L = 40 models 2311.17179 §2.2.

Other pretrained location encoders

CSP, GPS2Vec, GeoCLIP and MOSAIKS are the comparison methods in the paper 2311.17179 §3.3 2311.17179 tab. 1. They have no cards here, so this page does not describe how they work.

When to use which

The rows follow the constraints. The results are the paper's own, on its own tasks, and the authors report them.

Two points limit these choices. Table 2 and Table 3 use different image encoders (ResNet50 and ViT16), so compare numbers only within a table 2311.17179 tab. 2 2311.17179 tab. 3. The paper also reports that the vision encoder changes results by under 1%, while L changes them more 2311.17179 §4.4.

Implementation sketch

Language-agnostic outline of the learned encoder's forward pass, following the paper's description. Values the paper leaves open are marked.

# Learned location encoder  [arxiv:2311.17179 §2.2]
# Input:  coords (N, 2) as (longitude, latitude) in degrees
# Output: embedding (N, 256)

def location_encoder(coords, L):
    # Real spherical-harmonic basis, degrees below L  [arxiv:2311.17179 §2.2]
    basis = spherical_harmonic_features(coords, L)     # (N, basis size)

    # First SIREN layer: linear, then sine  [arxiv:2311.17179 app. C.1]
    # w0 is the sine frequency; the paper does not state it, see Contradictions
    h = sin(w0 * (basis @ W1 + b1))

    # Second hidden layer, 512 units in the paper  [arxiv:2311.17179 app. C.1]
    h = sin(h @ W2 + b2)

    # Linear map to the 256-dimensional output  [arxiv:2311.17179 §2.2]
    return h @ W3 + b3

Pretraining objective, one batch of N coordinate and tile pairs 2311.17179 §2.1:

# Symmetric contrastive loss over one batch  [arxiv:2311.17179 §2.1]
# The true pair for row i is column i

def contrastive_loss(tile_emb, loc_emb, tau):
    # Unit-normalise both sides, so dot products are cosines
    tile_emb = tile_emb / norm(tile_emb)
    loc_emb = loc_emb / norm(loc_emb)
    # N x N similarity matrix, scaled by the temperature tau
    logits = (tile_emb @ loc_emb.T) / tau
    # Each tile against all coordinates, and each coordinate against all tiles
    # tau is a hyperparameter; the paper gives no number  [arxiv:2311.17179 §2.1]
    return (cross_entropy(logits, arange(N))
            + cross_entropy(logits.T, arange(N))) / 2

Notes:

• Input order matters. The encoder expects longitude first and latitude second in degrees. Swapping the columns gives a valid but wrong embedding.
• The batch size matters for the contrastive loss. The paper uses 8k, so each coordinate is compared against 8k tiles per step 2311.17179 §2.3.
• The released code differs from the paper in several settings, listed under Contradictions.

Contradictions

Each side is cited. None of these are resolved here.

• Batch size. The paper trains with 8k 2311.17179 §2.3. The repository's default config uses 512 with 16 accumulation steps, which gives 512 negatives per loss computation. See the SatCLIP card, Paper vs code.
• SIREN width. The paper uses 512 hidden units 2311.17179 app. C.1. The repository's default is 256 (same card).
• Degree limit. The paper releases L = 10 and L = 40 2311.17179 §2.2. The repository's default is 32 (same card).
• Image encoder. The paper uses a frozen MoCo encoder with only the last projection trained 2311.17179 §2.2. The repository's default config builds a smaller ViT from scratch with no freezing (same card).
• ViT16 backbone. The paper gives about 22M parameters for ViT16 2311.17179 §2.2. The repository's README names ViT-B/16 and its code uses a ViT-Small (same card).
• Sine frequency and temperature. The paper gives no number for the sine frequency or the temperature τ 2311.17179 §2.1. The repository's code sets a specific first-layer frequency and learns τ (same card).
• Dataset licence. The paper does not state a licence for its satellite tile dataset 2311.17179 app. A. Check it before reusing the tiles commercially. The paper's own CC BY 4.0 licence covers only the paper.
• Author claims. The authors report the encoder best on seven of nine in-distribution tasks 2311.17179 §4.1. Table 2 has GeoCLIP equal or ahead on the two income and housing tasks 2311.17179 tab. 2. All results are author-reported, and the released checkpoints have not been checked against the paper here (see the card).

No contradictions with other cards are known, since this is the only card so far.

Papers

• SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery: contrastive pretraining of a spherical-harmonic and SIREN location encoder against Sentinel-2 tiles, with L = 10 and L = 40 models.