---
kind: concept
slug: molecular-representation-learning
title: Molecular representation learning
summary: >-
  Turning a molecule into a vector a model can use. Variants: fixed or learned fingerprints, RDKit descriptors, CNNs and RNNs over SMILES, Transformers over SMILES or substructure tokens, 2D message passing, and 3D equivariant networks. Generative 3D models such as GeoDiff output atom coordinates instead. The sources test different tasks, so their numbers are not comparable.
tags: [drug-discovery, representation-learning, chemistry]
papers: ["2004.08919", "2505.01912", "2203.02923", "2106.05234", "2110.07875", "1509.09292", "2103.09430"]
related: [convolutional-neural-network, recurrent-neural-network, graph-neural-network, graph-transformer, positional-encoding, attention-mechanism, equivariance-constraint, masked-language-modeling, hyperparameter-optimization, diffusion-models, benchmark-validity]
status: draft
updated_at: 2026-10-24
---

## Overview

A model that predicts a molecule's behaviour needs the molecule as numbers. A
compound encoder maps the molecule to a vector, and the rest of the model works
on that vector. In DeepPurpose the compound input is a SMILES string, a text
notation for the molecule [arxiv:2004.08919 §2.1]. BOOM uses several inputs:
descriptors, SMILES, a 2D bond graph, and 3D atom positions or distances
[arxiv:2505.01912 tab. 3]. GeoDiff goes the other way: it takes a bond graph
and produces 3D atom coordinates [arxiv:2203.02923 §3.1].

The encoders differ in what they read. They use precomputed fingerprints or
descriptors, fingerprints that are learned end to end with the predictor,
learned sequence models over characters or substructure tokens,
message passing over the bond graph, attention over all atom pairs with
graph-structure biases, or 3D models that also read geometry
[arxiv:2004.08919 app. A] [arxiv:2505.01912 tab. 3] [arxiv:2106.05234 §3.1]
[arxiv:2103.09430 §2.3].

The shared invariant is that each predictive encoder outputs a vector. In
DeepPurpose that vector is joined with a protein embedding and passed to an MLP
[arxiv:2004.08919 §2.2]. In BOOM the models are used off the shelf, and only
non-architectural settings were tuned [arxiv:2505.01912 app. C.1]. GeoDiff does
not fit this pattern. Its network predicts noise on coordinates inside a
sampling loop, and the conformation that loop produces is the output
[arxiv:2203.02923 alg. 1]. Protein encoders are out of scope for this page.

The sources test different things. DeepPurpose compares encoder pairs on
drug-target binding benchmarks [arxiv:2004.08919 §3]. The learned-fingerprint
paper tests three regression sets: solubility, antimalarial drug efficacy and
organic photovoltaic efficiency [arxiv:1509.09292 §4.2]. BOOM compares 15 models on
10 molecular properties, with a split that holds out the molecules with the most
unusual property values [arxiv:2505.01912 §2]. OGB-LSC regresses the HOMO-LUMO
gap of PubChem molecules from 2D graphs, split by compound ID, with a fixed
inference budget [arxiv:2103.09430 §2.3]. GeoDiff measures generated
conformations against reference ensembles on GEOM-QM9 and GEOM-Drugs
[arxiv:2203.02923 §5.1]. Their numbers are not comparable.

## Variants

### Fingerprints or descriptors with an MLP or random forest

DeepPurpose computes Morgan, PubChem, Daylight and RDKit 2D fingerprints from the
molecule and passes them to an MLP. The paper's sizes are Morgan with radius 2
and 1024 bits, PubChem with 881 bits, Daylight with 2048 bits, and RDKit 2D with
200 values [arxiv:2004.08919 app. A]. Card:
[DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction](/papers/2004.08919).

BOOM feeds RDKit molecular descriptors to a random forest and to an MLP. The
descriptor set is 125 chemistry features and 86 functional-group fractions
[arxiv:2505.01912 app. B.2]. The MLP has 153k parameters [arxiv:2505.01912 tab. 3].
On the 10 OOD tasks, the random forest's error is higher than the MLP's on 8
[arxiv:2505.01912 tab. 7]. Card: [BOOM](/papers/2505.01912).

OGB-LSC uses a 2048-bit Morgan fingerprint as the input of an MLP baseline
for PCQM4M, with a grid over radius 2 and 3 and a 16.1M-parameter model
[arxiv:2103.09430 §2.3] [tab. 6]. Its test MAE is 0.2070 eV, against 0.1494
for the best single GNN in the same table [arxiv:2103.09430 tab. 6]. Card:
[OGB-LSC](/papers/2103.09430).

### Learned fingerprints trained end to end

Neural graph fingerprints replace the hashing and indexing steps of circular
fingerprints with differentiable ones [arxiv:1509.09292 §3] [alg. 2]. Each
layer lets every atom take in its neighbours' features, and each atom then
gives a softmax over the fingerprint slots. The votes are summed into the
fingerprint, so the fingerprint is trained together with the prediction head
[arxiv:1509.09292 §3] [alg. 2]. In the paper's Table 1, the neural fingerprint
with a one-hidden-layer head gives 0.52 on solubility against 1.40 for circular
fingerprints with the same head, and 1.43 against 2.00 on photovoltaic
efficiency [arxiv:1509.09292 tab. 1]. The table does not name its metric, and
the values are author-reported. On drug efficacy, the circular fingerprint with
a linear head (1.13) is slightly better than the neural fingerprint with a
linear head (1.15) [arxiv:1509.09292 tab. 1]. The paper's depth search says
"up to 6"; the released code searches depths 1 to 4 [arxiv:1509.09292 §4.2].
Card: [Convolutional Networks on Graphs for Learning Molecular
Fingerprints](/papers/1509.09292).

### CNN over SMILES characters

A one-dimensional CNN runs over the characters of the SMILES string, followed by
global max pooling over positions [arxiv:2004.08919 app. A]. The learned part is
the network, not the input features. Card:
[DeepPurpose](/papers/2004.08919).

### CNN with a bidirectional recurrent layer

The CNN output feeds a bidirectional GRU or LSTM [arxiv:2004.08919 app. A]. The
paper does not state the recurrent layer sizes. Card:
[DeepPurpose](/papers/2004.08919).

### Transformer over substructure tokens

The input is split into substructure tokens rather than characters, and a
Transformer runs over them [arxiv:2004.08919 app. A]. The paper does not state
the Transformer sizes. Card: [DeepPurpose](/papers/2004.08919).

### Pretrained Transformers over SMILES

BOOM evaluates ChemBERTa (83M parameters), MoLFormer (48M), Regression
Transformer (27M) and ModernBERT (111M), all reading SMILES
[arxiv:2505.01912 tab. 3]. Pretraining improved ID RMSE by 31% for ChemBERTa,
35% for MoLFormer and 12% for the Regression Transformer. OOD RMSE did not change
significantly, and binned OOD R² fell by 53% for MoLFormer and 39% for ChemBERTa
[arxiv:2505.01912 §4.2] [fig. 4]. ModernBERT had lower OOD RMSE than the other
Transformers on heat of formation and C_v [arxiv:2505.01912 §4.6] [fig. 8]; the
size of that gain is disputed, see Contradictions. Card:
[BOOM](/papers/2505.01912).

### Message passing on the 2D graph

DeepPurpose's MPNN runs message passing on the molecular graph
[arxiv:2004.08919 app. A]. BOOM's Chemprop and TGNN use atom and bond features
with permutation symmetry [arxiv:2505.01912 tab. 3]. Chemprop's OOD error fell
after a random search over 50 configurations per property, for density (−60%),
heat of formation (−23%) and ZPVE (−50%) [arxiv:2505.01912 §4.3] [fig. 6]. That
search selected each configuration on the OOD test set, so these gains are
optimistic [arxiv:2505.01912 app. C.4]. Cards:
[DeepPurpose](/papers/2004.08919), [BOOM](/papers/2505.01912).

OGB-LSC's GIN and GCN models run on the 2D graph, with a virtual node linked to
every atom in both directions and sum pooling over atoms [arxiv:2103.09430
§2.3]. The GIN with a virtual node has validation MAE 0.1396 eV and test MAE
0.1494 eV, the best single model in its table [arxiv:2103.09430 tab. 6]. The
paper's model-size sweep gives 0.1410 for the same width and depth
[arxiv:2103.09430 tab. 7], so cite one source and say which. The KDD Cup
winners reached about 0.121 on test, but they used ensembles [arxiv:2103.09430
§2] [tab. 6]. Card: [OGB-LSC](/papers/2103.09430).

### Graph Transformer with structural encodings

Graphormer runs a standard Transformer over all nodes of a graph, plus one
virtual graph token whose final state is the readout [arxiv:2106.05234 §3.2].
Three structural terms enter the attention: a learned embedding for node degree,
a learned bias for each shortest-path distance, and a learned bias from the edge
features along the path [arxiv:2106.05234 §3.1] [eq. 6] [eq. 7]. In the
authors' PCQM4M-LSC ablation, validation MAE is 0.2276 with no node relation and
no centrality, and 0.1304 with spatial relation, centrality and the attention
bias [arxiv:2106.05234 tab. 5]. Author-reported single-model results include
MolHIV AUC 80.51 with FLAG, MolPCBA AP 31.39 with FLAG and ZINC test MAE 0.122
for the Slim model [arxiv:2106.05234 tab. 2] [tab. 3] [tab. 4]. Attention cost
grows with the square of the node count, which the authors name as a main
limitation [arxiv:2106.05234 §D] [§6]. Card:
[Do Transformers Really Perform Bad for Graph Representation?](/papers/2106.05234).

### Learnable positional vectors on message passing

LSPE adds a separate positional vector to each node. It starts from random-walk
return probabilities (RWPE) or Laplacian eigenvectors, and the vector is updated
in every layer alongside the node features [arxiv:2110.07875 §3.1] [§3.2]. The
authors apply it to two sparse GNNs, GatedGCN and PNA, and to two fully connected
Transformer GNNs, SAN and GraphiT [arxiv:2110.07875 tab. 1]. With RWPE, GatedGCN's
ZINC test MAE falls from 0.251 to 0.090 with the optional Laplacian loss, and PNA's
from 0.141 to 0.095 [arxiv:2110.07875 tab. 1]. On the two OGB molecular sets the
authors report results comparable to the state of the art, not clear wins:
GatedGCN-LSPE reaches 0.7754 test ROC-AUC on MOLTOX21, against 0.7757 for GIN-VN
[arxiv:2110.07875 §4.2] [tab. 2]. The Laplacian term in the released code is
detached, so the Laplacian-loss rows may not reflect the written loss
[arxiv:2110.07875 §3.1]. Card: [GNN with learnable structural and positional
representations](/papers/2110.07875).

### 3D equivariant and invariant networks

BOOM's 3D models read atom positions or pairwise distances. EGNN and IGNN each
have 217k parameters, MACE has 3.9M and GotenNet has 6M [arxiv:2505.01912 tab. 3].
The hybrid models Graphormer-3D, GeoFormer and Equivariant Transformer also read
3D inputs [arxiv:2505.01912 app. B.5]. GotenNet has the lowest OOD RMSE on 7 of
10 tasks, and MACE on two [arxiv:2505.01912 §4.1] [tab. 7]. The paper's text does
not mention EGNN, which has the lowest OOD RMSE on density [arxiv:2505.01912 tab. 7].
Card: [BOOM](/papers/2505.01912).

### Diffusion over 3D coordinates

GeoDiff models the distribution of 3D atom coordinates given the bond graph.
Sampling starts from Gaussian noise and reverses a diffusion process in many
small steps [arxiv:2203.02923 §4.1] [arxiv:2203.02923 alg. 1]. Its noise
network is built from equivariant layers [arxiv:2203.02923 eq. 5] to
[arxiv:2203.02923 eq. 7]. The prior has no centre-of-mass drift, and the paper
proves that the generated density is then invariant to rotation and translation
if every reverse kernel is equivariant [arxiv:2203.02923 §4.2]. The paper argues
that predicting distances first and solving for coordinates accumulates error,
which direct generation avoids [arxiv:2203.02923 §2].

Sampling needs one network evaluation per step, 5000 steps at the paper's
default [arxiv:2203.02923 tab. 4]. On GEOM-Drugs without force field, GeoDiff-C
reports a mean COV-R of 89.13%, against 67.16% for GeoMol, the best baseline in
that table [arxiv:2203.02923 tab. 1]. The main model is the chain-rule variant
GeoDiff-C, not GeoDiff-A [arxiv:2203.02923 §5.2]. The force-field rows are a
different protocol and should not be compared with the rows without it
[arxiv:2203.02923 tab. 2]. Card: [GeoDiff](/papers/2203.02923).

## When to use which

The sources do not rank encoders for every case. The choices below follow from
each encoder's inputs and from the reported results.

| Situation | Use | Why |
|---|---|---|
| You have a SMILES string and a drug-target binding task | A DeepPurpose compound encoder paired with a protein encoder | In the author-reported DAVIS figure, MPNN with AAC has the lowest MSE of the DeepPurpose rows [arxiv:2004.08919 §3] [fig. 1] D. The pairs also differ in protein encoder, so the compound encoder's effect is not isolated |
| You need a cheap molecular-property baseline | MLP on RDKit descriptors | It trains in about 0.009 s per epoch on the 10k dataset, against 10 s for Chemprop on the same device [arxiv:2505.01912 tab. 4] |
| Atom positions are available and the target may lie outside the training range | Equivariant 3D network such as GotenNet or MACE | GotenNet has the lowest OOD RMSE on 7 of 10 tasks [arxiv:2505.01912 §4.1]; each epoch costs more than for 2D models [arxiv:2505.01912 tab. 4] |
| You only have 2D topology | 2D graph model such as Chemprop | Graph models beat SMILES models on 9 of 10 averaged columns [arxiv:2505.01912 tab. 1]; see Contradictions |
| You must predict a DFT-level property from the 2D graph within a fixed inference budget, with no quantum calculation at inference | GIN with a virtual node | Test MAE 0.1494 eV for one model, against 0.2070 for the MLP on 2048-bit fingerprints [arxiv:2103.09430 tab. 6]; the benchmark allows 0.1 s per molecule [arxiv:2103.09430 §2.3]; the KDD winners' lower scores used ensembles |
| You have a 2D molecular graph, can afford attention cost that grows with the square of the atom count, and want global context | Graphormer-style Transformer with structural biases | Ablation validation MAE fell from 0.2276 to 0.1304 once structural terms were added [arxiv:2106.05234 tab. 5]; the cost is named as a limitation [arxiv:2106.05234 §6] |
| You use a sparse message-passing GNN on molecular graphs and want node positions carried through the layers | LSPE with RWPE initialisation | ZINC gain for GatedGCN, 0.251 to 0.090 with the Laplacian loss [arxiv:2110.07875 tab. 1]; on the OGB sets the authors claim comparable, not better, results [arxiv:2110.07875 §4.2] |
| You need a pretrained SMILES model for in-distribution accuracy | ChemBERTa or MoLFormer | Pretraining cut ID RMSE by 31% and 35%, with no significant OOD gain [arxiv:2505.01912 §4.2] |
| The target is an electronic property (HOMO, LUMO, gap, μ) and OOD reliability matters | Plan extra checks; none of the 15 models was reliable | No model achieved good OOD performance on these four [arxiv:2505.01912 §4.1] |
| You need 3D conformers from a bond graph and can afford thousands of sampling steps | Diffusion over coordinates, as in GeoDiff | Reported coverage and accuracy gains on GEOM-Drugs without force field [arxiv:2203.02923 tab. 1]; sampling costs one network call per step [arxiv:2203.02923 tab. 4] |
| You can afford fewer sampling steps for conformers | GeoDiff with T = 1000 | Authors report COV-R mean 82.96%, against 89.13% at T = 5000; it still beats the baselines in tab. 1 [arxiv:2203.02923 app. D] [tab. 6] |
| You want a fixed feature set and only an MLP to train | Fingerprint with MLP | The fingerprint is computed from the molecule, so the learned part is the MLP [arxiv:2004.08919 app. A] |
| You have labelled molecules, want the fingerprint learned for the task, and can implement a differentiable fingerprint | Learned fingerprint with a one-hidden-layer head | Solubility 0.52 against 1.40 for circular fingerprints with the same head, and photovoltaic 1.43 against 2.00 [arxiv:1509.09292 tab. 1]; on drug efficacy a linear head on circular fingerprints was slightly better [arxiv:1509.09292 tab. 1]. Batch statistics affect the output; see Where the variants differ |

Two cautions apply across rows. First, ID and OOD performance are only moderately
correlated across tasks, so a model's ID score is a weak guide to its OOD score
[arxiv:2505.01912 §4.7]. Pick models on OOD results. Second, BOOM's
Lipophilicity check uses five models, and its split procedure is not stated
[arxiv:2505.01912 tab. 9] [app. C.9]. MoLFormer had the lowest OOD RMSE there
(0.956), and Chemprop had 1.051 [arxiv:2505.01912 tab. 9].

**DeepPurpose reported evidence.** The results are on DAVIS, with values read
from a low-resolution figure, so check them against the PDF. Of the two
DeepPurpose pairs listed for DAVIS, MPNN with AAC has the lower MSE and the
higher concordance index than CNN with CNN [arxiv:2004.08919 §3] [fig. 1] D.
The pairs also differ in protein encoder, so the gap cannot be attributed to the
compound encoder alone [arxiv:2004.08919 §3].

**BOOM LLM check.** Llama 3.1 8B without fine-tuning had very large OOD errors,
for example 262.99 for heat of formation. The authors caution against comparing
these numbers with the main table, because of possible training-data overlap
[arxiv:2505.01912 app. D] [tab. 11].

## Implementation sketch

The common core is a function that maps one compound to a vector. The variants
differ only in the branch taken for the molecule's input.

```pseudocode
# Compound encoder, one call per molecule
# Input:  smiles (str); positions (n, 3) for 3D variants only
# Output: a vector; its length depends on the variant

def encode_compound(smiles, variant, positions=None):
    if variant == "fingerprint":
        # Computed bit vector, then an MLP  [arxiv:2004.08919 app. A]
        bits = fingerprint(smiles)
        return mlp(bits)
    if variant == "descriptors":
        # RDKit descriptor vector, then a random forest or MLP
        # [arxiv:2505.01912 app. B.2]
        return descriptors(smiles)
    if variant == "learned_fingerprint":
        # Differentiable fingerprint, trained with the head  [arxiv:1509.09292 alg. 2]
        return mlp(learned_fingerprint(molecular_graph(smiles)))
    if variant == "cnn":
        # 1D convolutions over SMILES characters, then global max pooling
        # over positions  [arxiv:2004.08919 app. A]
        return global_max_pool(conv1d(embed_chars(smiles)))
    if variant == "cnn_rnn":
        # Same CNN, then a bidirectional GRU or LSTM  [arxiv:2004.08919 app. A]
        return rnn_summary(conv1d(embed_chars(smiles)))
    if variant == "transformer":
        # Input is substructure tokens, not characters  [arxiv:2004.08919 app. A]
        return transformer_summary(embed_tokens(substructure_tokens(smiles)))
    if variant == "smiles_transformer":
        # Tokens are SMILES characters; pretraining is optional
        # [arxiv:2505.01912 app. B.3] [§4.2]
        return transformer_summary(embed_tokens(smiles_tokens(smiles)))
    if variant == "graph":
        # Message passing on the 2D bond graph, then a readout
        # [arxiv:2004.08919 app. A] [arxiv:2505.01912 tab. 3]
        return readout(message_passing(molecular_graph(smiles)))
    if variant == "geometric":
        # Needs atom positions as well as the graph  [arxiv:2505.01912 app. B.4]
        return readout(equivariant_passing(molecular_graph(smiles), positions))
    if variant == "graph_transformer":
        # Attention over all node pairs, plus structural biases  [arxiv:2106.05234 §3.1]
        return readout(transformer(embed_nodes(graph), structural_bias(graph)))
    if variant == "learnable_pe":
        # Positional vector per node, initialised from RWPE, updated each layer
        # [arxiv:2110.07875 §3.1]
        return readout(message_passing(graph, pos=rwpe(graph)))
    raise ValueError(variant)
```

The generative branch does not fit this function. GeoDiff's network returns a
noise estimate inside a sampling loop, and the coordinates are the output, so
it needs its own sampler [arxiv:2203.02923 alg. 1].

### Where the variants differ

- **Input units.** Character CNNs, SMILES Transformers and substructure
  Transformers read different tokens from the same string
  [arxiv:2004.08919 app. A] [arxiv:2505.01912 tab. 3].
- **Output.** Predictive encoders return a vector for a downstream head. A
  generative model returns coordinates from a sampling loop
  [arxiv:2203.02923 alg. 1]. OGB-LSC clamps its predicted gap to [0, 50] eV
  for v1 and [0, 20] eV for v2, and the code applies the clamp only at
  evaluation [arxiv:2103.09430 §2.3] [app. C] [code: pcqm4m/gnn.py#L61-L65].
- **Geometry.** The 3D variants need atom positions or distances. A SMILES string
  alone does not provide them, and this page does not cover how to produce them
  [arxiv:2505.01912 tab. 3]. GeoDiff generates positions, so it does not need
  them as input [arxiv:2203.02923 §3.1]. In the PCQM4M v2 dataset, 3D
  structures exist only for training molecules, so a 3D model cannot use them
  at test time [arxiv:2103.09430 app. C].
- **Readout.** Fingerprints and descriptors need no readout. The MPNN readout is
  named as mean or sum in DeepPurpose; see Contradictions. Graphormer reads out
  from its virtual graph token [arxiv:2106.05234 §3.2].
- **Batch dependence.** The learned fingerprint says batch normalisation was
  used [arxiv:1509.09292 §4.2]. The released code normalises over every atom in
  the input array and keeps no running statistics, so the same molecule can get
  a different fingerprint depending on its batch. The card's Paper vs code
  section records this. Keep batch composition fixed when comparing models, or
  use running statistics and say so.
- **Length caps.** The papers do not state the maximum SMILES or token length
  for every encoder. Check the reference implementation before relying on a cap
  [arxiv:2004.08919 app. A].
- **Paper and code.** GeoDiff's released code differs from its paper in ways
  that matter for reimplementation. The paper's noise network takes the step t
  [arxiv:2203.02923 eq. 4], and its loss has per-step weights [arxiv:2203.02923
  eq. 9]. The released denoiser takes no timestep input, scales its coordinates
  differently, and hard-codes its loss weights, as read from the repository in
  this wiki's card. A time-conditioned reimplementation will not match the
  released weights.

## Contradictions

- **MPNN readout.** DeepPurpose names the readout as "mean/sum"
  [arxiv:2004.08919 app. A]. This page does not resolve which one an
  implementation should use.
- **Descriptor models labelled SMILES.** BOOM's results table labels the random
  forest and MLP as SMILES, while its model summary and appendix say they use RDKit
  descriptors [arxiv:2505.01912 tab. 7] [tab. 3] [app. B.2]. This page follows the
  descriptor description.
- **Graph versus SMILES.** The paper's text says graph models perform comparably
  to Transformer models if the Regression Transformer is discounted
  [arxiv:2505.01912 §4.4]. The averaged table, which includes that model, shows
  graph models with lower OOD RMSE than SMILES models on 9 of 10 columns
  [arxiv:2505.01912 tab. 1]. The Regression Transformer's OOD errors are very
  large [arxiv:2505.01912 tab. 7], so the averages depend on it.
- **ModernBERT gain.** The text says ModernBERT cuts OOD RMSE for heat of
  formation by more than 58% against the best other Transformer
  [arxiv:2505.01912 §4.6]. The table values give about 52%: 44.49 for ModernBERT
  against 93.6 for MoLFormer, the lowest other Transformer on that task
  [arxiv:2505.01912 tab. 7]. For C_v the table values give about 78% against
  ChemBERTa, which agrees with the text. This page reports the table values and
  does not resolve the gap.
- **Cross-paper ZINC comparison.** LSPE reports GatedGCN-LSPE at 0.090 test MAE
  and compares it with 0.122 for Graphormer, claiming a 26.23% gain
  [arxiv:2110.07875 §4.2] [tab. 2]. The Graphormer card reports 0.122 for its
  Slim model [arxiv:2106.05234 tab. 4]. Each paper runs its own training setup,
  so this page does not rank the two.

## Papers

- [Convolutional Networks on Graphs for Learning Molecular Fingerprints](/papers/1509.09292): differentiable circular fingerprints trained end to end, compared with circular fingerprints on solubility, drug efficacy and photovoltaic efficiency.
- [DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction](/papers/2004.08919): an open-source library with fingerprint, CNN, recurrent, Transformer and message-passing compound encoders, and DAVIS and KIBA comparisons.
- [BOOM](/papers/2505.01912): a benchmark of 15 models across descriptor, SMILES, 2D graph and 3D representations, with ID and OOD results on 10 properties.
- [GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generation](/papers/2203.02923): a diffusion model over 3D atom coordinates with equivariant noise networks, evaluated on GEOM-QM9 and GEOM-Drugs.
- [Do Transformers Really Perform Bad for Graph Representation?](/papers/2106.05234): Graphormer, a Transformer over graph nodes with degree, shortest-path and edge-path structural biases, evaluated on PCQM4M-LSC, MolHIV, MolPCBA and ZINC.
- [Graph Neural Networks with Learnable Structural and Positional Representations](/papers/2110.07875): LSPE, learnable per-node positional vectors for GNNs and Transformer GNNs, initialised from RWPE or Laplacian eigenvectors, evaluated on ZINC and two OGB molecular sets.
- [OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs](/papers/2103.09430): three large graph benchmarks, including PCQM4M, a 2D-graph HOMO-LUMO regression with GCN, GIN and MLP-fingerprint baselines and a KDD Cup 2021 summary.