DeepPurpose: A Deep Learning Library for Drug-Target Interaction Prediction

In plain words

Predicting whether a drug molecule will bind a given protein is a core step in drug discovery. Many deep learning models exist for this, but each research group usually publishes its own code, with its own input format and its own training script. That makes them hard to run, compare or combine.

DeepPurpose is a software library that puts many of these models behind one interface. A molecule and a protein are each turned into a list of numbers by an "encoder" that the user picks by name. The two lists are joined and passed to a small prediction network, which outputs either a strength-of-binding number or a yes/no probability. The library handles data splitting, training, saving, and a ranked list of candidate molecules for a given protein.

The authors present it as a way for people with limited deep learning or biochemistry background to train and use these models without writing much code.

TL;DR

• A Python library that trains encoder-decoder models for drug-target interaction (DTI) prediction; the user chooses the compound and protein encoders by name.
• Compound encoders: fingerprints with an MLP, CNN, CNN+RNN, Transformer and MPNN. Protein encoders: composition descriptors with an MLP, CNN, CNN+RNN and Transformer §2.1 app. A.
• Author-reported: on the DAVIS and KIBA benchmarks, several encoder pairs match or beat KronRLS, GraphDTA and DeepDTA on MSE and concordance index fig. 1.
• Most training settings (optimizer, learning rate, epochs, batch size, dropout, split ratio) are not stated in the paper; the code defaults are listed below.

Problem

Drug-target interaction (DTI) prediction estimates how strongly a compound binds a protein target, or whether it binds at all §1. Deep learning models for this exist, but they are hard to use: each has a different programming interface and input format, and running them needs programming and biochemistry skills §1. The paper argues this blocks comparison and ensembling across tools §1.

Method

DeepPurpose is an encoder-decoder framework §2. The user names a compound encoder and a protein encoder. The compound input is a SMILES string and the protein input is an amino-acid sequence §2.1.

Encoders §2.1 app. A:

1. Compound: Morgan, PubChem, Daylight and RDKit 2D fingerprints, each with an MLP; a 1D CNN over SMILES characters with global max pooling; a bidirectional GRU or LSTM on top of that CNN; a Transformer over substructure tokens; and a message-passing neural network (MPNN) on the molecular graph app. A.
2. Protein: AAC, PseAAC, Conjoint Triad and Quasi-sequence-order descriptors, each with an MLP; a 1D CNN over residues with global max pooling; a bidirectional RNN on top of that CNN; and a Transformer over substructure tokens app. A.
3. The paper notes that not every feature works with every architecture. Some inputs are incompatible with some architectures, and the supported pairings are those already used in the literature app. A.

Decoder §2.2: the compound and protein embeddings are concatenated and fed to an MLP. The library picks the task type from the labels. For continuous labels it uses MSE loss eq. 1. For binary labels it uses binary cross-entropy eq. 2. Metrics are MSE, concordance index and Pearson correlation for regression, and AUROC, AUPRC and F1 at threshold 0.5 for classification §2.2 app. A.

# DeepPurpose forward pass  [§2] [§2.2] [app. A]
# Input:  smiles (str), sequence (str)
# Output: binding score, or logit for binary tasks

def predict(smiles, sequence):
    # Encode the compound with the chosen compound encoder  [§2.1] [app. A]
    compound_vector = compound_encoder(smiles)
    # Encode the protein with the chosen protein encoder  [§2.1] [app. A]
    protein_vector = protein_encoder(sequence)
    # Concatenate both embeddings  [§2.2]
    joint = concatenate(compound_vector, protein_vector)
    # Predict with the MLP decoder  [§2.2] [app. A]
    return decoder(joint)

# Decoder and loss  [§2.2] [app. A] [eq. 1] [eq. 2]
# Regression: mean squared error on the raw score.
# Binary: binary cross-entropy on sigmoid(score).

def decoder(joint):
    # Hidden layers use ReLU and dropout  [code: DeepPurpose/DTI.py#L30-L58]
    for layer in hidden_layers:
        joint = dropout(relu(layer(joint)), rate=0.1)
    # Output layer is linear, one value per pair
    # [code: DeepPurpose/DTI.py#L47-L58]
    return output_layer(joint)

def loss(score, label, binary):
    if binary:
        # Binary cross-entropy  [eq. 2]
        probability = sigmoid(score)
        return binary_cross_entropy(probability, label)
    # Mean squared error  [eq. 1]
    return mean((label - score) ** 2)

Data and training. The library splits data into train, validation and test sets; the example in figure 1C uses a random 70/10/20 split with seed 1 fig. 1. Setting an encoder name configures the matching model automatically §2.1. Models can be saved, reloaded and used for repurposing and virtual screening §2.3. The repurposing and screening functions train five encoder pairs and rank compounds by predicted binding score §2.3. Training with a validation set keeps the epoch with the best validation score and tests that model DTI.py.

Hyperparameters

The paper gives no training hyperparameters in the main text, and its comparison script is in the Supplementary Information, which is not in the LaTeX source. The rows below are either stated in the paper or are defaults read from the code. Code defaults are not the paper's settings unless the paper says so.

Results

Author-reported. Setup: DAVIS and KIBA DTI benchmarks §3; baselines are KronRLS, GraphDTA and DeepDTA §3; metrics are MSE and concordance index, with a parenthesised value printed next to each mean fig. 1 D. The text does not say what the parenthesised value is (standard deviation, standard error or other), and the comparison split and hyperparameters are in the Supplementary Information, which we do not have.

The text says "many DeepPurpose models achieve comparable prediction performance" §3. Figure 1D lists the values below. They were read from a low-resolution raster of the figure; verify them against the PDF before setting reviewed.

KIBA values are in fig. 1 D. We did not transcribe them, because the raster did not give us a reliable reading of every digit.

The paper also says the library's web interface gives predictions "with less than 1-second latency" §4. No measurement is given, so the latency is not stated.

Paper vs code

Code: github.com/kexinhuang12345/DeepPurpose at commit 866be98bad0b11e78d4363589b9ee59b5befa307 (the paper links this repository §2.4). The owner is an author, and the paper links it.

• Repurposing ensemble. The paper says the repurposing and screening functions "train five deep learning models", and that without a training dataset "DeepPurpose uses a pre-trained deep model" §2.3. The code uses five encoder pairs (MPNN+CNN, CNN+CNN, Morgan+CNN, Morgan+AAC, Daylight+AAC), and with no training data it loads pretrained weights for all five oneliner.py. The code comment still says "load 6 pretrained model" oneliner.py.
• MPNN readout. The paper says the MPNN readout is "mean/sum" app. A. The code uses only the mean over atoms encoders.py.
• Task type. The paper says the library detects regression or classification "by counting the number of unique labels" app. A. The code checks for exactly two unique labels in the training split DTI.py. A regression dataset with exactly two distinct values would be trained as classification.
• Loss and metrics. The paper's equations for MSE eq. 1 and BCE eq. 2 match the code's losses DTI.py. The reported metrics match DTI.py.
• Feature sizes. Morgan 1024 bits, PubChem 881, Daylight 2048, RDKit 200 and AAC 8,420 match between the paper and the code utils.py utils.py.
• Paper's experiments. The paper refers to a complete comparison script in its Supplementary Information §3. That file is not in the LaTeX source. The closest demo in the repository, DEMO/DeepDTA_Reproduce_KIBA.ipynb, uses learning rate 0.001 and 100 epochs DeepDTA_Reproduce_KIBA.ipynb. It reproduces DeepDTA, not the DeepPurpose entries, so we did not use it as the paper's setup.

Pitfalls

• Hyperparameters are not in the paper. Optimizer, learning rate, epochs, batch size, dropout and split for the reported results are not stated. The code defaults (learning rate 1e-4, 10 epochs, batch 256) differ from the figure 1C example (learning rate 1e-3, 100 epochs). An implementer must pick one or ask the authors.
• Silent zero features. If RDKit or a protein descriptor fails on an input, the featurizer prints a message and returns a zero vector. The model then trains on it without an error utils.py utils.py. Check for all-zero vectors before training.
• Silent truncation and padding. CNN and CNN+RNN encoders cut SMILES to 100 characters and protein sequences to 1000 residues. Transformer encoders cut drug tokens to 50 and protein tokens to 545 utils.py utils.py. Long inputs lose their tail, which the paper does not mention.
• Unknown tokens. Unknown SMILES or protein tokens map to index 0 in the Transformer encoders utils.py.
• Random RNN initial state. CNN+RNN encoders start the RNN from a random normal hidden state, not zeros encoders.py. Outputs therefore depend on the random number state, not only on the input. Fix the seed and check for reproducibility if you need deterministic predictions.
• Validation split seed. The validation subset is always drawn with random state 1, whatever the fold seed is utils.py. Changing random_seed changes the test split and the train pool but not the validation sampling seed.
• Model selection. The best epoch is chosen on the validation set, and the test set is scored with that model DTI.py. Without a validation set the last epoch is kept DTI.py.
• Costly encodings. The code prints timing for protein features: about 24 s for roughly 100 sequences for AAC, and about 462 s for PseAAC utils.py. Budget for this when the dataset has many unique sequences.
• Ambiguous parenthesised values. Figure 1D prints a value in parentheses next to each mean, without defining it fig. 1 D.
• Pretrained models. Repurposing with no training data depends on downloaded pretrained weights oneliner.py. The paper does not describe those weights.
• Abstract claims. "State-of-the-art" is the authors' claim based on the comparison in figure 1D only. "Fewer than ten code lines" is illustrated by figure 1C and is not measured fig. 1 C.

Lineage

• Compared against: KronRLS, GraphDTA and DeepDTA §3. Named in the text, not on this wiki.
• Uses Gradio to build the web interface §4. Named in the text, not on this wiki.