Synthetic minority oversampling (SMOTE)

Overview

SMOTE is an oversampling method that makes a rare class more common by adding synthetic examples. It creates each new minority point by interpolating between a minority instance and nearby minority instances in feature space 2108.00071 §4.2.2. The exact interpolation details are left to the original SMOTE paper, so this wiki does not state them.

Two cards use SMOTE on tabular classification data, in different roles. One applies it once before a gradient-boosted model 2310.00505 §3.4. The other compares it with other resampling methods under logistic regression 2108.00071 §4. Neither states an invariant the other shares, so no shared invariant is listed.

Variants

SMOTE as one step before LightGBM

The card is Unveiling the Unborn: Advancing Fetal Health Classification through Machine Learning. The pipeline takes 22 numeric features from cardiotocography records, applies SMOTE to balance the classes and fits a LightGBM classifier 2310.00505 §3.2 2310.00505 §3.3 2310.00505 §3.4. The model settings are listed on the card 2310.00505 §3.4. The SMOTE parameters (neighbour count, sampling strategy) and the point in the pipeline where SMOTE runs are not stated 2310.00505 §3.4 2310.00505 §4.4. The paper reports no run without SMOTE, so its effect on the reported scores is not measured 2310.00505 §5.

SMOTE compared with other resamplers under logistic regression

The card is Foundations of data imbalance and solutions for a data democracy. It is a tutorial chapter that fits logistic regression on the Porto Seguro Safe Driver Prediction data (about 595k rows, 22 of 59 columns kept) 2108.00071 §3. It compares SMOTE with random oversampling, ADASYN, random undersampling, Tomek links and edited nearest neighbours, against a baseline with no resampling 2108.00071 §4. The SMOTE run uses five neighbours and seed 42 2108.00071 §4.2.2.

Author-reported, single run 2108.00071 §4:

• No resampling: accuracy 0.96, recall 0, AUC 0.62. The baseline finds no positive cases.
• SMOTE: accuracy 0.58, precision 0.57, recall 0.59, F1 0.58, AUC 0.62.
• Random oversampling: accuracy 0.59, recall 0.60, AUC 0.63.
• ADASYN: accuracy 0.58, recall 0.59, AUC 0.62.

The chapter concludes that oversampling methods do better than undersampling here 2108.00071 §5. The gaps among the three oversamplers are 0.01 in accuracy or AUC, and the chapter does not test them 2108.00071 §4.

When to use which

Implementation sketch

The sketch uses the split-first order. The first card does not state its order, so that step is our recommendation. Steps the papers leave open are marked.

# Split first, then oversample the training part only
# Order in arxiv:2310.00505 not stated  [arxiv:2310.00505 §3.4]
# Order in arxiv:2108.00071 appears to be the reverse  [arxiv:2108.00071 §4]
def fit_with_smote(features, labels):
    # 80/20 split as in the first card  [arxiv:2310.00505 §3.4]
    # Shuffling and stratification not stated
    train_x, train_y, test_x, test_y = split(features, labels,
                                             test_size=0.2)

    # Five neighbours and seed 42 as in the second card  [arxiv:2108.00071 §4.2.2]
    # The first card does not state these  [arxiv:2310.00505 §3.4]
    res_x, res_y = smote(train_x, train_y, k_neighbors=5,
                         random_state=42)

    # LightGBM settings as stated in the first card  [arxiv:2310.00505 §3.4]
    model = LGBMClassifier(learning_rate=0.1, n_estimators=100,
                           num_leaves=31, random_state=123)
    model.fit(res_x, res_y)
    return model, test_x, test_y

Contradictions

• Accuracy values in the first card. The card reports 98.31% in the abstract and 98.32% in §5 2310.00505 §5. Its model comparison gives 97.91% for LightGBM 2310.00505 §5. The paper does not say which split or run produces each figure, so these scores cannot be attributed to SMOTE on their own.
• SMOTE's effect on accuracy across the two cards. The first card reports about 98% accuracy with SMOTE 2310.00505 §5. The second reports accuracy falling from 0.96 to 0.58 when SMOTE is used 2108.00071 §4. The datasets, models, splits and metrics differ, and the first card has no run without SMOTE. The two results are not comparable, so neither confirms or refutes the other.
• Precision and recall in the second card. The card's own confusion matrices do not match the reported precision and recall in several tables. For the SMOTE table, the reported values appear swapped relative to the matrix, and the card attributes this to the argument order in the chapter's code 2108.00071 §4. Check these before quoting the SMOTE row.
• Resampling order in the second card. The chapter's code appears to call resampling on the full data before the split. The card infers this from the size of the test sets, and the chapter does not state the order 2108.00071 §4. If so, the reported SMOTE scores may be inflated 2108.00071 §4.

Papers

• Foundations of data imbalance and solutions for a data democracy: tutorial comparison of SMOTE, random oversampling, ADASYN and undersampling with logistic regression on one dataset, one run per method.
• Unveiling the Unborn: Advancing Fetal Health Classification through Machine Learning: applies SMOTE to balance the classes before a LightGBM classifier on 22 cardiotocography features; no run without SMOTE is reported.