Class Imbalance Problem in Data Mining Review

In plain words

Many real data sets have one class that is far rarer than the others, such as fraud, disease or intrusion cases. A standard classifier learns to favour the common class, so it can score well overall while missing the rare cases that matter most.

This short review lists the known fixes and groups them into three families. One family changes the data, by copying rare examples or dropping common ones. A second changes the learning method, for example by making some mistakes cost more or by training an ensemble that focuses on the rare class. The third keeps only the most useful input features. The authors say each family has trade-offs and that combining methods often helps. They do not run new experiments, so the paper is a map of the field rather than a benchmark.

TL;DR

• A review, not a method. It groups class-imbalance remedies into sampling, algorithmic and feature-selection approaches §2.
• Sampling: random under-sampling discards majority samples, random over-sampling copies minority samples, and SMOTE generates synthetic minority samples §2.1 fig. 1 fig. 2.
• Algorithmic: one-class learning, cost-sensitive learning with a cost matrix, SVM variants and boosting §2.2 tab. 1.
• Feature selection: score features, keep j of them, and use it together with the other two families §2.3.
• No experiments and no numbers of its own. The comparison in tab. 2 is qualitative.

Problem

In a skewed data set one class has many more samples than the other §1. Most classifiers are biased toward the majority class and can predict only the majority class while ignoring the minority class §1. The paper argues that missing a rare event can be far more costly than a false alarm, citing medical diagnosis as an example §1. It also states that most machine learning algorithms optimise overall accuracy, which causes minority misclassification §1.

The paper gives no formal definition of the imbalance ratio and no threshold at which imbalance becomes a problem §1.

Method

This is a taxonomy, not a method. The paper organises the remedies as follows.

1. Sampling (data pre-processing) §2.1.

1. Under-sampling removes majority samples. The paper names random under-sampling as the most important form and notes that it loses information §2.1 fig. 1.
2. Over-sampling replicates minority samples. The paper notes that this adds no new information §2.1 fig. 2.
3. Synthetic over-sampling generates new minority samples. SMOTE is named as the example, and the paper says that learning takes more time because the original minority set is small §2.1.
4. The paper also mentions combining over- and under-sampling §2.

# Random under-sampling  [§2.1] [fig. 1]
# Input:  samples (N, d), labels (N,), majority class m
# Output: a subset with the majority class reduced to the minority size

def random_under_sample(samples, labels, majority, minority_size):
    # Indices of majority samples, which are the only ones dropped  [§2.1]
    majority_idx = indices_where(labels == majority)
    # Keep a random subset of them, of the minority count  [§2.1]
    kept_majority = random_choice(majority_idx, minority_size)
    # Keep every minority sample  [§2.1]
    minority_idx = indices_where(labels != majority)
    return samples[concat(kept_majority, minority_idx)]

Random over-sampling is the mirror image: it replicates minority samples at random until the classes are balanced §2.1 fig. 2. The paper does not give a sampling rate or stopping rule for either method.

2. Algorithmic approaches §2.2.

1. One-class learning models one class and rejects the rest. The paper says it can outperform other methods on multi-dimensional data under certain conditions, citing one reference §2.2.
2. Cost-sensitive learning assigns a cost λ_ij to each kind of error, with zero on the diagonal, as in the cost matrix tab. 1 §2.2. The goal is to pick the class with the minimum conditional risk §2.2. The paper says the costs are usually unknown and must be estimated from the data §2.2. It does not write the risk formula.
3. Kernel and SVM methods map the data to a higher-dimensional space. The paper says they may be combined with over-sampling or with an ensemble §2.2.
4. Boosting can be combined with SVMs in the presence of imbalance §2.2.

3. Feature selection §2.3.

1. Select a subset of j features, where j is a user-defined parameter §2.3.
2. For high-dimensional data, use filters that score each feature independently with a rule §2.3.
3. The paper says feature selection matters because imbalance often comes with high dimensionality, and that its effect depends on the problem §2.3 §5.

4. Methods surveyed in related work §3. Section 3 summarises earlier papers and gives no new algorithm.

1. Random under-sampling (RUS) removes majority examples until the desired class distribution is reached §3.
2. RUSBoost combines RUS with boosting. The paper says it is simpler and faster than SMOTEBoost §3 §4.
3. SMOTEBoost combines SMOTE with AdaBoost §3 §4.
4. AdaBoost.NC combines negative correlation learning with boosting. The paper says it targets multi-class imbalance §3.
5. Infinitely imbalanced logistic regression (IILR) builds the imbalance into the logistic regression formulation §3.
6. Cost-sensitive neural networks use sampling and threshold-moving §3.
7. Boosting SVM and an incremental LPSVM for imbalanced data streams are also described §3.

Hyperparameters

The paper has no method with tuned parameters. The table lists the values the text states.

Results

The paper reports no experiments and no numerical results of its own. The only comparison is the qualitative table below, which the authors summarise from the cited methods tab. 2 §4. The paper says the best technique depends on the nature of the data used §4.

The table's reference numbers do not match the text for some rows; see Pitfalls.

Paper vs code

No official code.

The paper is a review and describes no implementation. Its arXiv metadata has no code link. No GitHub repository is recorded for it, so no search was run. No third-party implementations were used.

Pitfalls

• Author list. The raw arXiv metadata lists two authors. The PDF lists three: Rushi Longadge, Snehalata S. Dongre and Latesh Malik [PDF p. 1]. The frontmatter keeps the metadata list. The attribution line in the body names all three. Check the metadata before publishing.
• Title. The metadata title is "Class Imbalance Problem in Data Mining Review". The PDF title is "Class Imbalance Problem in Data Mining: Review" [PDF p. 1].
• Reference numbers in the text do not match the reference list. Section 2.3 cites "Ertekin [17]" for feature-selection metrics on Yahoo Web data, but reference [17] is titled "Active Learning in Imbalanced Data Classification" §2.3 [PDF p. 5]. Section 3 cites "Cost-sensitive neural networks ... [8]", but reference [8] is about mine classification, while the cost-sensitive neural network paper is reference [13] §3.
• Table 2 contradicts the text. Section 3 says AdaBoost.NC works "without losing the overall performance" §3. Table 2 lists "Ignore overall performance of classifier" as its disadvantage tab. 2. Table 2 cites IILR as [6], but the text cites it as [8] §3.
• Unclear attributions. Several claims use a bare author first name, for example "Chris [2]", "Lei [7]" and "Wang et al. [2]". Reference [2] is RUSBoost by Seiffert et al., so the other attributions cannot be checked from the list without the original papers §3.
• Vague claims. "Under certain condition ... one-class learning gives better performance" names no condition §2.2. "Threshold-moving is a good choice ... effective on all the data sets" is an unsupported general claim §3. Treat both as the authors' opinion.
• Oversampling cost. The paper says over-sampling adds no information and that SMOTE makes learning slower §2.1. It gives no timings.
• Feature selection. The paper writes that feature selection is essential for high-dimensional imbalance problems §2.3. It gives no evidence for this and no selection criterion beyond per-feature filter scores.
• Ensemble claims. The paper says RUS "decreases the time required to construct a model" §3. This is stated without numbers.
• Missing cost formula. The cost-sensitive decision rule is described in words only §2.2. Implement it from a standard source, not from this card alone.

Lineage

• Builds on: the works listed in §3 and the reference list, including RUSBoost (Seiffert et al., 2010), AdaBoost.NC (Wang and Yao, 2012) and SMOTE (named in §2.1). None of these arXiv ids are in this wiki, so none are linked.
• Compared against: the five methods in tab. 2, compared qualitatively only.
• Superseded by: not stated.