Application of Artificial Intelligence for Fraudulent Banking Operations Recognition

In plain words

Banks need to spot card or online payments that are fraud. This paper takes a public dataset of past card transactions, where each one is labelled as genuine or fraudulent, and trains ordinary machine-learning classifiers on it. Frauds are very rare in that data, so the authors remove many normal transactions from the training part to balance the classes, and they rescale two of the columns.

Seven common classifiers are then compared on how well they rank fraud above genuine transactions. The best single model is logistic regression. Combining several models in a stack scores a little higher. The differences between models are small, and the authors say they all cope about equally well.

The paper is a practical comparison rather than a new algorithm. It is useful as a worked example of the workflow, and as a warning that the reported differences are small and rest on one dataset.

TL;DR

• Seven standard classifiers are trained on the Kaggle "Credit Card Fraud Detection" dataset and ranked by ROC AUC §4.1 §4.2.
• Training data are balanced by random undersampling; only the Time and Amount columns are standardized §3.4.
• Logistic regression is the best single model at AUC 0.946; a stacked ensemble reaches AUC 0.954 fig. 5 §5.
• All results are author-reported, on one dataset, with no official code and several unstated settings (see Pitfalls).

Problem

Fraudulent banking operations cause financial losses, and online and charity-related fraud grew during the COVID-19 pandemic §1. The authors want an automated classifier that labels a transaction as genuine or fraudulent from historical labelled transactions §1 §3.

The difficulty named by the paper is class imbalance: fraud is a tiny fraction of transactions, so a model trained on the raw data can learn to predict "genuine" nearly always §4.1 §3.4. The paper also notes that the AUC metric ignores unequal costs of false positives and false negatives, and may not suit every use case §3.3.

Method

The paper's workflow is a fixed pipeline with a set of candidate classifiers fig. 3 §3:

1. Load the labelled dataset fig. 3 §4.1.
2. Split it into training, validation and test sets, 70 %, 15 % and 15 % §4.2.
3. Standardize the features that are not already encrypted, which the paper says are Time and Amount §3.4.
4. Randomly undersample the majority class in the training set until the classes are balanced §3.4 fig. 3.
5. For each candidate model, fit it with hyperparameter tuning and cross-validation fig. 3 §4.2.
6. Score each model by ROC AUC and record the result §3.3 §4.2.
7. Output the model with the best score fig. 3 §4.2.

Candidate models are random forest, k-nearest neighbours, logistic regression, stochastic gradient descent classifier, decision tree, naive Bayes and support vector machine §3.1. The paper's Table 2 is meant to list their pros and cons, but its content is not present in the PDF (see Pitfalls).

Standardization is defined as centring each variable on its mean and dividing by its standard deviation §3.4 eq. 5.

The paper also describes stacked generalization: train K weak classifiers on random subsets of size B of the data, then combine their outputs with a meta-model m §3.2 eq. 2. The paper does not say which base classifiers, how many subsets, or which meta-model were used, so this part cannot be reimplemented from the paper alone.

# Fraud-detection pipeline  [fig. 3]  [§3.4]  [§4.2]
# Input:  labelled transactions (label 1 = fraud)
# Output: best candidate model, ranked by AUC

def run_pipeline(transactions):
    # 1. Load the labelled transactions  [fig. 3]
    frame = load_dataset(transactions)
    # 2. Split 70 % train, 15 % validation, 15 % test  [§4.2]
    train, validation, test = split_70_15_15(frame)
    # 3. Standardize Time and Amount only  [§3.4] [fig. 3]
    train, validation, test = standardize_time_amount(
        train, validation, test
    )
    # 4. Randomly undersample the majority class in training  [§3.4]
    train = random_undersample(train)
    scores = {}
    for model in candidate_models:
        # 5. Fit with grid search and cross-validation  [fig. 3] [§4.2]
        fitted = grid_search_cv(model, train)
        # 6. Score by ROC AUC  [§3.3] [§4.2]
        scores[model] = roc_auc(fitted, test)
    # 7. Keep the model with the highest AUC  [fig. 3] [§4.2]
    return best_by(scores)

# Stacked generalization  [§3.2] [eq. 2]
# Input:  training features and labels; K base classifiers
# Output: a meta-model over the base classifiers' outputs

def stack(features, labels, base_classifiers):
    # 1. Draw K random subsets of size B from the data  [eq. 2]
    subsets = draw_random_subsets(features, labels, K, B)
    # 2. Train one weak classifier per subset  [§3.2]
    weak = [
        fit(classifier, subset)
        for classifier, subset in zip(base_classifiers, subsets)
    ]
    # 3. Combine the weak outputs with a meta-model m  [§3.2]
    meta = fit_meta_model(weak, features, labels)
    return meta

Metrics: ROC curve from true positive rate against false positive rate across thresholds; AUC as the area under it §3.3 eq. 3 eq. 4. F1 is also used to compare models §4.2 fig. 6.

Hyperparameters

Results

Author-reported. Setup: one Kaggle "Credit Card Fraud Detection" dataset of 284,807 transactions over two days, 492 of them fraudulent §4.1; Python with scikit-learn, pandas and matplotlib on Kaggle notebooks §4.2; no hardware or run time stated. The baseline for stacking is the best single model.

The F1 values for individual models are read from the bar chart fig. 6 and are approximate. The paper also says its stacked results were compared with state-of-the-art Kaggle notebooks §4.2, but gives no numbers for those.

Paper vs code

No official code. The paper links only Kaggle dataset and notebook pages §4.1 §4.2 §5. Those notebooks are third-party, and the paper does not claim them as its own. We searched for a project page: none is linked. We searched GitHub for "fraudulent banking recognition": seven repositories, none owned by the authors or their institutions. We searched GitHub users for "Mytnyk Shakhovska": none. No code field is recorded.

Pitfalls

• Balancing contradicts the dataset description. Section 3.4 says undersampling was used "as the dataset was not imbalanced" §3.4, while Section 4.1 says the same dataset is "highly unbalanced" with 492 frauds in 284,807 transactions §4.1. The method only makes sense for the imbalanced case described in 4.1.
• Model selection may use the test split. The workflow says to select "the model with the best evaluation metric on the testing data" fig. 3, while Section 4.2 describes a separate validation split §4.2. If selection uses the test split, the reported AUC values are optimistic. Which split was used for tuning is not stated.
• Stacking cannot be reproduced from the paper. Number of subsets K, subset size B, base classifiers, meta-model, and the data the meta-model is trained on are all not stated §3.2. The paper itself notes the meta-attributes differ between training and test sets §3.2.
• Stacking gain is small and inconsistently reported. The conclusion says the stacked classifier improves on the best single model "with an AUC of 0.008" §5. That number is the difference 0.954 − 0.946, not an AUC [abstract].
• Standardization formula is printed incorrectly. The PDF prints the equation as X′ = X − μσ, with no division eq. 5. The usual definition is (X − μ) / σ, which the text describes in words §3.4. Use the usual form; it is not confirmed by the paper's code, which is absent.
• Fitting scope of the scaler is not stated. Whether the mean and standard deviation come from the training split only is not said §3.4 fig. 3. Fitting them on all data would leak test information.
• Random undersampling discards data. Each run keeps only as many majority-class rows as there are fraud rows (equal counts). This discards most genuine transactions, and the paper does not report any run-to-run variance.
• Differences between models are small. The AUC spread is about 0.05 (0.908 to 0.954), and the authors conclude that all seven models "more or less equally cope" §5. A single run with no confidence intervals does not show that one model is better.
• Single dataset, no variance reported. One dataset from European cardholders §3.4. Most features are encrypted with principal components, and only Time and Amount are open §3.4. The paper's own limitations note the sample size and online-banking-only scope §1.
• Dataset license not stated. The paper gives Kaggle links §4.1 but no license for the dataset. This card does not verify it. Do not redistribute the data.
• Table 1 and Table 2 are empty in the PDF. Both are rendered as blank grids. Their content is not available to this card.
• Figure 4 panel labels are hard to read at print resolution. The AUC values are taken from the text list §4.2 instead.
• Metric definitions in the equations are garbled in the PDF. The math glyphs for the true-positive rate, false-positive rate and recall are partly broken eq. 3 eq. 4. The meanings are given in words: recall is TP / (TP + FN), and FPR is the share of true negatives that are classified as positive. Check the TeX source before quoting the formulas.

Lineage

• Builds on: the Kaggle dataset and the standard classifiers, with no earlier paper named as a base §3 §4.1.
• Compared against: the seven classifiers listed above, and the single-model baseline for stacking §4.2. Related work cited includes genetic-algorithm feature selection [13], CatBoost with a deep network [16] and an LSTM-based ensemble [17] §2. None of these is on the wiki, so they are not linked.
• Superseded by: not known.