---
kind: concept
slug: stacked-generalization
title: Stacked generalization
summary: >-
  Stacked generalization trains a meta-model on several base models' outputs. A rice-image stack reports 100% on its own set, and a card-fraud stack adds 0.008 AUC over logistic regression. A class-imbalance benchmark finds stacks best on four of 16 sets. All results are author-reported.
tags: [ensembles, stacking, meta-learning, image-classification, fraud-detection, class-imbalance, time-series-classification]
papers: ["2104.07551", "2304.02858", "2608.07471", "2609.10524"]
related: [ensemble-learning, meta-learning, auto-context-model, data-augmentation, time-series-classification]
status: draft
updated_at: 2026-11-01
---

## Overview

Stacked generalization combines several models with a second, trained model.
The base models make their predictions first. A meta-model then takes those
predictions as its inputs and learns how to combine them into one answer
[arxiv:2609.10524 §4.4] [arxiv:2608.07471 §3.2] [arxiv:2304.02858 §4.1]. The
meta-model is trained on the base models' outputs, not on the raw data alone.

The wiki has four cards on this family so far. Three use a trained meta-model:
one on image classification, one on tabular fraud detection and one on
class-imbalance benchmarks that include stacks. The fourth, on time-series
classification, is a close relative. It combines its members with fixed
weights and trains no meta-model [arxiv:2104.07551 §3]. The four use different
base models, data and metrics, so their numbers are not comparable.

In the three trained-meta-model papers, the paper does not state whether the
meta-model is trained on out-of-fold base outputs. The image paper also does
not say which split its headline figure uses [arxiv:2609.10524 §4.4]
[arxiv:2609.10524 tab. 6]. Those are the details that decide whether a stacked
result can be trusted, so an implementation has to choose and state them itself.
The fixed-weight paper avoids the question differently: its weights come from
accuracy estimates built on separate bagged fits or out-of-bag samples, not from
the training fit itself [arxiv:2104.07551 §3] [arxiv:2104.07551 §6].

## Variants

### Stacked CNNs with an XGBoost meta-learner

Ten ImageNet-pretrained CNNs are fine-tuned on the same 20-class image set. The
four best are kept: EfficientNetV2L, VGG16, MobileNetV2 and DenseNet201
[arxiv:2609.10524 §4.2] [arxiv:2609.10524 §5]. Their outputs feed a meta-model.
The authors compare 14 meta-learners and choose XGBoost [arxiv:2609.10524 §4.4]
[arxiv:2609.10524 tab. 6]. Card:
[Precision in Rice Variety Classification using Stacking-Based Ensemble Learning](/papers/2609.10524).

Reported result: with the top four base models and XGBoost, accuracy is 1.000
on the authors' own set [arxiv:2609.10524 tab. 6]. The base models reach
0.9747 to 0.9924 validation accuracy [arxiv:2609.10524 tab. 4]. On a separate
external set, the paper reports 698 of 700 correct for one of five varieties,
and the text does not name the model behind that result
[arxiv:2609.10524 §5].

Decision tree, random forest and extra trees also reach 1.000 on the same
combination [arxiv:2609.10524 tab. 6]. The paper does not explain why XGBoost
is preferred over them.

### Stacked classifiers on imbalanced tabular data

Seven standard classifiers are trained on the Kaggle card-fraud data:
random forest, k-nearest neighbours, logistic regression, SGD, decision tree,
naive Bayes and SVM [arxiv:2608.07471 §3.1]. The training split is balanced by
random undersampling, and only the Time and Amount columns are standardized
[arxiv:2608.07471 §3.4]. Card:
[Application of Artificial Intelligence for Fraudulent Banking Operations Recognition](/papers/2608.07471).

The stacked variant trains K weak classifiers, each on a random subset of size
B, and combines their outputs with a meta-model [arxiv:2608.07471 §3.2]
[arxiv:2608.07471 eq. 2]. The paper does not give K, B, the base classifiers or
the meta-model, so this variant cannot be rebuilt from the paper alone.

Reported result: the stacked ensemble reaches AUC 0.954, against 0.946 for the
best single model, logistic regression [arxiv:2608.07471 §5]
[arxiv:2608.07471 fig. 5]. Its F1 of 0.96 is the highest the paper reports
[arxiv:2608.07471 §4.2] [arxiv:2608.07471 fig. 6]. The other single models
score from 0.908 to 0.938 AUC [arxiv:2608.07471 §4.2] [arxiv:2608.07471 fig. 4].
The authors conclude that all seven models "more or less equally cope"
[arxiv:2608.07471 §5]. The gap is about 0.008 AUC, from one run on one dataset,
with no variance reported.

### Stacking-I and Stacking-II in a resampling benchmark

The review pairs nine resampling methods with nine ensemble models, two of
them stacks, on 23 binary and four multi-class data sets
[arxiv:2304.02858 §4.1] [arxiv:2304.02858 §5.1]. Stacking-I uses decision
trees, random forests, k-nearest neighbours and XGBoost as base models.
Stacking-II uses random forests and XGBoost. Both use logistic regression as the
meta-learner [arxiv:2304.02858 §4.1] [arxiv:2304.02858 fig. 2]. Card:
[Khan et al., ensemble and augmentation benchmark](/papers/2304.02858).

Reported result: in the paper's per-set summary, Stacking-I is the best pair on
four of the 16 binary sets listed. Those are Ecoli-0vs1, Ecoli-3, Glass-5vs12
and Yeast-2vs8, paired with CT-GAN or SVM-SMOTE [arxiv:2304.02858 tab. 8].
Stacking-II is not the best pair on any listed set, and it is the worst pair
on three of them [arxiv:2304.02858 tab. 8]. Each pair is run 30 times on a
60:40 split, and results are reported as means [arxiv:2304.02858 §5.1]
[arxiv:2304.02858 §5.3].

The text says Stacking-II beats Stacking-I on AUC heatmaps for six data sets
[arxiv:2304.02858 §5.3]. This is a different summary from the best-pair table,
and the review does not reconcile the two.

### Accuracy-weighted meta-ensemble, without a meta-model

HIVE-COTE 2.0 trains four time-series classifiers on the same training set: the
Shapelet Transform Classifier, the Temporal Dictionary Ensemble, the Diverse
Representation Canonical Interval Forest and Arsenal [arxiv:2104.07551 §3].
Each one gives class probabilities for a new series, plus an estimate of its own
accuracy from the training data. The probabilities are combined with weights
taken from those estimates, raised to the power 4 [arxiv:2104.07551 §3]. No
second model is trained. The paper does not write the combination out as a
formula; the form on this page is our reading of §3. Card:
[HIVE-COTE 2.0: a new meta ensemble for time series classification](/papers/2104.07551).

Reported result: on the 112 univariate UCR problems, the full system is
significantly more accurate than its four components and than the four
benchmark methods [arxiv:2104.07551 §5.1] [arxiv:2104.07551 fig. 6]. Its own
ensembling comparison puts the accuracy-weighted average, called CAWPE, ahead
of the alternatives it tested. These include choosing the best member on the
training estimates, forward selection, ensemble selection and random-forest
stacking [arxiv:2104.07551 §6.1] [arxiv:2104.07551 fig. 18]. This is the
paper's own comparison, on its own benchmark.

The cost is compute. A sequential build of one UCR resample takes about 340
hours, against about 2.9 hours for ROCKET [arxiv:2104.07551 tab. 4]. A time
contract caps training at some cost in accuracy [arxiv:2104.07551 §7].

## When to use which

The evidence is four papers on four different tasks, so each row is tied to
the setting it comes from.

| Situation | Use | Why |
|---|---|---|
| Several strong base models of similar accuracy on one labelled task | Stacking with a trained meta-model, as in the image card [arxiv:2609.10524 §4.4] | The paper combines four base models that each score above 97% validation accuracy [arxiv:2609.10524 tab. 4] |
| Several diverse base models, and you want no second training stage to tune | A fixed accuracy-weighted average, as in HIVE-COTE 2.0 [arxiv:2104.07551 §3] | The paper reports this weighting beat random-forest stacking and other combiners on its benchmark [arxiv:2104.07551 §6.1] [arxiv:2104.07551 fig. 18] |
| Imbalanced binary labels on tabular data, and a strong single model already exists | Compare the stack with that model on held-out data before adopting it | The fraud stack beats logistic regression by about 0.008 AUC, and the authors call the models roughly equal [arxiv:2608.07471 §5] [arxiv:2608.07471 fig. 5] |
| Imbalanced binary labels with a resampler in the pipeline | Include a stack as one candidate, and compare it with LightGBM under the same resampler | In the review's summary table, LightGBM is the best pair on nine of 16 sets and a stack on four [arxiv:2304.02858 tab. 8]. The stacks' AUC values are on a different scale from the other rows (see Contradictions) |
| The stack must be trusted, not just scored | Evaluate on data that neither the base-model choice nor the meta-model choice used | The 1.000 figure does not say which split it comes from, and the base models were picked by their results [arxiv:2609.10524 §5] [arxiv:2609.10524 tab. 6]. The fraud paper's workflow may also select on the test split [arxiv:2608.07471 fig. 3] |
| Weights must come from data the members did not fit on | Estimate weights from separate bagged builds or out-of-bag samples | HIVE-COTE 2.0's train-based estimates sit a mean difference of +0.47 points from observed test accuracy [arxiv:2104.07551 §6] [arxiv:2104.07551 tab. 6] |
| Compute or time is a hard limit | Measure the build time before choosing a stack or a full ensemble | HIVE-COTE 2.0 needs about 340 sequential hours per UCR resample, and a time contract trades some accuracy for a cap [arxiv:2104.07551 tab. 4] [arxiv:2104.07551 §7]. The image and fraud papers report no training or inference time for their stacks [arxiv:2609.10524 §5] [arxiv:2608.07471 §4.2] |
| Several meta-learners tie on the same row | Pick the cheaper one and test it on held-out data | Trees and XGBoost tie at 1.000 in the image paper's table [arxiv:2609.10524 tab. 6]. This is our suggestion, not the paper's finding |
| Small labelled image data set | Not covered by these cards | The image paper's data set has 27,000 augmented images, and no card gives a stacked result for a smaller image set [arxiv:2609.10524 §4.1] |

## Implementation sketch

The core is the same for any base models and meta-model. Only the base
architectures and the meta-learner change. The open choice is where the
meta-model's training inputs come from. Whether they are out-of-fold is the
decision the three trained-meta-model papers leave unstated.

The fraud paper's subset variant differs in one step. Each base classifier is
trained on its own random subset, rather than on the full training set
[arxiv:2608.07471 §3.2] [arxiv:2608.07471 eq. 2]. The subset count and size
must be chosen by the implementer.

The review resamples the training rows inside each run, before fitting, and
leaves the test rows alone [arxiv:2304.02858 §5.1].

HIVE-COTE 2.0 uses the same loop with one change: the meta-model step is
replaced by a weighted average. Each base model's weight is its accuracy
estimate from the training data, raised to the power 4. The estimate comes from
a separate bagged build, or from out-of-bag samples, rather than from one fit on
all rows [arxiv:2104.07551 §3] [arxiv:2104.07551 §6].

```pseudocode
# Stacked generalization: a meta-model learns to combine base models
# Input:  train_x, train_y, test_x
# Output: predicted label for each test row

def stacked_fit_predict(train_x, train_y, test_x, base_specs, meta_spec):
    # Optional for imbalanced labels: resample the training rows only [arxiv:2304.02858 §5.1]
    train_x, train_y = resample(train_x, train_y)
    # Fit each base model on the training data [arxiv:2609.10524 §4.2]
    base_models = [fit(spec, train_x, train_y) for spec in base_specs]
    # Turn each base model's outputs into meta-features [arxiv:2609.10524 §4.4]
    # Out-of-fold or in-sample training outputs: not stated in the three trained-meta-model papers
    meta_train = stack_outputs(base_models, train_x)
    meta_test = stack_outputs(base_models, test_x)
    # Fit the meta-model on the stacked outputs [arxiv:2609.10524 §4.4]
    # The review's meta_spec is logistic regression [arxiv:2304.02858 §4.1]
    meta_model = fit(meta_spec, meta_train, train_y)
    return meta_model.predict(meta_test)
```

## Contradictions

The four papers do not disagree on a shared result. Their numbers come from
different tasks and metrics, so they cannot be compared. Each card also has
internal inconsistencies, listed below.

Image card [arxiv:2609.10524]:

- **Stacking inputs.** The text describes the meta-model as taking predicted
  outputs [arxiv:2609.10524 §4.4]. Section 4.2 and fig. 3 describe stacking
  "feature sets" [arxiv:2609.10524 §4.2] [arxiv:2609.10524 fig. 3].
- **Cross-validation for the meta-model.** The text says the meta-model is
  trained "using cross-validation", and gives no folds for it
  [arxiv:2609.10524 §4.4].
- **Split behind the headline figure.** The text calls the 1.000 figure
  "100% test accuracy", while table 4 is labelled validation
  [arxiv:2609.10524 §5] [arxiv:2609.10524 tab. 4]. Table 6 does not say which
  split it uses [arxiv:2609.10524 tab. 6].

Fraud card [arxiv:2608.07471]:

- **Balancing.** Section 3.4 says undersampling was used "as the dataset was not
  imbalanced" [arxiv:2608.07471 §3.4]. Section 4.1 calls the same data "highly
  unbalanced" [arxiv:2608.07471 §4.1].
- **Selection split.** The workflow selects the model "with the best evaluation
  metric on the testing data" [arxiv:2608.07471 fig. 3]. Section 4.2 describes a
  separate validation split [arxiv:2608.07471 §4.2].
- **Gain size.** The conclusion calls the 0.008 difference "an AUC of 0.008"
  [arxiv:2608.07471 §5]. It is the gap between 0.954 and 0.946, not an AUC.

Review card [arxiv:2304.02858]:

- **AUC.** The paper defines AUC with a single-threshold formula
  [arxiv:2304.02858 eq. 21]. The card records that the authors' code applies
  its ROC-AUC function to hard predictions, which gives balanced accuracy. The
  stacked "AUC" values in its tables are therefore not threshold-based AUCs.
  See [the card's Paper vs code section](/papers/2304.02858).
- **Stacking-II versus Stacking-I.** The AUC heatmaps favour Stacking-II
  [arxiv:2304.02858 §5.3], while the best-pair table lists Stacking-I four times
  and Stacking-II never [arxiv:2304.02858 tab. 8].
- **Split size.** The split is not stratified. On Yeast-2vs8, a 40 % test set
  holds only about eight minority rows [arxiv:2304.02858 tab. 2], so a single
  error moves minority recall a lot. The paper does not test whether this
  explains its high standard deviations.

HIVE-COTE 2.0 card [arxiv:2104.07551]:

- **Combination rule.** The paper describes the tilt and the accuracy weights
  in words and gives no formula [arxiv:2104.07551 §3]. The form in the
  implementation sketch is our reading.
- **Voting in the code.** The text says Arsenal classifies new cases by majority
  vote [arxiv:2104.07551 §3.3]. The card records that the authors' code weights
  each Arsenal member by its ridge cross-validation score, raised to the power 4
  [arxiv:2104.07551 §3.3]. The paper does not name the voting scheme used to
  combine the four modules.
- **Fixed weights against a learned stack.** The paper reports that its weighted
  average beats random-forest stacking on its benchmark [arxiv:2104.07551 §6.1].
  The three trained-meta-model cards above do not test a fixed-weight baseline,
  so that result is not checked in their settings.

## Papers

- [Precision in Rice Variety Classification using Stacking-Based Ensemble Learning](/papers/2609.10524): stacks four fine-tuned CNNs under 14 meta-learners, with XGBoost chosen as the meta-learner. Author-reported, and the split details are unclear.
- [Application of Artificial Intelligence for Fraudulent Banking Operations Recognition](/papers/2608.07471): stacks seven classifiers' outputs on card-fraud data and reports AUC 0.954 against 0.946 for logistic regression. Author-reported, on one dataset, with the stacking details not stated.
- [Khan et al., ensemble and augmentation benchmark](/papers/2304.02858): pairs two stacks (Stacking-I and Stacking-II, logistic regression as meta-learner) with resampling methods on 27 imbalanced sets. Author-reported, with an AUC that the code computes from hard labels.
- [HIVE-COTE 2.0: a new meta ensemble for time series classification](/papers/2104.07551): combines four time-series ensembles with accuracy-based weights and no meta-model. Compared against random-forest stacking on 112 UCR problems. Author-reported, and about 340 sequential hours per UCR resample.