Precision in Rice Variety Classification using Stacking-Based Ensemble Learning

In plain words

Rice comes in many varieties that look alike, and mixing a cheap type into an expensive one is a known form of fraud. This paper asks whether a computer can tell the varieties apart from a photo of a single grain. The authors photographed grains of 20 varieties up close, and they flipped and rotated each photo to get more examples.

Their method borrows image models that were already trained on large general photo collections, and retrains several of them on the grain photos. Each model then makes its own guess. A final model learns how to combine those guesses into one answer. The authors report that this combination is correct on every image in their own collection, and they also built a phone app that sends a grain photo to a server for a guess.

TL;DR

The paper fine-tunes ten ImageNet CNNs on a self-curated set of 20 rice varieties, keeps the four with the highest validation accuracy, and combines them with a stacked ensemble. The best meta-learner is XGBoost, which the paper reports at 100% on the self-curated set. The base models reach 97.5-99.2% validation accuracy, and on a separate five-variety external set 698 of the 700 Jasmine images are classified correctly. All numbers are author-reported, and the split and leakage details are unclear (see Pitfalls).

Problem

Varieties are hard to tell apart by eye or by simple measurements, and that makes fraud and quality control difficult §1. The authors say earlier studies used small sample sizes or few varieties, and that many did not balance variety count, dataset size and morphology §2 tab. 1. They also note that deep learning is under-used in rice studies §2.

Method

Overview. The pipeline has five parts: build and augment the grain image dataset §4.1; fine-tune several pretrained CNNs §4.2 §4.3; select four of them §5; stack their outputs with a meta-learner §4.4; evaluate with cross-validation and an external set §4.5 §5. The paper also converts the final model for a phone app §5.

1. Dataset. The self-curated set has 20 varieties, each captured grain by grain at 5x magnification. This gives about 4,500 images, 225 per class §4.1. Augmentation with horizontal flips, vertical flips and rotations of 45, 90 and 180 degrees gives 27,000 images, 1,350 per class §4.1 fig. 2. The augmented images are split into training and test sets tab. 2.

2. Base models. Ten ImageNet-pretrained architectures are each trained individually: VGG16, MobileNetV2, EfficientNetV2L, DenseNet201, NASNetLarge, ResNet50V2, Xception, InceptionV3, VGG19 and ResNet152V2 §4.2. Input size is 224 x 224 x 3 fig. 4. Hyperparameters are set in preliminary experiments (see Hyperparameters) §4.3.

3. Selection. The paper prioritises models that exceed a 95% accuracy threshold and names four: EfficientNetV2L, VGG16, MobileNetV2 and DenseNet201 §5 tab. 4. The selection criterion is described only as performance §5; the data it was measured on is not stated.

4. Stacking. The four base models are combined in a stacked ensemble. The base-model outputs are the inputs to a higher-level meta-model, which learns how to combine them §4.4. The paper compares 14 meta-learners: logistic regression, k-nearest neighbours, support vector classifier, decision tree, random forest, AdaBoost, XGBoost, gradient boosting, Gaussian naive Bayes, passive aggressive, ridge, stochastic gradient descent, extra trees and an MLP §4.4 tab. 6. XGBoost is chosen as the final meta-learner §4.2 §4.4.

5. Cross-validation. The paper runs 5-fold cross-validation: the data is split into five folds, four are used for training and one for validation, and each fold serves once as validation. Accuracy, precision, recall and F1 are averaged over the five runs §5 fig. 8.

6. Metrics. Precision, recall, F1 and accuracy are computed from the confusion matrix §4.5. The paper also reports AUC-ROC using one-vs-one pairs of classes §4.5.

7. Deployment. The chosen model is converted to TensorFlow Lite and quantised. It runs on a cloud server behind an API, and an Android app (built in Kotlin) sends captured grain images to that API and displays the returned variety §5 fig. 12 fig. 13.

Equations. The paper's equations are not numbered in the source. The metric formulas and the stacking relation are cited to the section §4.4 §4.5.

Key algorithm: stacked ensemble. Our restatement of §4.4 and §4.2. The paper does not give a listing.

# Stacked ensemble with an XGBoost meta-learner  [§4.2] [§4.4] [fig. 3]
# Input:  train_images, train_labels, test_images
# Output: one variety label per test image

def stacked_predict(train_images, train_labels, test_images):
    # Train one base CNN per selected architecture  [§4.2] [§5]
    base_models = [
        train_base_model(name, train_images, train_labels)
        for name in SELECTED_ARCHITECTURES  # four names, see step 3
    ]
    # Collect each base model's output for every image  [§4.4]
    # Whether these are out-of-fold outputs is not stated.
    meta_train = stack_outputs(base_models, train_images)
    meta_test = stack_outputs(base_models, test_images)
    # Fit the meta-learner on the stacked outputs  [§4.4] [§4.2]
    # XGBoost hyperparameters are not stated.
    meta_model = fit_xgboost(meta_train, train_labels)
    return meta_model.predict(meta_test)

Key algorithm: base-model training. Our restatement of tab. 3 and §4.3.

# Fine-tune one base CNN  [§4.3] [tab. 3] [fig. 4]
# Input: 224 x 224 x 3 images with one of 20 variety labels

def train_base_model(architecture, images, labels):
    # Start from ImageNet weights; whether layers are frozen is not stated
    # [§4.3]
    model = pretrained_model(architecture, num_classes=20)  # [fig. 4]
    # Adam with learning rate 0.0001  [tab. 3]
    optimizer = adam(learning_rate=0.0001)
    # 25 epochs  [tab. 3]
    for epoch in range(25):
        # Batches of 32 images  [tab. 3]
        for batch_images, batch_labels in batches(images, labels, 32):
            # Softmax output, categorical cross-entropy  [tab. 3]
            loss = cross_entropy(model(batch_images), batch_labels)
            update(optimizer, loss)
    # Dropout and L2 differ per architecture; see Hyperparameters  [§4.3]
    return model

Hyperparameters

Values as stated in the paper. Where the paper gives a value only in a figure or gives two different values, both are listed and the conflict is marked.

Results

Author-reported. Setup: the self-curated 20-variety set §4.1; the four base models are evaluated on validation data tab. 4. The paper does not say which split Table 6 is computed on.

Base models, validation tab. 4:

AUC-ROC, one-vs-one, as stated: EfficientNetV2L 0.99999945876, VGG16 0.99999925977, MobileNetV2 0.9999985491, DenseNet201 0.99999825273. The text does not say whether these are micro- or macro-averaged §5 §4.5 fig. 6.

Per-class results of the four base models are in tab. 5. The lowest precision values for DenseNet201 are 0.89 for 2_Bashmoti and 0.90 for 15_Katari_Polao tab. 5. Confusion matrices are in fig. 7.

Stacking, XGBoost meta-learner tab. 6:

Combinations are numbered 1 EfficientNetV2L, 2 VGG16, 3 MobileNetV2, 4 DenseNet201 §5. In the TOP (1,2,3,4) row, Decision Tree, Random Forest and Extra Trees also show 1.000 tab. 6. The paper does not explain why XGBoost is preferred over them.

Cross-validation fig. 9: box plots for the 14 meta-learners over five folds. The text describes MLP, gradient boosting and XGBoost as stable across folds and KNN and AdaBoost as the least stable. No numeric table is given.

External set §5 fig. 10: 3,500 images, 700 per class, five varieties (Arborio, Basmati, Ipsala, Jasmine, Karacadag). Arborio, Basmati, Ipsala and Karacadag are 700 of 700. Jasmine is 698 of 700 (99.71%), with both errors predicted as Arborio. Total: 3,498 of 3,500 (99.94%, our count from fig. 10). The text does not name which model produced these results.

Comparison with prior work tab. 7: the paper lists 100% for its method on 27,000 images and 20 varieties. Prior rows with the highest accuracy include Tasci et al. at 99.87% (5 varieties, 75,000 images) and Saxena et al. at 99.85% (5 varieties, 75,000 images) tab. 7. The rows use different datasets, variety counts and protocols.

Paper vs code

No official code.

Searched: the paper text (no repository or project page link); the arXiv metadata (no comments field with a link); GitHub repository search for "rice variety", "stacking rice ensemble" and "aruzz" (no repository owned by an author); GitHub user search for the author names (no matching account). The self-curated dataset is cited as a Mendeley Data deposit with a DOI in the reference list §4.1 [refs]. We did not check that deposit's licence and do not redistribute it.

Pitfalls

• The 100% headline. The 100% is one number in tab. 6, for XGBoost on the TOP (1,2,3,4) combination, and the table does not say which split it uses. The four base models were picked on their performance, prioritising those above a 95% threshold §5. If the stacked model was evaluated on the same data used for that choice, the result is optimistic. The Results section also calls the figure "100% test accuracy" §5 while Table 4 is labelled validation.
• Split and augmentation order. The text says the augmented data was split 80:20 §4.1. Table 2 gives 22,500 training and 4,500 test images, which is 5:1 per class, not 80:20. The paper does not say whether augmentation happened before or after the split. If before, rotated and flipped copies of one grain can sit in both training and test sets, and the test accuracy would overstate generalisation. The 225-per-class test count equals the pre-augmentation count, which hints at a split on originals, but the paper does not say so.
• Stacking inputs. Section 4.4 says the meta-model takes the base models' predicted outputs §4.4. Section 4.2 and Figure 3 describe stacking "feature sets" §4.2 fig. 3. The implementation needs one of these choices. The paper does not say whether the training-set outputs are out-of-fold, which matters a lot for stacking.
• Dropout and L2 conflict. The text gives dropout 0.3 for all four base models, then 0.2 for MobileNetV2 and 0.4 for DenseNet201 §4.3. Figure 4 labels 0.3 for both fig. 4. The text gives an L2 coefficient of 0.001 §4.3; Figure 4 labels 0.01 for EfficientNetV2L and DenseNet201 fig. 4. An implementer must pick one reading and say which.
• Epoch count. Results peaked near epoch 20, and the authors extended training to 25 to let the model keep improving §4.3. The paper does not say which data was watched to choose this.
• External set provenance. Section 4.1 describes the external set as the public Cinar and Koklu dataset §4.1 tab. 1. Section 5 says it was "collected independently" §5. The 3,500-image subset is "carefully selected" §4.1; the selection rule is not stated.
• Deployment. The text says the model is quantised, but the quantisation method is not stated §5. It also says the app sends each image to a cloud API §5. Whether any inference runs on the phone is not stated.
• Capture conditions. The authors list mixed-grain samples, environmental variability, lighting, camera quality and user error as untested or problematic §4.1 §5 fig. 13.
• Variance. No standard deviations or seeds are reported. Five-fold results appear only as box plots fig. 9.
• Comparison table. Table 7 compares rows that use different varieties, sizes and protocols tab. 7. The "surpassed all prior benchmarks" claim §6 cannot be checked against a common benchmark.
• Tables label. Table 3 is headed "Optimization Space" but gives single values. The ranges that were searched appear only in the prose §4.3.

Lineage

• Builds on: the ten pretrained ImageNet architectures named in §4.2. None are cards in this wiki.
• Compared against: the prior rice studies in tab. 7 and tab. 1. None are cards in this wiki.
• Superseded by: not recorded.