AQUA20: A Benchmark Dataset for Underwater Species Classification under Challenging Conditions

In plain words

AQUA20 is a labelled photo set of underwater scenes: 8,171 pictures sorted into 20 classes, such as coral, fish, turtle, shark and a human diver. Many pictures are murky, dim or partly hidden by sand or plants, which is what makes recognition hard. The authors trained 13 common image networks on the set, from small lightweight ones to large transformers, and measured how often each one guessed right. A large convolutional network, ConvNeXt, scored best. They also looked at which parts of a picture the networks relied on, which showed that the best model focused on biologically relevant parts and the weaker one often on the seabed. The dataset is the main contribution; the model scores are a starting point for others to beat.

TL;DR

• AQUA20 is a public set of 8,171 underwater images in 20 classes, with a fixed train/test split of 6,559 and 1,612 images tab. 1 §3.1.
• 13 ImageNet-pretrained models were fine-tuned and compared §3.2 §4.1. ConvNeXt (87,586,964 parameters) is best: top-1 0.9069, top-3 0.9882, macro-F1 0.8892 tab. 3.
• All numbers are author-reported. The released code selects checkpoints by lowest loss on the test split, so the test numbers are not from held-out data (see Pitfalls).
• The paper states 32×32 input; the released notebook uses 224×224. Treat the input size as an open conflict.

Problem

Underwater images suffer from low visibility, colour distortion, low light, turbidity and occlusion, and standard vision systems degrade on them [abstract] §1. Existing underwater datasets cover one scenario or few classes, rarely vary depth, illumination and turbidity, or are not taken in live water: one fish collection uses market photos taken on land §1 §2. Some datasets also have no public test annotations, so results cannot be compared §2. The authors conclude that no widely recognised benchmark covers the diversity of underwater life for classification §2.

Method

Dataset construction §3.1

• Source. Most images come from the EUVP dataset, which was made for underwater image enhancement and is unlabelled. Classes that needed more examples received images from "other publicly available sources", which the paper does not name §3.1.
• Selection. Images were kept if they clearly showed marine animals or plants. They were grouped by content. Categories with fewer than 30 images were dropped §3.1.
• Split. About 80% of images go to training and 20% to testing, and every class has at least 10 test images §3.1 tab. 1. The split procedure and its random seed are not stated.
• Annotation. One initial annotator labelled each image. Three further annotators reviewed each label independently and flagged images with several species, very small, partly hidden, blurry or dark subjects. A joint consensus round followed. Images still in doubt were discarded §3.1. Inter-annotator agreement is not reported.
• Classes. The 20 classes are listed with their biological group and ecological role in tab. 2. Biological groups follow the World Register of Marine Species; roles come from ecology literature tab. 2.
• Challenges. The paper groups difficulty into inter-class similarity fig. 2, intra-class variation fig. 3 and environmental adversities (lighting and background, turbidity, occlusion) fig. 4 §3.1.
• The dataset is hosted at huggingface.co/datasets/taufiktrf/AQUA20 §3.1.

Class counts, copied from tab. 1:

The notebook loads the split from two folders, train and test, and counts the same numbers per class Aqua20_Main.ipynb Aqua20_Main.ipynb. How those folders were built is not in the repository.

Models §3.2

Thirteen architectures, each initialised from ImageNet weights §3.2 §4.1: VGG19, InceptionV3, ResNet50, SqueezeNet1_1, DenseNet121, MobileNetV2, ShuffleNetV2X10, EfficientNetB0, RegNetX32GF, ViT-B/16, SwinV2-B, ConvNeXt and MaxViT. Each model's original classifier is replaced by a new output layer for the 20 classes. The paper does not describe these heads; the notebook's heads are:

• MobileNetV2 and EfficientNetB0: dropout 0.2, then a linear layer from 1,280 features Aqua20_Main.ipynb Aqua20_Main.ipynb.
• ResNet50: linear layer from 2,048 features Aqua20_Main.ipynb.
• DenseNet121: linear layer from 1,024 features Aqua20_Main.ipynb.
• ViT-B/16: linear layer from 768 features Aqua20_Main.ipynb.
• ConvNeXt (torchvision convnext_base): last layer of the classifier replaced, from 1,024 features Aqua20_Main.ipynb.
• MaxViT (torchvision maxvit_t): pooling, LayerNorm(512), linear 512→512, tanh, then a linear 512→20 layer without bias Aqua20_Main.ipynb.
• ShuffleNetV2X10: linear layer from 1,024 features Aqua20_Main.ipynb.
• SqueezeNet1_1: a 1×1 convolution from 512 channels Aqua20_Main.ipynb.
• VGG19: last linear layer from 4,096 features Aqua20_Main.ipynb.
• SwinV2-B: linear layer from 1,024 features Aqua20_Main.ipynb.
• RegNetX32GF: linear layer from 2,520 features Aqua20_Main.ipynb.
• InceptionV3: auxiliary logits disabled, adaptive average pooling set to 1×1, linear layer from 2,048 features Aqua20_Main.ipynb.

Training protocol §4.1

• Framework: PyTorch and TorchVision §4.1.
• Pretraining: ImageNet weights §4.1. The notebook names the torchvision IMAGENET1K_V1 weights for each model Aqua20_Main.ipynb.
• Input: images resized (paper: to 32×32) and normalised with ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225) §4.1 Aqua20_Main.ipynb.
• No data augmentation §4.1. The notebook applies only resize, tensor conversion and normalisation Aqua20_Main.ipynb.
• Optimiser: Adam with a fixed learning rate of 0.001; batch size 32; cross-entropy loss; 100 epochs §4.1 Aqua20_Main.ipynb Aqua20_Main.ipynb.
• Two stages §4.1. Stage 1 trains only the classifier while the rest of the network is frozen, until it converges. Stage 2 fine-tunes the whole model end to end. The paper reports that most models peaked after stage 1; only EfficientNetB0 improved over classifier-only training §4.1.
• Checkpoint: the paper reports "the checkpoints that achieved the highest performance during evaluation" §4.1.
• Hardware: NVIDIA Tesla P100, 16 GB §4.1.

The notebook's training procedure, with the citations for each step:

# Two-stage fine-tuning as in the released notebook  [§4.1]
# Input:  pretrained model, train_loader, test_loader
# Output: model restored to its best checkpoint

def train_two_stage(model, train_loader, test_loader):
    # Stage 1: train only the new classifier head  [§4.1]
    # [code: Aqua20_Main.ipynb#L849-L865]
    freeze_all_but(model, head)
    # Lowest test loss seen so far; not reset between stages  [code: #L315]
    best_loss = infinity
    waited = 0
    stage1_epochs = 0
    # Up to 100 epochs  [code: #L87] [code: #L321]
    for epoch in range(100):
        # Train mode also updates BatchNorm statistics  [code: #L222]
        train_one_epoch(model, train_loader)
        # Loss on the TEST split decides the checkpoint  [code: #L349]
        test_loss = mean_loss(model, test_loader)
        if test_loss < best_loss:
            best_loss = test_loss
            save_checkpoint(model)
            waited = 0
        else:
            waited = waited + 1
        # Patience 10 on test loss  [code: #L88] [code: #L362]
        if waited >= 10:
            stage1_epochs = epoch + 1
            break
    model = load_checkpoint(model)  # [code: #L366]

    # Stage 2: every parameter trainable, same Adam object  [code: #L372-L375]
    unfreeze_all(model)
    waited = 0
    # Rest of the budget: 100 if stage 1 never stopped early  [code: #L377]
    for epoch in range(100 - stage1_epochs):
        train_one_epoch(model, train_loader)
        test_loss = mean_loss(model, test_loader)
        # Must beat the stage-1 best to save anything  [code: #L406-L408]
        if test_loss < best_loss:
            best_loss = test_loss
            save_checkpoint(model)
            waited = 0
        else:
            waited = waited + 1
        if waited >= 10:
            break
    return load_checkpoint(model)  # [code: #L422]

Evaluation metrics §4.2

• The paper defines accuracy, precision, recall, F1 and ROC-AUC with binary TP, TN, FP and FN counts §4.2. It does not say how these are averaged over the 20 classes.
• The headline precision, recall and F1 of ConvNeXt equal the macro average over the 20 per-class values, and the top-1 value equals the overall accuracy tab. 4 convnext_classwise_analysis.csv. The paper does not state this.
• Top-2 and top-3 accuracy are reported in tab. 3 but not defined §4.2.

Explainability §4.5

Grad-CAM and LIME were applied to ConvNeXt (top-1 90.69%) and InceptionV3 (top-1 76.36%) on the same images §4.5 fig. 7. The paper states that Grad-CAM uses gradients from the target class and that LIME perturbs images to fit local decision boundaries §4.5. The layer used for Grad-CAM and the LIME settings are not stated. The analysis is qualitative; no faithfulness metric is reported §4.5.

Hyperparameters

Results

Author-reported. Setup: 8,171 images, 80/20 split, 13 ImageNet-pretrained models, single fine-tuning run per model in the notebook, evaluation on the test split §4.1 tab. 1. The paper's input size is 32×32 §4.1; see Pitfalls.

Full table of model results tab. 3:

These values match the notebook's results file row for row model_metrics_comparison.csv.

Reading of the paper's text §4.3:

• ConvNeXt is best on precision, recall, F1 and top-1/top-3 accuracy. SwinV2-B is second on F1 (0.8429) and top-3 (0.9876). ViT-B/16 reaches top-3 0.9845.
• EfficientNetB0 reaches top-3 0.9541 with about 4 million parameters; MobileNetV2 (top-3 0.9566) and ShuffleNetV2 (top-3 0.9522) are close.
• VGG19 has the most parameters (139 million) and the lowest F1 of all 13 models (0.5172).
• MaxViT reaches top-1 0.8623 at about 30 million parameters.
• The paper says the top-3 accuracy of most models is often above 95% §4.3.

Per-class results for ConvNeXt tab. 4:

The paper's class-wise discussion §4.3 notes that classes with few examples (octopus, marine dolphin, 10 test images each) have lower F1, driven by low recall, and that fish (538 test images) and coral (348) have high F1. The per-class values match the notebook's results file convnext_classwise_analysis.csv.

Errors §4.4: the confusion matrix shows coral as the most frequently misclassified class for ConvNeXt fig. 5. Example confusions in the paper are coral and starfish, turtle and coral, fish and shark, and sea slug and flatworm fig. 6. The paper attributes these to visual overlap such as branching shapes, tentacle-like anemones and mottled patterns, plus poor lighting, motion blur and cluttered backgrounds §4.4.

Explanations §4.5: ConvNeXt's Grad-CAM and LIME maps focus on biologically relevant parts, such as a turtle's head and flippers or a dolphin's midsection. InceptionV3's attention spreads more often onto the seabed fig. 7 §4.5. These observations are qualitative.

Paper vs code

Code: Aqua20_Main.ipynb (one notebook) and results/ CSV files in github.com/taufiktrf/AQUA20 at commit dcd462c8b9e16ecc16f339095ffa1d1b05ae3a4b. Licence: MIT, LICENSE copyright Taufikur Rahman. The repository owner's GitHub display name is the first author's name. The arXiv abstract and comments do not link the code, and the paper links only the dataset. The repository README cites a journal version (Arabian Journal for Science and Engineering, DOI 10.1007/s13369-026-11231-z), which was not compared with arXiv v3. Searches: arXiv metadata and the paper for code links; a GitHub repository search for "AQUA20" returned this repository. The paper's declarations say the code will be released on acceptance [Declarations].

The table values match the released results files (see Results). The following differ:

1. Input size. Paper: 32×32 §4.1. Code: 224×224 for every model Aqua20_Main.ipynb. The results files do not record the input size, so it is unknown which setting produced the reported numbers.
2. Checkpoint selection. Paper: "the checkpoints that achieved the highest performance during evaluation" §4.1. Code: the checkpoint with the lowest loss on the test loader, saved whenever the test loss improves Aqua20_Main.ipynb Aqua20_Main.ipynb. The paper mentions no validation split.
3. Early stopping. Paper: 100 epochs §4.1. Code: 100 is the maximum, with early stopping after 10 epochs without a new best test loss Aqua20_Main.ipynb Aqua20_Main.ipynb.
4. Stage 2 budget. Paper: no per-stage epoch count §4.1. Code: stage 2 runs 100 - stage1_epochs epochs Aqua20_Main.ipynb. The variable is set only when stage 1 stops early Aqua20_Main.ipynb. If stage 1 runs all 100 epochs, stage 2 runs another 100.
5. Best-loss carry-over. The code sets the best loss once, before stage 1, and does not reset it Aqua20_Main.ipynb. Stage 2 saves a checkpoint only after beating the stage-1 minimum Aqua20_Main.ipynb. If it never does, the final model is the stage-1 checkpoint Aqua20_Main.ipynb. The paper does not describe this.
6. Stage 1 is not frozen in BatchNorm. The code freezes parameters but calls model.train() in every training step Aqua20_Main.ipynb, so BatchNorm statistics in the "frozen" backbone still update. The paper says the rest of the network is frozen §4.1.
7. Metrics. Precision, recall, F1, top-k accuracy, ROC-AUC, Grad-CAM and LIME are not computed in the notebook; the per-class and summary results files are the only record model_metrics_comparison.csv convnext_classwise_analysis.csv. The averaging rule (macro) is visible only in those files.
8. Dataset construction. The notebook loads the train and test folders from Kaggle paths Aqua20_Main.ipynb. The image selection, the split and the annotation pipeline described in the paper §3.1 are not in the repository.
9. Inception-v3 input. The code feeds 224×224 images to InceptionV3 and sets aux_logits=False Aqua20_Main.ipynb. The paper does not mention InceptionV3's input size.

Pitfalls

• Test split used for model selection. Checkpoints and early stopping use the test loss Aqua20_Main.ipynb Aqua20_Main.ipynb. The reported test scores are therefore not from data the selection never saw. A held-out validation split would be needed to measure this effect; none is reported.
• Input size conflict. The implementer must pick 32×32 (paper) or 224×224 (code). The notebook is the only executable evidence, so it is the more likely basis of the numbers. Neither setting is verified against the table.
• Small test classes. Eight classes have 10 to 13 test images tab. 1. One extra error in a 10-image class moves that class's recall by 0.1 (arithmetic from the support counts). Macro averages give such classes the same weight as the large ones.
• Macro averages. The headline precision, recall and F1 are unweighted class means tab. 4 convnext_classwise_analysis.csv. Octopus recall of 0.6 pulls the mean down; a weighted average would be higher [same file].
• Top-k definition. Top-2 and top-3 accuracy are not defined. Tie handling is not stated.
• Single seed, no variance. The notebook fixes seed 42 and runs each model once Aqua20_Main.ipynb. The paper reports no repeats or standard deviations. Some gaps in the table are about one point (for example ConvNeXt top-3 0.9882 vs SwinV2-B 0.9876), and such gaps cannot be judged without repeats.
• Logged accuracy is a batch average. The per-epoch accuracy in the logs is the mean of batch accuracies, so the last batch (12 of 1,612 test images) carries the same weight as a full batch of 32 Aqua20_Main.ipynb Aqua20_Main.ipynb. The table values come from the separate results files, whose code is not in the repository.
• Image provenance and licences. Most images come from EUVP, and others from unnamed public sources §3.1. The licences of these images are not stated. The dataset page on Hugging Face (checked 2026-11-01) shows no licence field and no source statement. The code licence (MIT) does not cover the images. Check the licences of the source images before redistributing any of them. This card contains no images.
• Annotation. No inter-annotator agreement is reported §3.1. Discarding doubtful images §3.1 may leave the set easier than the raw collection; the paper does not measure this.
• Explainability. Grad-CAM layer, LIME settings and the number of LIME samples are not stated §4.5. The maps are qualitative; no faithfulness test is reported.
• Weights version. The paper says only "ImageNet". The notebook uses the torchvision IMAGENET1K_V1 weights Aqua20_Main.ipynb. Other weight versions may give different results.

Lineage

• Builds on: the EUVP dataset, which supplied most of the images §3.1; ImageNet pretraining §4.1; the 13 architectures, each cited in §3.2.
• Compared against: other underwater or fish datasets reviewed in §2: Fish4Knowledge, URPC, UDD and DUO, WildFish and WildFish++, Fish-gres, Moorea Labeled Corals, UIEB and UICD. The paper does not run them.
• Superseded by: not stated in the paper.