Comparing YOLOv11 and YOLOv8 for instance segmentation of occluded and non-occluded immature green fruits in complex orchard environment

In plain words

This paper asks which of two versions of a popular fast object-detection family, YOLO11 and YOLOv8, is better at outlining each unripe apple in a photo of an orchard. Each apple gets its own outline, which is called instance segmentation. The photos came from a camera on a farm robot. Every visible fruit was labelled as either fully visible or partly hidden by leaves.

Both families were trained the same way in five sizes each. The models were then compared on accuracy, model size, compute, training time and speed. The medium YOLO11 model was the most accurate overall. The smallest YOLOv8 model was the fastest at running inference.

TL;DR

• A comparison study, not a new method: 10 trained models (YOLO11-seg and YOLOv8-seg, sizes n, s, m, l, x) on one dataset of 991 orchard images §3.2 tab. 1.
• Best overall mAP@50 is YOLO11m-seg (box 0.876, mask 0.860); YOLOv8n-seg has the lowest inference time, 3.3 ms §4.1 §4.4.
• Results are author-reported, from a single run per model, with no variance and no code. Several numbers in the text contradict the tables (see Pitfalls).

Problem

Green apples blend into foliage, and partly hidden fruit is hard to segment. The authors want to know which YOLO11 or YOLOv8 configuration gives the best accuracy and speed for segmenting immature fruitlets, split into occluded and non-occluded classes, in a commercial orchard §1. They also compare model complexity (parameters, GFLOPs, convolution layers) and training time §3.5 §4.2 §4.3.

Method

This is an empirical protocol, not a new architecture.

Data. Images were captured in a commercial 'Scifresh' apple orchard in Prosser, Washington, USA, during the immature-fruitlet stage. The camera was a Microsoft Azure Kinect DK RGB sensor on a UR5e arm, mounted on a Warthog ground robot §3.1 fig. 4. A total of 991 images were manually annotated in two classes: non-occluded (fully visible) and occluded (hidden by foliage or other elements) §3.2. Annotation was done in Roboflow §3.2. The data was formatted for YOLO11 and YOLOv8 and split 8:1:1 into training, testing and validation sets §3.2 fig. 4.

Training. All models were trained with the same configuration: 300 epochs maximum, input size 640×640, batch 8, deterministic mode with seed 0, horizontal flip 0.5, no rotation, IoU threshold 0.7, learning rate 0.01 and weight decay 0.0005. Each model started from pre-trained weights (which ones is not stated) §3.3. Hardware is one workstation with an Intel Xeon W-2155 CPU, an NVIDIA TITAN Xp GPU and 31.1 GiB of memory §3.3.

Evaluation. Box and mask precision, recall and mAP@50 are reported for the All, non-occluded and occluded classes §3.4 tab. 1 tab. 2. Mean IoU (MIoU), precision, recall and F1 are defined eq. 1–eq. 4, but MIoU and F1 are not reported in any table. Parameters, GFLOPs and convolution layer count are reported per model §3.5 eq. 5–eq. 7. Training time and epochs come from the training logs §4.3 fig. 9. Inference, preprocess and postprocess times are in milliseconds tab. 3.

Pseudocode of the protocol, written from the text (not from code):

# Experimental protocol  [§3.2] [§3.3] [§3.4]
# Input:  991 annotated RGB images, 2 classes (occluded, non-occluded)
# Output: box and mask precision, recall, mAP@50 for each model

def run_comparison(images, labels, sizes):
    # Split 8:1:1 into train, test and validation  [§3.2]
    train, test, val = split_ratio(images, labels, (8, 1, 1))
    results = {}
    # Same recipe for every YOLO11-seg and YOLOv8-seg size  [§3.3]
    for family in ["YOLO11-seg", "YOLOv8-seg"]:
        for size in sizes:
            model = load_pretrained(family, size)  # weights not stated
            model.train(
                data=train,
                epochs=300,              # maximum; early stop occurred
                imgsz=640,
                batch=8,
                deterministic=True,
                seed=0,
                fliplr=0.5,
                degrees=0.0,
                iou=0.7,
                lr0=0.01,
                weight_decay=0.0005,
            )
            # Score boxes and masks at IoU 0.5  [§3.4] [tab. 2]
            # Which split is scored is not stated
            results[(family, size)] = model.validate(val)
    return results

Figures are described here and cited only; no images are used on this page.

Hyperparameters

Taken from the paper's text. The paper gives no table of training settings.

Results

Author-reported. Setup: 991 images, 2 classes, 10 models, one workstation §3.2 §3.3. There is no external baseline; the comparison is YOLO11-seg against YOLOv8-seg.

Headline numbers from the text, checked against the tables:

Model size and compute §4.2 fig. 8:

Training time §4.3 fig. 9:

Other sizes are not given in the text. fig. 9 shows them.

Table 1: precision and recall, per class tab. 1. Bold marks the best model in each column, as in the source.

Precision:

Recall:

Table 2: mAP@50 tab. 2. Bold marks the best value in each row pair, as in the source.

Table 3: processing time per image, in ms tab. 3. Measurement setup (batch size, image count, warm-up, hardware) is not stated.

Paper vs code

No official code. The paper links only Ultralytics (YOLOv5, YOLOv8, YOLO11 docs and repositories), Roboflow and Hugging Face §1 §7. Those are third-party upstreams, not the authors' code. I searched for the authors' own code in three ways: the paper text and LaTeX source for links, the GitHub repository search for "immature green fruit yolo11", "YOLO11 orchard" and "sapkota yolo11 yolov8 orchard", and a web search. Nothing matched an author-owned repository.

Inferred, not verified in code: the hyperparameter names used in the text (deterministic, seed, fliplr, degrees, iou, lr0, weight decay) match the training-argument names of the Ultralytics framework. The paper does not say which version it used.

Pitfalls

• Abstract and §4.1 credit YOLO11n-seg with the best mask precision (0.831). Table 1 gives 0.831 to YOLO11l-seg (bold). YOLO11n-seg's mask precision (All) is 0.795. Use the table.
• §4.1 says YOLO11l-seg has the highest occluded mAP@50 (0.81). Table 2 gives the best occluded mask mAP@50 to YOLO11m-seg (0.810), with YOLO11l-seg at 0.809. For occluded box mAP@50 there is a tie at 0.847 (YOLO11l-seg and YOLOv8l-seg).
• Training-time claims contradict the numbers. §4.3 calls YOLO11x-seg the "shortest" run (242 epochs over 6.046 h), while YOLO11l-seg took 3.005 h. The conclusion says YOLOv8l-seg has the longest training time (5.103 h), but §4.3 gives YOLOv8x-seg 5.619 h.
• Models did not train for the same length. The same recipe was applied to all, but runs stopped at 172 to 294 epochs. The stopping rule (early-stopping patience) is not stated. Comparisons of training time are confounded by this.
• Speed ranking depends on the stage. The text ranks speed by inference only. Postprocess for YOLO11n-seg is 56.7 ms, against 2.5 ms for YOLOv8n-seg. Summing the three columns (my arithmetic, not in the paper) gives about 61.8 ms for YOLO11n-seg and 6.1 ms for YOLOv8n-seg. Batch size, warm-up and hardware for the timing runs are not stated.
• GFLOPs definition is wrong in the text. §3.5 defines GFLOPs as operations per image divided by 10^9 (eq. 6), but §3.5 also glosses it as "Giga Floating Point Operations per Second". GFLOPs counts operations, not a rate.
• Defined metrics not reported. MIoU eq. 1 and F1 eq. 4 are defined, but no table reports them. mAP@50-95 is not reported. The AP computation (interpolation, how the "All" class is averaged) is not specified.
• Evaluation split not stated. The text says the data was split into train, test and validation, but not which split the tables use.
• No variance. One training run per model and no seeds beyond the fixed seed 0. Gaps of about 0.01 (for example 0.876 vs 0.875) cannot be separated from run-to-run noise on this evidence.
• Pre-trained weights and optimizer are not stated. The weights source (for example, a public COCO checkpoint) and the optimizer and schedule both change results, and the paper does not name them.
• IoU 0.7 role is ambiguous. The text presents it as a mask-accuracy threshold. The paper does not say whether it applies to matching, non-maximum suppression, or evaluation.
• Dataset details missing. Instance counts, class balance, per-split counts, capture resolution, annotation type (polygon or box) and dataset availability are all not stated. The paper calls the orchard 'Scifresh' apple, but the Figure 5 caption says "Scilate orchard".
• Single orchard, single variety, single camera. Generalization claims in §5 go beyond what was tested.
• Causal explanation is untested. §4.5 attributes YOLO11's better handling of foliage to architecture and training changes. The evidence is one qualitative example (fig. 10). No ablation is given.
• Background claim in §2.2 is unverified. The text says YOLO11m reaches 95.0% mAP on COCO with 22% fewer parameters than YOLOv8m. This number is not sourced here and is out of line with typical COCO mAP values, so it is likely a typo. Do not use it.
• Figure and equation numbers. Figure numbers here follow the order of figure environments in the LaTeX source (Figure 1 is the YOLOv8 architecture, Figure 10 is the false-positive example). Equation numbers follow the order of the equation environments (eq. 1 is MIoU). Neither was checked against the compiled PDF.

Lineage

• Builds on: YOLOv8 and YOLO11 from Ultralytics. The paper's earlier study by the same authors compared YOLOv8 with Mask R-CNN in orchards §1; it has no card here.
• Compared against: only the other family's sizes (YOLO11-seg vs YOLOv8-seg). No external baseline.
• Future work: YOLO26 and YOLO27 on the same dataset and protocol §5.