A Survey of Deep Learning Techniques for Weed Detection from Images

In plain words

Farmers pay to control weeds, and a weed often looks much like the crop growing next to it: similar green colour, similar leaves at an early stage. Automatic weed control needs a machine that can tell the two apart in a camera image. This survey reviews how deep learning, a kind of machine learning that learns visual features from labelled pictures, has been used for that job.

The authors read 70 research papers and sorted them by how the images were taken, how the data was prepared, how the model finds the plants, and how results are scored. Most studies learn in a supervised way, and many start from models already trained on other images. Many of the scores are high, but they come from small, site-specific datasets (see Pitfalls).

TL;DR

This is a literature survey, not a new method. It covers 70 deep-learning papers on detecting, localising and classifying weeds in crops, published from 2016 to August 2020 §4 §13. It gives a taxonomy of the steps (data acquisition, dataset preparation, detection and classification, evaluation) and lists 19 public datasets §5 §6.3 tab. 3. It reports no experiments of its own, and its accuracy figures are the cited studies' own reports §13.

Problem

Weeds compete with crops for nutrients, water, light and space, and the usual control methods (preventive, cultural, mechanical, biological, chemical) have costs and health or environmental drawbacks §1. Automating weed control needs a system that first detects and recognises weeds correctly §1. Weed detection is hard because weeds and crops often share colour, texture and shape, and a crop in one setting can be a weed in another §1 fig. 1. Lighting, shadows, occlusion, motion blur, growth stage and geography add further variation §1 fig. 1.

The authors say earlier reviews mostly covered traditional machine learning §1, covered only a handful of publications §2, or looked at selective spraying alone §1. Traditional machine learning depends on hand-built features, which are hard to design when crops and weeds look alike §3.

Method

This is a systematic literature review, not a method with an implementation.

1. Search. Keyword queries combining weed terms (detection, management, classification) with deep-learning terms were run in eight databases, for papers from 2010 to 30 August 2020 §4 tab. 1. After duplicates were removed, 988 documents remained §4.
2. Select. The authors counted how many of these used deep learning per year. The counts rise from 2016, so articles from 2016 onward were kept §4 fig. 3. The plotted counts of DL-based articles are 5 (2016), 16 (2017), 30 (2018), 46 (2019) and 38 (2020, to 30 August) fig. 3.
3. Classify. The 70 papers are placed in one table with the crop, the weed species, the DL architectures and the operations used tab. 2. The taxonomy has four stages: data acquisition, dataset preparation, detection and classification, and evaluation §5 fig. 2 fig. 4.
4. Acquire data. Cameras sit on UAVs, field robots, all-terrain vehicles, tripods or handheld rigs. One study uses satellite imagery §6.1 §6.2. Several UAV studies fly low (about 2 m) so each plant can be labelled, while one compares flight heights of 10, 15 and 20 m §6.1.1.
5. Use public data. The survey lists 19 public datasets, most in RGB and mostly site-specific. It notes that no benchmark like ImageNet or COCO exists for weeds §6.3 tab. 3.
6. Prepare the data. Images are resized, often to a fixed size (the survey lists sizes from 64×64 to 360×360 pixels) or cut into patches when the originals are very large §7.1. Common steps are background removal (for example with NDVI, Otsu thresholding or excess-green indices), denoising, and contrast enhancement. Some studies change the file format or colour model §7.1.
7. Augment and synthesise. Geometric transforms (rotation, flipping, scaling, shearing, cropping) are the most common augmentations. Colour and noise changes appear less often. Synthetic images are rarely used, but some studies generate plants in virtual fields or cut and paste plant parts onto real backgrounds §7.2 tab. 4.
8. Label. Annotation is mostly manual: bounding boxes, pixel-level masks, image-level labels, polygons, or synthetic labels. Some studies use clustering or semi-supervised labelling to cut the manual effort §7.3 tab. 5.
9. Detect. Two approaches dominate. Plant-level classification labels each plant or patch as crop or weed, and is used in most studies. Weed mapping estimates density across a field §8.1 §8.2.
10. Learn. Supervised learning is the norm. The survey covers unsupervised clustering (JULE and DeepCluster) and semi-supervised graph networks §9.1 §9.2 §9.3.
11. Choose an architecture. Pre-trained CNNs (AlexNet, VGG, GoogLeNet, ResNet, Inception, DenseNet) are common, and some studies train a CNN from scratch §10.1.1 §10.1.2. Region-proposal detectors (YOLO-v3, tiny YOLO-v3, Faster R-CNN, SSD, Mask R-CNN) are covered in §10.2. Fully convolutional networks (SegNet, U-Net, DeepLab, FCN) are in §10.3, graph convolutional networks in §10.4, and hybrids in §10.5.
12. Score. The survey lists 23 metrics tab. 6. The most used are classification accuracy, F1 and mean IoU §11.

# Weed detection pipeline as the survey describes it  [fig. 2] [§5]
# Input:  field images from a UAV, field robot, ATV or handheld rig
# Output: a weed or crop label per plant, or a weed density map

def weed_pipeline(images, labels):
    # Resize, remove soil background and reduce noise  [§7.1]
    processed = preprocess(images)

    # Optionally add augmented or synthetic training images  [§7.2]
    training_set = augment_or_synthesise(processed, labels)

    # Pick a classifier, detector, segmenter or graph network  [§8] [§10]
    model = choose_architecture(training_set)

    # Train, mostly supervised and often from pre-trained weights  [§9.1]
    model = fit(model, training_set)

    # Score with the study's metric, e.g. accuracy, F1 or mIoU  [§11]
    return predict(model, processed)

Hyperparameters

Not applicable. The survey trains no model. Settings used by the reviewed studies (input sizes, augmentations, architectures) are summarised in Method and in tab. 4 and tab. 2; the survey does not give a single training recipe §7.1 §10.

Results

Author-reported, as summarised by this survey. The survey runs no experiments of its own, and it does not state that the cited numbers were checked or re-run §13. The setups differ in crop, weed species, sensor, dataset size and metric, so the rows are not directly comparable §11.

Dataset context: 70 papers are tabulated tab. 2 and 19 public datasets are listed tab. 3. The survey states that most of the datasets are site- or crop-specific §6.3.

Paper vs code

No official code. This is a survey: it has no implementation of its own. The links it gives go to other authors' code and are not used here: DeepWeeds and k-FLBPCM §10.1.1, a Google Drive link for the Gao et al. tiny-YOLO work §10.2, and weedNet §10.3. No discrepancies with code apply.

Pitfalls

• Scores are not comparable. Accuracy, F1 and mIoU are reported on different crops, weed species, sensors, dataset sizes and class mixes. The survey itself says it is hard to compare the works §11.
• Most results come from small, specific datasets. The survey says the high accuracies were reached in limited setups, on small datasets with few crops and species §13. Dataset sizes in tab. 3 range from 39 images (Carrot-Weed) to 208,477 (CNU Weed).
• Pre-training is disputed. One study finds ImageNet weights still reduce the number of training iterations for weed classes §10.1.1. Another argues that a CNN initialised with weights never trained on plant images "would not work well" and builds a new network §10.1.2. The survey does not settle this.
• Input size and format matter. One ResNet-101 study reports accuracy rising with input size (128, 224 and 360 pixels) §10.1.1. One study converts PNG to JPEG because it reports that JPEG works better for training residual networks; the survey repeats that claim without testing it §7.
• Class imbalance is common. Ten of the 19 datasets in tab. 3 are marked as imbalanced. The survey recommends cost-sensitive learning and class-balancing classifiers as remedies; it runs no tests of its own §12.
• Labelling is expensive. Most datasets are labelled by hand. Semi-supervised and weakly supervised approaches are proposed as options, not evaluated here §7.3 §12.
• Search cut-off. The search ended on 30 August 2020, so later work is not covered §4.
• Table error. In tab. 3 the row for the Early crop weed dataset has one cell fewer than the others: "508" sits in the sensor column and the remaining cells are shifted left.
• Unstated values. Many rows of tab. 2 say "Not specified" for the weed species or the crop. Read the original study for those details.

Lineage

• Builds on: none. The survey draws on prior reviews, listed in §2: Kamilaris and Prenafeta-Boldú (2018, a survey of 40 DL agriculture papers), Merfield et al. (2016), Amend et al. (2019), Wang et al. (2019), Brown et al. (2005), Fernandez et al. (2018), Moazzam et al. (2019), Su et al. (2020) and Liu et al. (2020) §2. None of these have cards here.
• Compared against: the survey compares its scope with these reviews §2.
• Superseded by: not stated.