RDD2022: A multi-national image dataset for automatic Road Damage Detection

In plain words

Road agencies want to find cracks and potholes automatically from photos taken on the road. A detector can learn this, but it needs many labelled photos. The earlier sets in this series came from one country or three, and a detector trained on one of them did much worse on roads elsewhere.

This paper releases such a set. The authors collected road photos in six countries, drew boxes around four kinds of damage in each photo, and wrote down how the photos and labels were made. The photos came from phones on cars, high-resolution cameras on a van, street-view imagery, a camera on a motorbike and a drone. The labels are in the common Pascal VOC XML format.

This is a dataset paper. It does not propose a detection model, and it reports no detection scores. Its value is the data and the record of how it was made.

TL;DR

• Six countries, four damage classes (D00, D10, D20, D40), training set with 55,007 labels tab. 2 fig. 4 fig. 5.
• Images come from phones, a stitched two-camera van, street-view imagery, a motorbike camera and a drone, at several resolutions p. 8 tab. 2.
• Labels are hand-drawn bounding boxes in Pascal VOC XML, made with LabelImg and, for Norway, CVAT p. 10 fig. 13.
• Only the training images have labels; test images are released without labels and are scored through the CRDDC'2022 leaderboard p. 3 p. 11.
• All numbers are author-reported dataset statistics, not model results.

Problem

Automatic road damage detection needs labelled images, and earlier public sets covered few places. Models trained on the Japan-only RDD2019 data degraded significantly on roads outside Japan p. 1. They report that adding data from other countries improved how well models generalise p. 3. The aim here is a multi-country set for training and benchmarking detectors on roads in six countries p. 3 p. 14.

The paper gives no minimum accuracy target and no cost model. It says the data suit low-cost monitoring by municipalities and road agencies p. 1.

Method

This is a data-collection and annotation procedure, not a learning method. The procedure has two steps: image acquisition, then damage annotation p. 8.

Image acquisition, by country p. 8 fig. 7 fig. 8 fig. 9 fig. 10.

• India, Japan, Czech Republic: a smartphone mounted on a car, sometimes on the windshield inside the car p. 8 fig. 7. Japan and the Czech Republic are 600x600 pixels. India was captured at 960x720 and resized to 720x720 to match them p. 8.
• Norway: a specialised vehicle with two Basler_Ace2040gc cameras behind the windshield. Their images are stitched into one wide view of 3650x2044 p. 8 fig. 8 fig. 9.
• United States: Google Street View images, vehicle-based, at 640x640 p. 8.
• China, motorbike (China_MotorBike, or China_M): a camera on a motorbike moving at about 30 km/h p. 8.
• China, drone (China_Drone, or China_D): a six-motor DJI M600 Pro with a three-axis gimbal that rotates the camera through 360 degrees p. 8 fig. 10.
• All China images are 512x512 p. 8.
• Roads: asphalt concrete pavement, with a few exceptions p. 6. The authors say rigid pavement is not covered p. 15.

Annotation p. 10 fig. 11 fig. 12.

1. Every recognised damage instance is enclosed in a bounding box p. 10.
2. Each box gets a class label from the four damage classes p. 10.
3. The annotation pipeline is the same as for RDD2020 p. 10.
4. LabelImg was used for all countries except Norway; CVAT was used for Norway p. 10 tab. 2.
5. Each image gets one XML file with the class name and four box coordinates (xmin, ymin, xmax, ymax), in a format similar to Pascal VOC p. 10 fig. 13.

# Read the damage boxes of one annotated image  [p. 10] [fig. 13]
# Input:  xml_path, the annotation file of one image
# Output: (width, height) and a list of (class name, box) pairs

def read_annotation(xml_path):
    # Image width and height from the size element  [fig. 13]
    width, height = read_size(xml_path)
    pairs = []
    # One object element per annotated damage instance  [p. 10]
    for obj in read_objects(xml_path):
        # Class name, one of D00, D10, D20 or D40  [p. 10] [fig. 5]
        name = obj.name
        # Four coordinates as listed in the file  [p. 10] [fig. 13]
        box = (obj.xmin, obj.ymin, obj.xmax, obj.ymax)
        pairs.append((name, box))
    return (width, height), pairs

Folder layout and splits p. 11 fig. 14.

• Seven country folders: China_Drone, China_MotorBike, Czech, India, Japan, Norway, United_States p. 11.
• Each folder has a train subfolder with .jpg images and .xml annotations. Each folder also has a test subfolder with images only, except China_Drone, which has no test set p. 11 fig. 4.
• Test annotations are withheld, and the authors point users to the CRDDC'2022 leaderboards to score predictions p. 11.

Damage classes p. 1 p. 3 fig. 3 fig. 5.

• D00: longitudinal crack. D10: transverse crack. D20: alligator crack. D40: pothole p. 1 fig. 3.
• The classes differ from the RDD2018 definitions. D40 there covered rutting, bump, pothole and separation tab. 1. See Pitfalls.

Hyperparameters

The paper trains no model, so there are no learning hyperparameters. The table gives the dataset parameters an implementer needs.

Results

Author-reported dataset statistics. No model is trained or evaluated in this paper. Counts are per country in fig. 4 and per class in fig. 5.

The per-class totals are the sums of the per-country counts in fig. 5. We checked that the sums match the RDD2022 row of fig. 2.

For comparison with earlier releases tab. 2:

Paper vs code

The linked repository is sekilab/RoadDamageDetector at the commit above. The paper links it for data access p. 11. The owner is the authors' lab; one of the paper's authors is Yoshihide Sekimoto p. 1. The code is MIT licensed LICENSE. The image license is CC BY-SA 4.0, the same as the arXiv licence README.md.

• Classes. The paper lists four classes p. 1 fig. 5. The label map crackLabelMap.txt lists eight: D00, D01, D10, D11, D20, D40, D43, D44 crackLabelMap.txt. The README lists the four CRDDC classes README.md. The paper does not say which of the eight appear in RDD2022 annotations. We did not download the label zips, so this is not checked.
• Where the data is. The paper gives the GitHub repository as the access point p. 11. The repository README points to zip archives hosted elsewhere (RDD2022.zip and per-country zips) README.md. The repository itself holds code and a tutorial, not the zips.
• Privacy. The README says faces and car licence plates were blurred where they were clearly visible README.md. The paper does not describe this step.
• The repository also holds label-map config helpers and protobuf definitions in an Object Detection API style config_util.py. They are support code, not the paper's method, so they are not described here.

The paper's counts agree with its own figures (see Results). The README and the paper agree on the four CRDDC classes. The label-map conflict is listed above.

Pitfalls

• Class definitions changed. D40 meant rutting, bump, pothole and separation in RDD2018 tab. 1. In RDD2022 it is described as potholes p. 1. The paper does not say whether the definition changed. Check the annotations before merging with RDD2018 or RDD2019.
• Label map has extra classes. crackLabelMap.txt contains D01, D11, D43 and D44 crackLabelMap.txt, which the paper does not describe for RDD2022. Filter on the four classes in the paper.
• Image sizes are mixed. The sizes are 512, 600, 640, 720 and 3650x2044 wide p. 8 tab. 2. The paper states the India resize but gives no resizing rule for training. Pick and document one input size.
• Norway is a stitched view. The 3650x2044 image comes from two cameras stitched together p. 8. The paper does not say whether stitching seams are visible or whether the two halves were cropped.
• Coordinate convention not stated. The paper says "four decimal numbers" for the box corners p. 10, but the sample file uses integers fig. 13. The paper does not state the pixel origin or whether the max corner is exclusive. Check against a known image before training.
• Test labels are withheld. You can only score test predictions through the CRDDC'2022 leaderboard p. 11. Local evaluation is possible only on the labelled training split, and the paper does not define a validation split.
• Counts mix train and total. The Figure 2 footnote says its RDD2022 row covers the training set only fig. 2. Table 2 gives 47,420 images, which includes test tab. 2. Do not compare the two directly.
• Uneven classes and countries. Potholes (D40) are the rarest class with 6,544 labels fig. 2. Within one country the mix varies a lot. The Czech Republic has 197 D40 and 988 D00 train labels, while Norway has 461 D40 and 8,570 D00 fig. 5.
• China_Drone has no test set. Its images are used for training only p. 8 fig. 4.
• Norway vehicles are masked. The paper says vehicles in the Norway images were masked, so vehicle-related tasks should not use Norway p. 15.
• Pavement coverage is limited. The roads are mostly asphalt, and rigid pavement is not covered p. 6 p. 15. Results may not carry over to concrete roads.
• Licence terms for imagery. The US images are Google Street View images p. 8. The paper does not state Google's terms for them. The only licence statement is CC BY-SA 4.0 in the repository README README.md. CC BY-SA is a share-alike licence, so adapted images must keep that licence.
• Annotation quality is not measured. The paper describes the pipeline but gives no inter-annotator agreement or review procedure p. 10.

Lineage

• Builds on: RDD2018 tab. 1 p. 1, RDD2019 p. 1 and RDD2020 p. 1 p. 16. The RDD2019 change was a GAN-based augmentation p. 1.
• Related: arXiv:2008.13101 on transfer learning for road damage detection in multiple countries p. 16, ref. 5.
• Used by: the CRDDC'2022 challenge, which aims to build detectors on this data p. 14 README.md.
• Superseded by: not stated in this paper.