NTU VIRAL: A Visual-Inertial-Ranging-Lidar Dataset, From an Aerial Vehicle Viewpoint

In plain words

Self-driving cars have many public datasets, but drones have few, and the aerial ones usually lean on cameras and motion sensors. This paper records flights of a drone that carries a car-like sensor kit: two 3D laser scanners, two cameras, inertial units and radio ranging units that measure distance to fixed anchors on tripods. A laser tracker follows a reflective prism on the drone and gives a reference path, so software can be scored against it.

The authors also explain how to calibrate the sensors and how to compare an estimated path with the reference fairly. The sensor and the prism are not at the same spot on the drone, and the two clocks and frames must be lined up first. The goal is a shared test bed: anyone can run their navigation method on the same flights and compare the results.

TL;DR

A dataset paper with no method to reproduce. It releases nine indoor and outdoor flights from the NTU campus (EEE, SBS, NYA groups), with rosbag logs, calibration files, laser-tracker ground truth and an evaluation procedure. Author-reported ATE: VIRAL-SLAM is lowest on all nine sequences. Read the Pitfalls before comparing numbers; the paper's ATE definition and its released code differ.

Problem

Navigation research for aerial vehicles lacks public data that matches industrial inspection settings (GPS-denied places such as building complexes, cranes, ships and hangars) §1 fig. 1. Aerial datasets usually focus on high frame-rate cameras and seldom include 3D lidar, which the payload budget often rules out; ground datasets have many sensors but very different motion §1. The paper's own survey table of public datasets is tab. 1.

Onboard odometry drifts. The authors' earlier work shows that range measurements to fixed landmarks can reduce or remove that drift §1.2. A reference is needed to measure drift at all: the paper uses a laser tracker for that §2.5.

Method

The contribution is the data collection, the sensor description, the calibration procedure and the evaluation protocol. No learning method is involved. Every value below is taken from the paper or from its released code.

1. Platform and sensors §2 fig. 2 tab. 2

• A DJI M600 Pro hexacopter carries a payload bay with the sensors §2 fig. 2.
• Body frame: the IMU sits at the centre of the body frame, and all other extrinsics are referenced to it §2.1.
• Two 16-channel OS1 Gen1 lidars, one horizontal and one vertical, each with an internal IMU §2.2.
• Two monochrome global-shutter cameras, triggered by an external signal §2.3.
• Four UWB ranging nodes on the drone (two radios, each with antennas A and B) and three anchors (IDs 100, 101, 102), giving 12 drone-to-anchor pairs §2.4 fig. 3.
• A laser tracker tracks a crystal prism on the drone §2.5.

2. Sequences §3 tab. 3 fig. 4 fig. 5

• Three groups: EEE (car park), SBS (open square) and NYA (auditorium), each with three sequences §3.1–3.3.

3. Data format and conventions §4 eq. 1

• Messages carry the time of publication on ROS; the lidar stamp is the scan end time, and each point has a time relative to the scan start, which is what deskewing needs §4.2.
• UWB range messages carry extra fields, such as signal-to-noise ratio and line-of-sight status, and the anchor coordinates §4.3.
• Extrinsics in the calibration files follow the parent-child naming of ROS and EuRoC. A point in the camera frame C maps to the body frame B by the rotation and translation of the extrinsic, and orientations compose the same way §4.4 eq. 1.

4. Calibration §5 fig. 7

• Stereo cameras: intrinsics from a chessboard sequence, using a pinhole model with radial-tangential distortion; a reprojection-error threshold selects the images used for extrinsics §5.1. The number of images and the threshold are not stated.
• Camera to IMU: a Kalibr camera-IMU procedure with the stereo intrinsics as initial guess; IMU biases are calibrated beforehand with the manufacturer's software §5.2.
• Flexible parts: a VICON system checks most other extrinsics with the rig at rest, and tilting the payload bay adds a check on the prism displacement §5.3.

5. Ground truth §2.5 §5.3

• The tracker frame is aligned with gravity at start-up, its z axis pointing opposite to gravity §2.5.
• Tilting the payload bay moves the prism by at most about 2 cm relative to its static position §5.3.

6. Evaluation protocol §6 eq. 2 eq. 3

The paper recommends this sequence for each estimated trajectory.

# Evaluate one estimated trajectory  [§6]
# est_rotations (N, 3, 3), est_positions (N, 3), body frame
# gt_positions (M, 3), tracker frame L; times in seconds
# prism_offset (3,): body origin to prism, body frame  [§6.1]
# Returns the ATE value; units and scale: see Pitfalls

def evaluate_ate(
    est_times, est_rotations, est_positions,
    gt_times, gt_positions, prism_offset,
):
    # Move each estimate from the body origin to the prism  [§6.1] [eq. 2]
    prism_positions = est_positions + est_rotations @ prism_offset
    # Keep estimate times that have tracker samples on both sides, with a
    # gap below 0.1 s between those samples  [§6.2]
    kept = keep_bracketed(est_times, gt_times, max_gap=0.1)
    # Linearly interpolate the tracker positions at the kept times  [§6.2]
    reference = interpolate(gt_times, gt_positions, est_times[kept])
    estimated = prism_positions[kept]
    # Closed-form rigid fit (rotation, translation) between the two frames
    # that minimises the summed squared residuals  [§6.3] [eq. 3]
    rotation, translation = umeyama(estimated, reference)
    aligned = estimated @ rotation.T + translation
    # The paper calls the minimised summed squared residual the ATE  [§6.3]
    return sum(norm(aligned - reference, axis=1) ** 2)

Equations, as numbered in the compiled paper:

• eq. 1 (§4.4): body-frame position and orientation of an object seen in camera C, from the extrinsic rotation and translation.
• eq. 2 (§6.1): prism position in the world frame equals the body-frame estimate plus the body rotation applied to the body-to-prism translation.
• eq. 3 (§6.3): the rigid transform (rotation, translation) that minimises the summed squared residuals between the aligned estimate and the reference. The paper cites Umeyama's closed form for this fit §6.3.
• An unnumbered align block in §6.2 gives the matching rule: each kept time lies between two tracker samples whose gap is less than 0.1 s, and the tracker position at that time is linearly interpolated.

Hyperparameters

The paper trains nothing, so this table lists the sensor settings and the evaluation constants that an implementer or a reader needs.

Sequence statistics tab. 3, as reported:

Results

Author-reported. Setup: nine sequences of 181 to 429 seconds each, recorded on a drone, with laser-tracker ground truth §3 tab. 3. Table 4 compares four localisation frameworks on ATE without any time shift, in metres tab. 4 §5.3. Which evaluation steps from §6 were used for Table 4 is not stated.

Source: tab. 4. In the paper, the best value per row is bold and the second best is underlined; here the best (VIRAL-SLAM) is bold, and LIO-SAM is second in every row. VINS-Fusion is the largest in every row.

Other reported results:

• A sweep of ground-truth time shifts from −0.5 s to 0.5 s shows that all visual-inertial, lidar-inertial and combined methods reach their minimum error within ±0.1 s §5.3 fig. 8.
• Camera-IMU time offset found by Kalibr: 20 ms, reported but not applied §5.2.
• Calibration was checked with VINS-Mono, VINS-Fusion and OpenVINS, which gave meaningful trajectories §5.2. No numbers are given.

Paper vs code

The code checked is the authors' repository ntu-aris/ntu_viral_dataset (clone at code_commit). The paper's footnote links it §1.2. Its licence is contradictory, so this card cites a few facts from it and quotes none; see Pitfalls. The MATLAB evaluation that the paper mentions §6 is in a separate repository, ntu-aris/viral_eval, which has no licence; it was not used.

• Body-to-prism offset: the paper gives it as "almost 0.4 m" and says it is read from the calibration files §6.1. The notebook hard-codes the vector (−0.293656, −0.012288, −0.273095) m ntuviral_evaluate.ipynb. The values agree with the paper's magnitude; the code does not read the calibration file.
• Resampling: the paper uses linear interpolation of the ground truth between samples less than 0.1 s apart §6.2. The notebook matches each estimate time to a ground-truth time within 0.05 s ntuviral_evaluate.ipynb ntuviral_evaluate.ipynb. No interpolation step was found in the notebook.
• Alignment: the paper uses a closed-form Umeyama fit §6.3. The notebook calls the alignment method of a trajectory library ntuviral_evaluate.ipynb. Whether it estimates scale is not stated in the notebook.
• Metric: the paper defines the ATE as the summed squared residual after alignment, with no square root or division by the number of samples §6.3 eq. 3. The notebook reports the root-mean-square of the translation error ntuviral_evaluate.ipynb ntuviral_evaluate.ipynb. These are different quantities.
• The notebook's comment says "Create the spline" in the ground-truth loader, but no spline is built there ntuviral_evaluate.ipynb. Treat the comment as stale.

Pitfalls

• ATE definition versus units. The paper's ATE is a summed squared residual, so its unit is m², and it grows with the number of samples §6.3 eq. 3. Table 4 is labelled in metres tab. 4, and the released code reports an RMSE in metres. The three are not interchangeable. Pick one definition, and state it when you report a number.
• Prism versus body origin. Comparing the body-frame estimate with the tracker directly can bias the error by about 0.4 m in the worst case §6.1. Apply the rotation at each sample, since the drone rotates during flight eq. 2.
• Time alignment. The ground truth is merged with the onboard bag after recording, by temporal synchronisation whose method the paper does not describe §4.1. The paper says the residual error is within 0.1 s §5.3. Estimates are sensitive to this, as the time-shift sweep shows fig. 8. A ground-truth gap longer than 0.1 s drops the affected estimate samples §6.2.
• Camera-IMU offset not applied. Kalibr finds a 20 ms offset, but the authors leave the timestamps unchanged and say the value is for reference only §5.2. An implementer who needs the offset must apply it.
• Lidar deskew. The stamp is the scan end; per-point times are relative to the scan start §4.2. A deskew step needs those per-point times.
• UWB dropouts and rate. The effective UWB rate varies with the drone's orientation relative to the anchors, because the radiation pattern changes the received signal §2.4 §7.2. Signal loss and multipath appear in the NYA sequences §3.3 fig. 6.
• Exposure and images. Exposure is set to about one-third of the auto default to reduce glare and motion blur, which makes the images darker. Direct visual SLAM can struggle with this §7.1.
• Calibration target. A large roll of the chessboard raises the reprojection error on the 120-degree lens, so the authors keep the roll small §5.1. The number of calibration images and the error threshold are not stated.
• Ground-truth gaps. In SBS the tracker lost the prism for short periods because of its low vantage point §7.2.
• Figure cross-reference. Section 5.2 says the calibration file is in "Fig. 9", but the calibration file is figure 7 in compile order; figure 9 is the SLAM result §5.2 fig. 7 fig. 9. Check the figure before citing it.
• Licensing of the data and code. The repository's LICENSE file is MIT, but its copyright line is copied from a Jekyll theme, and its README says the work is CC BY-NC-SA 4.0 for "non-commercial academic use" README.md. The paper's text does not state a licence for the data. Treat the data as non-commercial until the authors clarify.
• Paper versus repository tooling. The paper says MATLAB scripts are included §6. The repository ships a Jupyter notebook as its main evaluation tool README.md, and the MATLAB code is in another repository.

Lineage

• Builds on: the authors' earlier UWB-aided fusion work, cited as nguyen2021viral (VIRAL-fusion) §2.4, and the VIRAL SLAM system, cited as nguyen2021viralslam §5.3 with arXiv:2105.03296 in the bibliography.
• Compared against: VINS-Fusion, LIO-SAM and M-LOAM (arXiv:2010.14294) in Table 4 tab. 4. Their details are outside this card.
• Related datasets: the paper's comparison table names, among others, EuRoC, KITTI, Oxford RobotCar, TUM VI and the Newer College dataset tab. 1. Their content is not used here.