Paper · 2025-01-16 · cs.RO, cs.CV
Comparison of Various SLAM Systems for Mobile Robot in an Indoor Environment
Runs ten ROS-based SLAM systems (2D lidar, monocular and stereo cameras) on one recorded indoor office trajectory of a UGV prototype and scores eight by absolute trajectory error against Hector SLAM. Cartographer has the lowest RMSE (0.024 m); among camera systems, stereo RTAB-Map (0.163 m) and monocular ORB-SLAM (0.166 m) do best (author-reported, single run).
Simultaneous localization and mapping (SLAM)slamlidarvisual-slamrosbenchmarkmobile-robot
AI-compiled by claude-haiku-5-5 on 2026-11-12, not yet reviewed by a human. Every claim cites its source; check it when a detail matters.
In plain words
A robot that drives into an unknown room has to answer two questions at once: where it is, and what the room looks like. Software that does both is called SLAM. Many open-source SLAM packages exist, and they use different sensors: a spinning laser, a single camera, or a pair of cameras. This paper asks which of them tracks a small robot most accurately in an ordinary office.
The authors built a small wheeled robot with a laser scanner and two kinds of camera. They drove it once around a marked rectangle on the floor, recorded all sensors, and then ran each SLAM package on that recording. They compared the path each package estimated against the path of a laser-based reference system. The laser-based systems did best. Among the camera-based systems, a stereo package and a monocular package came out ahead. The study covers one room and one run, so it shows what happened there rather than what happens in general.
TL;DR
- Ten ROS-based SLAM systems are run offline on one recorded trajectory of a UGV prototype in an office; eight have error results §III §IV-A tab. 5.
- Error is the absolute trajectory error (ATE) against a trajectory from Hector SLAM, not against an independent ground truth §IV-B eq. 1.
- Cartographer has the lowest RMSE (0.024 m); RTAB-Map (stereo, 0.163 m) and ORB-SLAM (monocular, 0.166 m) lead among camera systems tab. 5 (author-reported).
- Use it as a sanity check for a single indoor sequence, not as a general ranking: there is one run, no variance, and monocular scale handling is not stated.
Problem
Indoor mobile robots need a pose estimate to navigate. Lidar SLAM is the usual choice for indoor robots, while cameras are cheap and passive but raise scale and robustness problems §I. Earlier comparisons either covered only lidar methods or only monocular methods, or used different datasets for each system §II. The paper asks which ROS-available SLAM system gives the best trajectory on the same recorded data from a single prototype §I §II.
Method
Platform (hardware). A ground vehicle built on a 4WD radio-controlled chassis with Ackermann steering, called Labcar. It carries a 2D lidar, a monocular camera and a ZED stereo camera. The vehicle's computer is an NVIDIA Jetson TX1; the recordings are made on the vehicle, and all SLAM runs are done later on a separate ground station §III-A tab. 3 tab. 4 fig. 2. The authors say the Ackermann chassis should help visual methods, since it does not rotate much at low speed §III-A.
Data collection. The robot is teleoperated along a closed loop over a rectangular area in an office with plain walls. The loop is marked with a thread on the floor. The authors call the thread the ground truth, but note that the robot followed it only on straight segments; the turns have a radius of about 1 m §IV-A fig. 4 fig. 5. Sensor data for the lidar, the monocular camera and the stereo camera is recorded to ROS bag files §IV-A.
Systems compared. The compiled paper studies ten systems tab. 2 §V-A:
- 2D lidar: GMapping, Hector SLAM, Cartographer §I.
- Monocular camera: LSD-SLAM, ORB-SLAM (mono), DSO §I tab. 5.
- Stereo camera: ZEDfu, RTAB-Map, ORB-SLAM (stereo), S-PTAM §I tab. 5.
The introduction also lists PTAM, SVO and DPPTAM among the monocular methods examined. The conclusion says they lost track because of too few features, so they produced no trajectory and do not appear in the results table §VI. The table of feature types (Table I) describes each visual system's features, descriptor, density and loop closure, but the paper does not use it in the results tab. 1.
Evaluation metric. Let the reference trajectory be the Hector SLAM output. Its x-y positions are linearly interpolated (described as a first-degree polynomial) at the timestamps of the system under test. The error at each timestamp is the Euclidean distance between the interpolated reference point and the estimated point, in the x-y plane §IV-B eq. 1. The paper reports RMSE, mean, median, standard deviation, minimum and maximum of these errors §IV-B tab. 5.
# Absolute trajectory error (ATE) per pose [§IV-B] [eq. 1]
# Input: method_xy (N, 2) at method_t (N,)
# reference_xy (M, 2) at reference_t (M,) (Hector SLAM)
# Output: errors (N,) in metres
def absolute_trajectory_error(method_t, method_xy,
reference_t, reference_xy):
# Linear interpolation of the reference at each method timestamp [§IV-B]
# Timestamp matching and handling outside the reference span: not stated.
reference_at_t = interpolate_linear(reference_t, reference_xy, method_t)
# Euclidean distance in the x-y plane only [§IV-B] [eq. 1]
# Alignment (rotation, translation, scale) before comparison: not stated.
errors = norm(reference_at_t - method_xy, axis=1)
# Statistics reported per system: RMSE, mean, median, std, min, max [§IV-B]
return errorsQualitative map comparison. The authors compare the maps each system builds. Lidar systems output occupancy grids, and the paper calls Cartographer the best of the three for 2D lidar. The direct methods (LSD-SLAM, DSO) give dense point clouds, and the feature-based ones (ORB-SLAM, RTAB-Map, S-PTAM) give sparse point clouds §V-B fig. 6. This part of the paper has no numbers.
Hyperparameters
The paper reports hardware and software, not algorithm settings. The per-system configuration files, thresholds and feature counts are not stated.
| Name | Value | Where | Notes |
|---|---|---|---|
| Vehicle chassis | 4WD Traxxas #74076, Ackermann steering | §III-A fig. 2 | Labcar platform |
| On-board computer | NVIDIA Jetson TX1: quad ARM A57, Maxwell GPU, 4 GB RAM | tab. 3 | Used for acquisition |
| Lidar | Hokuyo UTM-30LX (2D) | tab. 3 | |
| Monocular camera | Basler acA1300-200uc | tab. 3 | Frame rate not stated in the compiled paper |
| Stereo camera | ZED | tab. 3 | Resolution and baseline not stated |
| JetPack, OS, ROS | 3.1, Ubuntu 16.04, ROS Kinetic | tab. 3 | |
| Ground station | Intel Core i7-6500U, NVIDIA GTX 950M, 12 GB RAM | tab. 4 | Runs all SLAM systems, offline |
| Ground station OS, ROS | Ubuntu 16.04, ROS Kinetic | tab. 4 | |
| Reference trajectory | Hector SLAM (2D lidar) | §IV-B | Assumed more accurate; not checked independently |
| Interpolation | first-degree polynomial (linear) | §IV-B | Edge handling not stated |
| Error | Euclidean x-y distance, ATE | §IV-B eq. 1 | Alignment not stated |
| Statistics | RMSE, mean, median, std, min, max | §IV-B tab. 5 | Units metres |
| Runs per system | not stated | One trajectory; results are one value per system tab. 5 | |
| Monocular scale handling | not stated | Conclusion says extra scale recovery is needed §VI | |
| SLAM system parameters | not stated | No configuration given in the compiled paper | |
| Trajectory length, duration | not stated | Number of laps not stated in the compiled paper | |
| Frame rate, processing time | not stated | Runs are offline; no timing reported |
Results
Author-reported. Setup: one recorded office trajectory on the Labcar prototype; each system run offline on the ground station; ATE in metres against Hector SLAM §IV-B tab. 5. The table gives one value per system, with no variance or repeated runs.
| System | RMSE (m) | Mean (m) | Median (m) | Std. (m) | Min (m) | Max (m) | Source |
|---|---|---|---|---|---|---|---|
| Cartographer (2D lidar) | 0.024 | 0.017 | 0.013 | 0.021 | 0.001 | 0.07 | tab. 5 |
| LSD-SLAM (mono) | 0.301 | 0.277 | 0.262 | 0.117 | 0.08 | 0.553 | tab. 5 |
| ORB-SLAM (mono) | 0.166 | 0.159 | 0.164 | 0.047 | 0.047 | 0.257 | tab. 5 |
| DSO (mono) | 0.459 | 0.403 | 0.419 | 0.219 | 0.007 | 0.764 | tab. 5 |
| ZEDfu (stereo) | 0.726 | 0.631 | 0.692 | 0.358 | 0.002 | 1.323 | tab. 5 |
| RTAB-Map (stereo) | 0.163 | 0.138 | 0.110 | 0.085 | 0.004 | 0.349 | tab. 5 |
| ORB-SLAM (stereo) | 0.190 | 0.151 | 0.102 | 0.115 | 0.004 | 0.414 | tab. 5 |
| S-PTAM, no loop closure | 0.338 | 0.268 | 0.244 | 0.206 | 0.001 | 0.768 | tab. 5 |
| S-PTAM, with loop closure | 0.295 | 0.257 | 0.242 | 0.145 | 0.006 | 1.119 | tab. 5 |
The paper names RTAB-Map as the best camera method by RMSE and ORB-SLAM (mono) as second §V-A. Among 2D lidar systems it recommends Cartographer, and it reports Cartographer and Hector SLAM agreeing closely (RMSE 0.024 m) §V-B §VI tab. 5. Hector SLAM is the reference and has no row of its own in the table.
Paper vs code
No official code. The arXiv abstract page lists no code repository and no
project page. A GitHub repository search for "slam comparison filipenko"
returned no results. These are the only two searches, so an official repository
may exist that they did not find. Nothing was cloned. The LaTeX source contains
an implementation section, but main.tex does not include it (its \input is
commented out), so its content is not part of the compiled paper and is not
used here.
Pitfalls
- The reference is an estimate, not ground truth. Hector SLAM is assumed to be more accurate and the paper says this is confirmed later, but the compiled paper does not show that comparison beyond Cartographer's agreement §IV-B tab. 5. The marked thread is called ground truth, yet the robot followed it only on straight segments §IV-A. Errors in the table measure agreement with Hector SLAM, not true position.
- Monocular scale is not handled in a stated way. The conclusion says no monocular system solves scale without extra information §VI. The paper does not say whether the monocular trajectories were scaled before the ATE was computed. The monocular values in the table therefore depend on an unstated choice.
- ATE has no alignment step stated. Equation 1 is a pointwise distance in x-y eq. 1. The paper does not say whether the two trajectories were aligned in rotation, translation or scale, nor how timestamps were matched beyond linear interpolation §IV-B. Heights and orientations are not compared.
- One sequence, one run. There is no repeated run, seed or confidence interval tab. 5. Differences of a few centimetres (for example between the two S-PTAM rows, or between ORB-SLAM mono and RTAB-Map) are not shown to be stable.
- "Most stable" is not backed by the table. The conclusion calls ORB-SLAM (stereo) the most robust and stable camera system §VI. Its RMSE (0.190 m) is higher than RTAB-Map's (0.163 m), and its standard deviation (0.115 m) is higher than RTAB-Map's (0.085 m) tab. 5. The stability claim rests on text, not on a reported metric.
- Failed systems have no numbers. PTAM, SVO and DPPTAM are said to lose track §VI. They are absent from the table, so the paper gives no error for them.
- Area shape is inconsistent. The experiment section says rectangular and the conclusion says square §IV-A §VI. The paper does not give the dimensions.
- Loop closure is listed, not tested. Table I lists loop closure per system, and the S-PTAM rows differ by it tab. 1 tab. 5. The paper does not report the effect of loop closure for other systems.
- Costs are not measured. All SLAM systems ran offline on a laptop-class ground station §III-A tab. 4. The paper reports no processing time, CPU or memory use, and it lists the effect of computing resources as future work §VI. Real-time behaviour on the Jetson TX1 is unknown.
- Map-quality claims are qualitative. The map comparison §V-B fig. 6 has no metric for map accuracy, only visual description.
- Figure label reuse. The label
fig:exenvis defined for two figures in the source §IV-A. Figure numbers in this card follow source order.
Lineage
- Builds on: none stated. The related work cites earlier comparisons of ROS SLAM packages on indoor robots §II.
- Compared against: the related-work studies that compare monocular and lidar SLAM on indoor platforms §II. The paper does not compare against any learned or non-ROS system.
- Superseded by: not known from content in this wiki.
Source and license
Adapted from “Comparison of Various SLAM Systems for Mobile Robot in an Indoor Environment” by Maksim Filipenko, Ilya Afanasyev (arXiv:2501.09490), used under CC BY 4.0. Changes: condensed and restructured for implementation, with tables, equations and algorithms taken from the paper and its code, each cited to its source. Not affiliated with or endorsed by the authors.