Challenges of Indoor SLAM: A multi-modal multi-floor dataset for SLAM evaluation

Abstract: Robustness in Simultaneous Localization and Mapping (SLAM) remains one of the key challenges for the real-world deployment of autonomous systems. SLAM research has seen significant progress in the last two and a half decades, yet many state-of-the-art (SOTA) algorithms still struggle to perform reliably in real-world environments. There is a general consensus in the research community that we need challenging real-world scenarios which bring out different failure modes in sensing modalities. In this paper, we present a novel multi-modal indoor SLAM dataset covering challenging common scenarios that a robot will encounter and should be robust to. Our data was collected with a mobile robotics platform across multiple floors at Northeastern University's ISEC building. Such a multi-floor sequence is typical of commercial office spaces characterized by symmetry across floors and, thus, is prone to perceptual aliasing due to similar floor layouts. The sensor suite comprises seven global shutter cameras, a high-grade MEMS inertial measurement unit (IMU), a ZED stereo camera, and a 128-channel high-resolution lidar. Along with the dataset, we benchmark several SLAM algorithms and highlight the problems faced during the runs, such as perceptual aliasing, visual degradation, and trajectory drift. The benchmarking results indicate that parts of the dataset work well with some algorithms, while other data sections are challenging for even the best SOTA algorithms.

The dataset is available at the following link

Downloads

ISEC dataset

Label	Size (GB)	Duration (s)	Appx. Length (m)	Description
full_sequence	515.0	1539.70	782	reflective surfaces, minimal dynamic content, daylight, symmetric floors, elevators, open atrium
5th_floor	145.8	437.86	187	one loop, one out and back
transit_5_to_1	36.8	109.00	N/A	transit from 5th to 1st floor in middle elevator
1st_floor	43.0	125.58	65	one loop, open layout different from other floors, many exterior windows
transit_1_to_4	112.4	337.40	144	transit across 1st floor, up to 3rd floor in freight elevator, across 3rd floor, up to 4th floor in right elevator
4th_floor	43.2	131.00	66	one loop, some dynamic content towards end
transit_4_to_2	21.9	65.00	22	transit from 4th floor to second floor in right elevator
2nd_floor	89.7	266.00	128	two loops in a figure eight
transit_2_to_5	22.2	65.86	128	transit from 2nd floor to fifth floor in right elevator

ISEC dataset calibration

Name	Description
cams_calib.yaml	calibration of front 5 cameras, including intrinsics and relative transformation between them
cam2_imu_calib.yaml	transformation between camera_2 and IMU, including time shift between camera_2 and IMU
cam5_imu_calib.yaml	calibration of left side camera and transformation between camera_5 and IMU, including time shift between camera_5 and IMU
cam6_imu_calib.yaml	calibration of right side camera and transformation between camera_6 and IMU, including time shift between camera_6 and IMU
imu_params.yaml	IMU parameters, including noise parameters of accelerometer and gyroscope, as well as sampling rate

Snell library dataset

Label	Size (GB)	Duration (s)	Appx. Length (m)	Description
full_sequence	573.5	1,700.6	699	feature rich rooms, featureless hallways, many obstacles, stationary and dynamic people in scene
1st_floor	144.6	428.70	221	two loops with shared segment, some dynamic content
transit_1_to_3	28.3	84.00	N/A	transit from 1st floor to 3rd floor in left elevator
3rd_floor	213.7	633.59	345	two concentric loops with two shared segments, narrow corridor with dynamic content, near field obstructions
transit_3_to_2	27.8	82.41	N/A	transit from 3rd floor to 2nd floor in right elevator
2nd_floor	126.1	374.00	186	one loop, out and back in featureless corridor
transit_2_to_1	33.0	97.90	N/A	transit from 2nd floor to 1st floor in right elevator, dynamic objects cover FOV near end

Snell library dataset calibration

Name	Description
cams_calib.yaml	calibration of front 5 cameras, including intrinsics and relative transformation between them
cam2_imu_calib.yaml	transformation between camera_2 and IMU, as well as time shift between camera_2 and IMU
imu_params.yaml	IMU parameters, including noise parameters of accelerometer and gyroscope, as well as sampling rate

Snell library dataset Zed calibration

Name	Description
zed_cam2_calib.yaml	calibration of Zed cameras, including intrinsics and relative transformation with respect to camera_2 of the front 5 cameras
zed_imu_params.yaml	Zed IMU parameters, including noise parameters of accelerometer and gyroscope, as well as transformation with respect to the body frame

Sensors

Sensor rig

Top view of the sensor rig showing sensor frames for the front-facing camera array (red), the non-overlapping side cameras (orange), the IMU (green) and the lidar (blue). Note the above image follows the convention that $\otimes$ indicates an axis into the plane of the image, and $\bullet$ indicates an axis out of the plane of the image. All of the cameras are z-axis forward, y-axis down.

Sensor setup

Dscription of various sensors and their settings used to collect our dataset. Note that Zed2i sensor is available only in the Snell dataset.

Citation

If you use our work, please cite our paper:

P. Kaveti et al., "Challenges of Indoor SLAM: A Multi-Modal Multi-Floor Dataset for SLAM Evaluation" 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE), Auckland, New Zealand, 2023, pp. 1-8, doi: 10.1109/CASE56687.2023.10260618.

@INPROCEEDINGS{10260618,
  author={Kaveti, Pushyami and Gupta, Aniket and Giaya, Dennis and Karp, Madeline and Keil, Colin and Nir, Jagatpreet and Zhang, Zhiyong and Singh, Hanumant},
  booktitle={2023 IEEE 19th International Conference on Automation Science and Engineering (CASE)}, 
  title={Challenges of Indoor SLAM: A Multi-Modal Multi-Floor Dataset for SLAM Evaluation}, 
  year={2023},
  volume={},
  number={},
  pages={1-8},
  doi={10.1109/CASE56687.2023.10260618}}