Evaluating SLAM in an urban dynamic environment

Abstract

Simultaneous Localization And Mapping (SLAM) algorithms provide accurate localization for autonomous vehicles and provide essential information for the path planning module. However, SLAM algorithms as- sume a static environment in order to estimate a location. This assumption influences the pose estimation in dynamic urban environments. The impact of this assumption on day-to-day scenarios of an intelligent vehicle is unknown. A deeper understanding on the effect of dynamic scenarios in an urban environment could lead to simple and robust solutions for SLAM algorithms in intelligent vehicles. The objective of this research is to develop a methodology that isolates the effect of an urban dynamic environment on the per- formance of a SLAM algorithm. This requires constant environment conditions including constant weather conditions, lighting conditions and identical trajectories over time. The methodology is tested with a stereo feature based V-SLAM algorithm called ORB SLAM [19], which illustrates the in-depth analysis that is possi- ble with this experiment. The main research question is: How does a dynamic urban environment influence the pose estimation accuracy of stereo ORB SLAM? Two specific dynamic scenarios are designed to represent a dynamic urban environment: driving behind another vehicle and vehicles approaching on the other side of the road. On these scenarios, an in-depth anal- ysis of ORB SLAM is performed to observe how the algorithm’s design influences the robustness to a dynamic environment. Functions within the algorithm are bypassed to analyze the effect on the performance. Specifi- cally, the place recognition function and map point filtering function are bypassed. The analysis proofs which functions assist in the overall robustness to a dynamic environment. Moreover, an analysis is performed of the algorithm in localization mode to research the effect of utilizing maps that were created under different conditions. The knowledge gained from the full analysis can be utilized to improve other V-SLAM algorithms. The experiment is performed in CARLA [6], an open source simulator. CARLA provides an elaborate sen- sor suite which support multiple camera setups and LIDAR sensors. Furthermore, the simulator provides free maps which represent realistic urban environments and allows for easy and accurate access to the ground truth position. A setup is designed with the simulator that allows complete isolation of the effect of a dy- namic environment. The setup allows full control of lighting conditions, weather conditions and allows iden- tical trajectories over time in different dynamic scenarios. Each scenario is simulated over several different trajectories in which the camera images are converted to rosbags. Each variation of the ORB SLAM algorithm is tested on the produced rosbags. The resulting pose estimations in dynamic conditions are compared to the pose estimations made during static conditions to analyze the effect of dynamic scenarios on the perfor- mance of the algorithm. The method successfully isolated the effect of a dynamic environment on the performance of stereo ORB SLAM. It allows for a detailed analysis which aids in finding the source of performance differences. In general, stereo ORB SLAM displays robust behavior to a dynamic environment. The experiment shows that the algo- rithm is sensitive to false relocalization when the stereo camera setup is driving 10 meters behind another vehicle for a long period of time. During these conditions, ORB SLAM cannot provide accurate pose esti- mations even when the place recognition module is deactivated. Furthermore, the map point filtering does increase the robustness in certain dynamic scenarios. Finally, the data suggests that utilizing maps created in different conditions does influence the pose estimation in localization mode. However, more data is needed to confirm these results. The methodology has proven its value for in depth analysis of robustness to an urban dynamic environ- ment for a SLAM algorithm. This experiment is not limited to ORB SLAM but could be utilized for other monocular and stereo V-SLAM methods, as well as LIDAR based methods. New solutions can be developed to increase robustness to a dynamic environment and tested on the same rosbags. This methodology could be an important tool for the development SLAM algorithms for intelligent vehicles.