Automated railway object mapping using imagery and point clouds

Abstract

Everything around us is rapidly changing. Whole new blocks of buildings are built, huge infrastructural projects are constructed and so on. Hence, there is a need of a reliable and up-to-date inventory of the area and the objects of interest for mapping and monitoring assets and their changes. An answer of this upcoming need is an automated inventory of infrastructure using Remote Sensing and artificial intelligence (AI) techniques. Rail sector shares the same need for fast and reliable inspection on its infrastructure. Monitoring frequently the condition of the railway infrastructure can improve the maintenance efficiency and the avoidance of hazards. The traditional monitoring techniques are costly, time consuming and in some cases dangerous, due to their reliance on the physical presence of the inspector. Hence, new state-of-the-art techniques that are able to frequently and without putting in risk human lives, inspect the condition of the railway and its infrastructure. This master thesis aims at developing an efficient workflow for combining 2D imagery and 3D light detection and ranging (LiDAR) point clouds for the automated detection and localization of the railroad infrastructural objects into 3D world coordinate system, for monitoring the railway infrastructure. Using deep learning (DL) methods in imagery we detected and mapped, approximately the 60% of the railroad equipment of our interest (i.e. light signals and equipment boxes). These detected equipment were analyzed with stereoscopic techniques to retrieve their position in 3D world coordinate system. That led to the automated creation of a geographical information system (GIS) map having the positional and class information of railway equipment. Once the detected objects were mapped, then the point cloud data were automatically cropped into voxels including the same objects. Hence, using various sophisticated machine learning (ML) techniques, the points referring to the objects were classified. Furthermore, combining the positional information provided via 2D analysis with 3D point clouds, the vertical position was refined and the height of the mapped objects was estimated. Lastly, the positional information estimated from the 2D analysis enhanced the unsupervised ML classification in point clouds. The product of this classification, has the potential to be used as training data to train supervised point cloud classifiers.