The Mobile Laser Scanning (MLS) data inevitably includes dynamic objects because there are always other vehicles (e.g., other cars, motorbikes, bikes, etc.) moving in the area near the MLS data collection vehicle on the road. These dynamic objects need to be removed in advance for many point cloud applications. This paper designs an efficient and memory-friendly data frame aware optimized Octomap-based dynamic object detection and removal method for MLS data. Firstly, the input MLS data is split into multiple data frames based on the timestamp. Each data frame is inserted into a separate Octomap with part of its neighbouring data frames. A statistics-based method is applied to each data frame to find the passable voxel cell space (free space) in Octomap and all points in the free space are extracted as free points. Second, the region of interest (ROI) related to the dynamic object is delineated to retain free points related to dynamic objects. Then the free-point rate and the multi-return rate are calculated to further remove noise and vegetation points from free points. Finally, the fixed radius search is used to extract dynamic objects from the filtered free points. The proposed method is tested in four case sites in Delft, the Netherlands. Results show that 84.98% of dynamic objects are detected and extracted correctly. The proposed method is 18.27% more efficient on average than the original Octomap method, can be further accelerated by parallel computing, and only needs 39.40% of the maximum memory consumption.

The enrichment of the point clouds with colour images improves the visualisation of the data as well as the segmentation and recognition processes. Coloured point clouds are becoming increasingly common, however, the colour they display is not always as expected. Errors in the colouring of point clouds acquired with Mobile Laser Scanning are due to perspective in the camera image, different resolution or poor calibration between the LiDAR sensor and the image sensor. The consequences of these errors are noticeable in elements captured in images, but not in point clouds, such as the sky. This paper focuses on the correction of the sky-coloured points, without resorting to the images that were initially used to colour the whole point cloud. The proposed method consists of three stages. First the region of interest where the erroneously coloured points are accumulated, is selected. Second, the sky-coloured points are detected by calculating the colour distance in the Lab colour space to a sample of the sky-colour. And third, the colour of the sky-coloured detected points is restored from the colour of the nearby points. The method is tested in ten real case studies with their corresponding point clouds from urban and rural areas. In two case studies, sky-coloured points were assigned manually and the remaining eight case studies, the sky-coloured points are derived from the acquisition errors. The algorithm for sky-coloured points detection obtained an average F1-score of 94.7%. The results show a correct reassignment of colour, texture, and patterns, while improving the point cloud visualisation.

Vehicle-related ground occlusion is a common problem in MLS data. This study aims to design a detection and reconstruction method of static vehicle-related ground occlusion for MLS data. Ground extraction and vehicle segmentation are performed on the input point cloud data in advance. Then an α-shape boundary based on the prior vehicle geometry is designed to split non-ground empty area and ground occlusions. The occlusion is detected and matched with its corresponding vehicle using the relative position between them. This relative position relation and the height difference are used to detect the curb direction as the local road direction. Finally, the occlusions are reconstructed using two different methods: (1) a cell-based linear interpolation and (2) a point-based mathematical morphology. The methodology is tested by original scanned data and multi-temporal evaluation data captured from a residential area in Delft, the Netherlands with vehicle-mounted LiDAR sensors. The result shows that all occlusions cause by vehicles are successfully detected and the curb (road) direction is correctly extracted in most of the occluded areas. Both reconstructed results can visually integrate the original scanned data and recover the curb structure. The reconstruction errors of the linear interpolation method are 0.045 m in the z-axis direction and 0.051 m in total and the reconstruction errors of mathematical morphology are 0.048 m in the z-axis direction and 0.052 m in total.

Point cloud data have rich semantic representations and can benefit various applications towards a digital twin. However, they are unordered and anisotropically distributed, thus being unsuitable for a typical Convolutional Neural Networks (CNN) to handle. With the advance of deep learning, several neural networks claim to have solved the point cloud semantic segmentation problem. This paper evaluates three different neural networks for semantic segmentation of point clouds, namely PointNet++, PointCNN and DGCNN. A public indoor scene of the Amersfoort railway station is used as the study area. Unlike the typical indoor scenes and even more from the ubiquitous outdoor ones in currently available datasets, the station consists of objects such as the entrance gates, ticket machines, couches, and garbage cans. For the experiment, we use subsets from the data, remove the noise, evaluate the performance of the selected neural networks. The results indicate an overall accuracy of more than 90% for all the networks but vary in terms of mean class accuracy and mean Intersection over Union (IoU). The misclassification mainly occurs in the classes of couch and garbage can. Several factors that may contribute to the errors are analyzed, such as the quality of the data and the proportion of the number of points per class. The adaptability of the networks is also heavily dependent on the training location: the overall characteristics of the train station make a trained network for one location less suitable for another.

Although point clouds are characterized as a type of unstructured data, timestamp attribute can structure point clouds into scanlines and shape them into a time signal. The present work studies the transformation of the street point cloud into a time signal based on the Z component for the semantic segmentation using Long Short-Term Memory (LSTM) networks. The experiment was conducted on the point cloud of a real case study. Several training sessions were performed changing the Level of Detail of the classification (coarse level with 3 classes and fine level with 11 classes), two levels of network depth and the use of weighting for the improvement of classes with low number of points. The results showed high accuracy, reaching at best 97.3% in the classification with 3 classes (ground, buildings, and objects) and 95.7% with 11 classes. The distribution of the success rates was not the same for all classes. The classes with the highest number of points obtained better results than the others. The application of weighting improved the classes with few points at the expense of the classes with more points. Increasing the number of hidden layers was shown as a preferable alternative to weighting. Given the high success rates and a behaviour of the LSTM consistent with other Neural Networks in point cloud processing, it is concluded that the LSTM is a feasible alternative for the semantic segmentation of point clouds transformed into time signals.

Traffic signs are one of the most relevant road assets for driving, as the safety of drivers depends to a great extent on their correct location. In this paper two methods are compared for the segmentation of the sign and the pole supporting it. Both methods are based on the morphological opening to identify the sign points, the first one directly employs the mathematical morphology directly applied to point clouds and the second one through point cloud rasterization into images. The comparison was conducted on twenty real traffic signs acquired with Mobile Laser Scanning obtaining point clouds from environments with signposts, traffic lights and lampposts. The results showed a correct segmentation of the signs, obtaining a F-score of 0.81 by the point-based method and a 0.75 by 2D image method. In particular, the point-based mathematical morphology proved to be more accurate in the segmentation of traffic sings installed on traffic lights and lampposts, avoiding over detection shown by the 2D image method.

Mathematical morphology is a technique recently applied directly for point cloud data. Its working principle is based on the removal and addition of points from an auxiliary point cloud that acts as a structuring element. However, in certain applications within a more complex process, these changes to the original data represent an unacceptable loss of information. The aim of this work is to provide a modification of the morphological opening to retain original points and attributes. The proposed amendment involved in the morphological opening: erosion followed by dilatation. In morphological erosion, the new eroded points are retained. In morphological dilation, the structuring element does not add its points directly, but uses the point positions to search through the previously eroded points and retrieve them for the dilated point cloud. The modification was tested on synthetic and real data, showing a correct performance at the morphological level, and preserving the precision of the original points and their attributes. Furthermore, the conservation is shown to be very relevant in two possible applications such as traffic sign segmentation and occluded edge detection.

Mobile Laser Scanning (MLS) systems have proven their usefulness in the rapid and accurate acquisition of the urban environment. From the generated point clouds, street furniture can be extracted and classified without manual intervention. However, this process of acquisition and classification is not error-free, caused mainly by disturbances. This paper analyses the effect of three disturbances (point density variation, ambient noise, and occlusions) on the classification of urban objects in point clouds. From point clouds acquired in real case studies, synthetic disturbances are generated and added. The point density reduction is generated by downsampling in a voxel-wise distribution. The ambient noise is generated as random points within the bounding box of the object, and the occlusion is generated by eliminating points contained in a sphere. Samples with disturbances are classified by a pre-trained Convolutional Neural Network (CNN). The results showed different behaviours for each disturbance: density reduction affected objects depending on the object shape and dimensions, ambient noise depending on the volume of the object, while occlusions depended on their size and location. Finally, the CNN was re-trained with a percentage of synthetic samples with disturbances. An improvement in the performance of 10–40% was reported except for occlusions with a radius larger than 1 m.

Occlusions accompany serious problems that reduce the applicability of numerous algorithms. The aim of this work is to detect and characterize urban ground gaps based on occluding object. The point clouds for input have been acquired with Mobile Laser Scanning and have been previously segmented into ground, buildings and objects, which have been classified. The method generates various raster images according to segmented point cloud elements, and detects gaps within the ground based on their connectivity and the application of the hit-or-miss transform. The method has been tested in four real case studies in the cities of Vigo and Paris, and an accuracy of 99.6% has been obtained in occlusion detection and labelling. Cars caused 80.6% of the occlusions. Each car occluded an average ground area of 11.9 m2. The proposed method facilitates knowing the percentage of occluded ground, and if this would be reduced in successive multi-temporal acquisitions based on mobility characteristics of each object class.

Traffic signs are a key element in driver safety. Governments invest a great amount of resources in maintaining the traffic signs in good condition, for which a correct inventory is necessary. This work presents a novel method for mapping traffic signs based on data acquired with MMS (Mobile Mapping System): images and point clouds. On the one hand, images are faster to process and artificial intelligence techniques, specifically Convolutional Neural Networks, are more optimized than in point clouds. On the other hand, point clouds allow a more exact positioning than the exclusive use of images. The false positive rate per image is only 0.004. First, traffic signs are detected in the images obtained by the 360° camera of the MMS through RetinaNet and they are classified by their corresponding InceptionV3 network. The signs are then positioned in the georeferenced point cloud by means of a projection according to the pinhole model from the images. Finally, duplicate geolocalized signs detected in multiple images are filtered. The method has been tested in two real case studies with 214 images, where 89.7% of the signals have been correctly detected, of which 92.5% have been correctly classified and 97.5% have been located with an error of less than 0.5 m. This sequence, which combines images to detection-classification, and point clouds to geo-referencing, in this order, optimizes processing time and allows this method to be included in a company's production process. The method is conducted automatically and takes advantage of the strengths of each data type.

Indoor furniture is of great relevance to building occupants in everyday life. Furniture occupies space in the building, gives comfort, establishes order in rooms and locates services and activities. Furniture is not always static; the rooms can be reorganized according to the needs. Keeping the building models up to date with the current furniture is key to work with indoor environments. Laser scanning technology can acquire indoor environments in a fast and precise way, and recent artificial intelligence techniques can classify correctly the objects that contain. The objective of this work is to study how to minimize the use of point cloud samples in Neural Network training, tedious to label, and replace them with images obtained from online sources. For this, point clouds are converted to images by means of rotations and projections. The conversion of a 3D vector data to a 2D raster allows the use of Convolutional Neural Networks, the achievement of several images for each acquired point cloud object and the combination with images obtained from online sources, such as Google Images. The images have been distributed among the validation and testing training sets following different percentages. The results show that, although point cloud images cannot be completely dispensed within the training set, only 10% of these achieve high accuracy in the classification.

Room segmentation is a matter of ongoing interesting for indoor navigation and reconstruction in robotics and AEC. While in robotics field, the problem room segmentation has been typically addressed on 2D floorplan, interest in enrichment 3D models providing more detailed representation of indoors has been growing in the AEC. Point clouds make available more realistic and update but room segmentation from point clouds is still a challenging topic. This work presents a method to carried out point cloud segmentation into rooms based on 3D mathematical morphological operations. First, the input point cloud is voxelized and indoor empty voxels are extracted by CropHull algorithm. Then, a morphological erosion is performed on the 3D image of indoor empty voxels in order to break connectivity between voxels belonging to adjacent rooms. Remaining voxels after erosion are clustered by a 3D connected components algorithm so that each room is individualized. Room morphology is retrieved by individual 3D morphological dilation on clustered voxels. Finally, unlabelled occupied voxels are classified according proximity to labelled empty voxels after dilation operation. The method was tested in two real cases and segmentation performance was evaluated with encouraging results.