Print Email Facebook Twitter Generative Adversarial Networks for Shadow Removal to Improve Semantic Segmentation for Autonomous Driving Title Generative Adversarial Networks for Shadow Removal to Improve Semantic Segmentation for Autonomous Driving Author Bruggink, Daan (TU Delft Mechanical, Maritime and Materials Engineering) Contributor Smith, C.S. (mentor) ter Haar, Frank (graduation committee) Piscaer, Pieter (graduation committee) van den Boom, A.J.J. (graduation committee) Degree granting institution Delft University of Technology Programme Mechanical Engineering | Systems and Control Date 2023-02-16 Abstract Traversability estimation is a key component in autonomous driving tasks. In many applications, semantic segmentation is used to pixel-wise classify a visual scene. The pixel-wise segmented map is used to estimate the traversability of different environments. The semantic segmentation accuracy can drop if environmental conditions change. The introduction of shadow in images can cause the segmentation network to misclassify pixels, which leads to aninaccurate traversability estimation. This inaccurate estimation could lead an autonomous vehicle to deviate from traversable paths, leaving it unable to continue or even cause accidents.To increase the segmentation accuracy in shadow conditions, shadow removal before semantic segmentation is proposed. In this research, the supervised Dual Hierarchical Aggregation Network (DHAN) and unsupervised cycle-based Shadow Generative Adversarial Network (SGAN) are used for shadow removal before semantic segmentation with Segmentation Network (SegNet). The shadow removal networks are evaluated on two datasets, containing image triplets, consisting of shadow, shadow-free and shadow-mask images. The structuralsimilarity is calculated for complete images and the non-shadow regions by invertingthe shadow-masks. The networks are applied to the Cambridge-driving Labeled Video Database (CamVid) dataset to evaluate the change in segmentation accuracy. In a second set of experiments, the DHAN shadow removal network is retrained on multiple datasets containing synthetic shadows. The obtained DHAN networks are tested on shadows with increasing intensity. The retrained DHAN networks are compared to evaluate the training dataset for shadow removal to increase segmentation accuracy.The experiments show that the DHAN increases the structural similarity of 99.8% and the SGAN for 74.5% of the datasets. After retraining on the synthetic shadow dataset, segmentation accuracy after DHAN shadow removal increases the segmentation Pixel Accuracy (PA) for a maximum of 91% and the segmentation mean Intersection over Union (mIoU) for a maximum of 88% of the images of the CamVid dataset. We conclude that segmentation accuracy increases after DHAN shadow removal if the DHAN is trained on the synthetic shadow dataset. Subject Generative Adversarial NetworksShadow removalSemantic segmentation To reference this document use: http://resolver.tudelft.nl/uuid:de0a6492-91af-46fe-868d-f0d7c9e2a68e Part of collection Student theses Document type master thesis Rights © 2023 Daan Bruggink Files PDF Master_Thesis_Daan_Bruggi ... riving.pdf 67.16 MB Close viewer /islandora/object/uuid:de0a6492-91af-46fe-868d-f0d7c9e2a68e/datastream/OBJ/view