Addressing Illumination-Based Domain Shifts in Deep Learning

Addressing Illumination-Based Domain Shifts in Deep Learning: A Physics-Based Approach

Lengyel, Attila (TU Delft Electrical Engineering, Mathematics and Computer Science)

van Gemert, Jan (mentor)
Reinders, Marcel (graduation committee)
Hildebrandt, Klaus (graduation committee)
Milford, Michael (mentor)

Delft University of Technology

2019-08-26

This work investigates how prior knowledge from physics-based reflection models can be used to improve the performance of semantic segmentation models under an illumination-based domain shift. We implement various color invariants as a preprocessing step and find that CNNs trained on these color invariants get stuck in worse local minima compared to RGB inputs, but can achieve comparable or even superior performance when applying knowledge transfer from RGB. We also find Batch Normalization to severely affect the performance of neural networks under an illumination-based domain shift and demonstrate that Instance Normalization offers a simple remedy to this issue. Additionally, we investigate different fusion models for combining color invariants with RGB. Using a combination of these methods we achieve a 14.5% performance increase on nighttime semantic segmentation without any additional training data.

Semantic segmentation
color invariants
deep learning
computer vision
domain adaptation

http://resolver.tudelft.nl/uuid:f8619273-0e7e-42e3-990b-67e2f6edc78a

Student theses

master thesis

© 2019 Attila Lengyel