Addressing Illumination-Based Domain Shifts in Deep Learning

A Physics-Based Approach

Master thesis (2019)

Authors

A. Lengyel Electrical Engineering, Mathematics and Computer Science

Contributors

J.C. van Gemert Pattern Recognition and Bioinformatics - EEMCS (supervisor 1)
M.J.T. Reinders Pattern Recognition and Bioinformatics - EEMCS (supervisor 2)
K.A. Hildebrandt Computer Graphics and Visualisation - EEMCS (supervisor 2)
Michael Milford Queensland University of Technology (supervisor 1)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2019 Attila Lengyel
More Info
expand_more

Published Date

26-08-2019

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

This work investigates how prior knowledge from physics-based reflection models can be used to improve the performance of semantic segmentation models under an illumination-based domain shift. We implement various color invariants as a preprocessing step and find that CNNs trained on these color invariants get stuck in worse local minima compared to RGB inputs, but can achieve comparable or even superior performance when applying knowledge transfer from RGB. We also find Batch Normalization to severely affect the performance of neural networks under an illumination-based domain shift and demonstrate that Instance Normalization offers a simple remedy to this issue. Additionally, we investigate different fusion models for combining color invariants with RGB. Using a combination of these methods we achieve a 14.5% performance increase on nighttime semantic segmentation without any additional training data.