Hybrid Semantic Mapping for Autonomous Off-Road Driving

Master thesis (2022)

Authors

R. Bos Mechanical Engineering

Contributors

C.S. Smith Team Carlas Smith - Mechanical, Maritime and Materials Engineering (supervisor 1)
Frank ter Haar TNO (supervisor 2)
Pieter Piscaer TNO (supervisor 2)
Faculty
Mechanical Engineering
Copyright: © 2022 Roel Bos
Deep Learning Convolutional Neural Networks (CNNs) Semantic Segmentation Bayesian Neural Networks (BNNs)
More Info
expand_more

Published Date

16-03-2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical Engineering

Abstract

Unmanned Ground Vehicle (UGV) navigation in unstructured off-road environments can benefit from accurate traversability estimation. Often, experiments with UGVs use semantic segmentation networks for visual scene understanding. Based on the pixel-wise classification of a semantic segmentation network, the UGV can distinguish traversable from non-traversable terrain. However, it is still an open challenge to design a model which is able to accurately estimate traversability in a variety of environments. Variation in terrain characteristics and different levels of structuredness requires a model with a high level of generalisability. Limited generalisability will result in inaccurate traversability estimation, which in the worst-case scenario can cause the UGV to crash.

In order to overcome limited generalisability, a hybrid semantic segmentation framework is presented that can switch between different operation modes. The hybrid framework contains multiple environment-specific segmenters. For each input frame, the hybrid framework selects an environment-specific segmenter, based on a decision parameter. In this work, two hybrid frameworks containing different decision parameters are designed. The first hybrid framework contains multiple Bayesian segmenters, which quantifies prediction uncertainty in addition to the pixel-wise classification. This uncertainty quantification is obtained by Monte Carlo sampling to generate a posterior distribution of pixel class labels. The second hybrid framework consists of multiple environment-specific segmenters and autoencoders. Every segmenter has a corresponding autoencoder trained on the same environmental dataset. The output of the environment-specific autoencoder is a reconstructed image of the input image. The error between the original input image and the reconstructed image is used as a decision parameter for selecting the best performing segmenter.

We experimented with a hybrid segmentation framework and observed that it could outperform a single semantic segmentation network with a 2.6% Intersection over Union increase. The hybrid framework with the autoencoder approach resulted in a model selection precision of 99.3% on all the test images. Therefore, we can conclude that UGV navigation can benefit from a hybrid semantic segmentation framework.