Semantic Segmentation using Deep Neural Networks for MAVs

Master thesis (2022)

Authors

T.V. Tran Aerospace Engineering

Contributors

G.C.H.E. de Croon (supervisor 1)
Y. Xu (supervisor 1)
C. de Wagter (supervisor 2)
J.C. van Gemert Pattern Recognition and Bioinformatics - EEMCS (supervisor 2)
Faculty
Aerospace Engineering
Copyright: © 2022 Tommy Tran
Deep Learning Recurrent Neural Network Convolutional Neural Network Semantic Segmentation Optical Flow Micro Air Vehicle
More Info
expand_more

Published Date

19-01-2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Aerospace Engineering

Abstract

Semantic segmentation methods have been developed and applied to single images for object segmentation. However, for robotic applications such as high-speed agile Micro Air Vehicles (MAVs) in Autonomous Drone Racing (ADR), it is more interesting to consider temporal information as video sequences are correlated over time. In this work, we evaluate the performance of state-of-the-art methods such as Recurrent Neural Networks (RNNs), 3D Convolutional Neural Networks (CNNs), and optical flow for video semantic segmentation in terms of accuracy and inference speed on three datasets with different camera motion configurations. The results show that using an RNN with convolutional operators outperforms all methods and achieves a performance boost of 10.8% on the KITTI (MOTS) dataset with 3 degrees of freedom (DoF) motion and a small 0.6% improvement on the CyberZoo dataset with 6 DoF motion over the single-frame-based semantic segmentation method. The inference speed was measured on the CyberZoo dataset, achieving 321 fps on an NVIDIA GeForce RTX 2060 GPU and 30 fps on an NVIDIA Jetson TX2 mobile computer.