EV-Mask-RCNN: Instance Segmentation in Event-based Videos

Bachelor thesis (2022)

Authors

A. Băltăreţu Electrical Engineering, Mathematics and Computer Science

Contributors

N. Tömen Pattern Recognition and Bioinformatics - EEMCS (mentor)
O. Strafforello Pattern Recognition and Bioinformatics - EEMCS (mentor)
X. Liu Pattern Recognition and Bioinformatics - EEMCS (mentor)
L. Cavalcante Siebert Interactive Intelligence - EEMCS (graduation committee member)
Faculty
Electrical Engineering, Mathematics and Computer Science, Electrical Engineering, Mathematics and Computer Science
Copyright: © 2022 Ana Băltăreţu
More Info
expand_more

Published Date

22-06-2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Instance segmentation on data from Dynamic Vision Sensors (DVS) is an important computer vision task that needs to be tackled in order to push the research forward on these types of inputs. This paper aims to show that deep learning based techniques can be used to solve the task of instance segmentation on DVS data. A high performing model was used to solve this task, using event-based data that was transformed into RGB-D images. The chosen model for this work was Mask R-CNN, with an alteration for depth images, because of its high performance on frame based data. The N-MNIST dataset provides the event-based input, and the transformation of such an input is presented in this study. Furthermore, the masks are generated with the help of the MNIST dataset and heuristics are used for placing them at the correct positions. The results are promising and comparable to other results from literature on the task of semantic segmentation.