Transferring a Segmentation Task Between Real and Synthetic Data

Abstract

In the field of ecology, camera traps are important tools to collect information on the wildlife of certain areas. The problem that arises with many camera traps is that they can collect more images than a human can realistically go trough all by themselves. To help classify these images computer vision is proposed as an alternative to manual classification. Many modern computer vision applications use neural networks. A hard part for the neural networks is that to train them well a large data set is needed, and sometimes it is almost impossible to build this dataset. This is where synthetic samples can be used instead of real samples. These samples are created by using computer graphics software to create realistic looking images to enlarge the dataset, or even be the whole dataset. This work evaluates how well a segmentation network was trained on only synthetic samples could perform on the real data. For this multiple segmentation networks were used like: U-Net and SegNet and the networks were trained on different datasets all derived from the synthetic data. The results show that while the networks can work real images that look similar to the synthetic samples, they fail to segment images that are captured in locations that look different from the synthetic samples.