Detecting Empty Wireframe Objects on Micro-Air Vehicles

Abstract

Autonomous MAV are an emerging technology that supports a wide range of applications such as medical delivery or finding survivors in disaster scenarios. As flying in such missions is difficult the robust estimation of an MAV's state within its environment is crucial to ensure safe operation. In indoor scenarios, cameras are one of the predominant choices for state estimation sensors. This requires Computer Vision algorithms to interpret the obtained high dimensional signal. An application that allows the competitive evaluation of control and state estimation algorithms is MAV Racing such as the IROS 2018 Autonomous Drone Race. Thereby a race court consisting of several race gates has to be followed. For a fast flight during such a race court the detection of the racing gates with a camera can be used in a high level control loop. As these objects consist only of small structures that are spread across large parts of the image, this gives rise to a challenging Object Detection problem. In recent years CNN showed promising results on various vision tasks. However, due to their computational complexity the deployment on mobile devices remains a challenge. Furthermore, CNN typically require a vast amount of training data. Finally, the objects typically studied in Object Detection consist of solid and complex features which is not the case for racing gates. Therefore, this work defines the class of EWFO and studies their detection on MAV with YoloV3. Thereby, the training data is created with a graphical engine. We are interested in how to detect EWFO with a CNN on a MAV, using synthetic data. We conduct several simple experiments about EWFO in simulation and compare their detection to more filled objects. Subsequently experiments in a more challenging environment such as an MAV race are conducted. The experiments show how EWFO are harder to detect than filled objects as the detector can be confused to patterns present in the empty part. Particularly for larger objects the detection performance decreases. We give several recommendations on how to generate data for the detection of EWFO on MAV. These include how to add variations in background as well as the camera placement. Finally, we study the incorporation of image augmentation techniques to transfer the detector to the real world. We can report that especially modelling lens distortion improves the performance on the real data. Nevertheless, a reality gap remains that can not fully be explained. Furthermore, different architectures are studied for the detection of EWFO. It can be seen how a relatively shallow network of 9 layers can be used for the detection of EWFO on MAV. A further reduction in weights leads to a gradual decrease in performance. Based on the gained insights the deployment of a detector on the example system JeVois is studied. A detection performance/speed trade-off is evaluated. The final detector achieves 32% average precision at a frame rate of 12 Hz on a real world test set created during this work. The gained insights can be used to deploy the detector in a control loop for MAV. This ensures the safe flight through a racing court of an autonmous drone race. The gained insights about the detection of EWFO can be transferred to objects with similar properties