This paper presents an encoder-decoder-style convolutional neural network (CNN) for the purpose of improving monocular and stereo depth estimation (SDE) estimates, by combining them with the corresponding monocular estimates through a fusion network, assisted by prior information to provide context for the fusion. Video cameras are commonly used for depth perception in robotics, especially weight-sensitive applications, such as on Micro Aerial Vehicles (MAV). The two primary paradigms for vision-based depth perception are monocular and stereo depth or disparity estimation, each having their own strengths and weaknesses. These strengths and weaknesses seem to be complementary, and thus a fusion of the two may result in more accurate predictions. In this paper, we investigate this fusion by training a CNN that combines stereo and monocular depth or disparity estimates. The fusion network is agnostic to the choice of the input networks, providing great flexibility. It was found that such a fusion network, while increasing the computational complexity of the depth perception pipeline, indeed improves the accuracy of the estimates. The number of outlier predictions has been significantly decreased, while also limiting some fundamental limitations of both stereo and monocular methods, such as errors arising from occluded regions. ... Read more

We present a computationally cheap 3D bug algorithm for drones, using stereo vision. Obstacle avoidance is important, but difficult for robots with limited resources, such as drones. Stereo vision requires less weight and power than active distance measurement sensors, but typically has a limited Field of View (FoV). In addition, the stereo camera is fixed on the drone, preventing sensor movement. For obstacle avoidance, bug algorithms require few resources. We base our proposed algorithm, Frustumbug, on the Wedgebug algorithm, since this bug algorithm copes with a limited FoV. Since Wedgebug only focuses on 2D problems, the Local-epsilon-Tangent-Graph (LETG) is used to extend the path planning to 3D. Disparity images are obtained through an optimised stereo block matching algorithm. Obstacles are expanded in disparity space to obtain the configuration space. Furthermore, Frustumbug has an improved robustness to noisy range sensor data, and includes reversing, climbing and descending manoeuvres to avoid or escape local minima. The algorithm has been extensively tested with 225 flights in two challenging simulated environments, with a success rate of 96%. Here, 3.6% did not reach the goal and 0.4% collided. Frustumbug has been implemented on a 20 gram stereo vision system, and guides drones safely around obstacles in the real world, showing its potential for small drones to reach their targets fully autonomously. ... Read more

Self-supervised deep learning methods have leveraged stereo images for training monocular depth estimation. Although these methods show strong results on outdoor datasets such as KITTI, they do not match performance of supervised methods on indoor environments with camera rotation. Indoor, rotated scenes are common for less constrained applications and pose problems for two reasons: abundance of low texture regions and increased complexity of depth cues for images under rotation. In an effort to extend self-supervised learning to more generalised environments we propose two additions. First, we propose a novel Filled Disparity Loss term that corrects for ambiguity of image reconstruction error loss in textureless regions. Specifically, we interpolate disparity in untextured regions, using the estimated disparity from surrounding textured areas, and use L1 loss to correct the original estimation. Our experiments show that depth estimation is substantially improved on low-texture scenes, without any loss on textured scenes, when compared to Monodepth by Godard et al. Secondly, we show that training with an application's representative rotations, in both pitch and roll, is sufficient to significantly improve performance over the entire range of expected rotation. We demonstrate that depth estimation is successfully generalised as performance is not lost when evaluated on test sets with no camera rotation. Together these developments enable a broader use of self-supervised learning of monocular depth estimation for complex environments. ... Read more

Widespread usage of Micro Aerial Vehicles (MAVs) has led to various airspace safety breaches, including near mid-air collisions with other aircraft. To ensure safe integration into general aviation, it is paramount that MAVs are equipped with an autonomous detect and avoid system when flying beyond the visual line-of-sight of the operator. The purpose of this research is to investigate the feasibility of acoustic-based aircraft detection, which has generally been overlooked in favor of optical or radar-based technology. Effective sound-based aircraft detection on-board an MAV requires suppressing the dynamic ego-noise it generates during flight, which would otherwise pollute the recorded environmental sound. This paper proposes using a recurrent neural network to predict the generated noise, given a sequence of MAV flight data, so that it can be effectively removed from noisy recordings. For aircraft detection, a convolutional neural network in combination with Mel spectrogram features is designed to classify noise-free environmental sound as either aircraft or non-aircraft, achieving 97.5\% accuracy. To reconstruct the noisy environment of an MAV flight, these noise-free sounds are mixed with ego-noise at mix ratios up to 1.00. When evaluating in these mismatched conditions, accuracy decreases to 95.0\% and 47.5\% with- and without ego-noise suppression, respectively. Although ego-noise suppression can not prevent a drop in performance, the large difference between the mismatched conditions does demonstrate the benefits of the proposed denoising approach on aircraft detection. ... Read more

The last years there is a wide interest in UAVs which can be attributed to their low cost and wide range of use in recreational, commercial and scientific applications. Despite the large increase in drones, UAV flights are permitted only in secluded areas. In order to be granted access to public areas, it must prove its capacity to sense and safely avoid collisions with other obstacles. Therefore, the need for a secure and reliable CAS is imperative. In this thesis a
low-cost, low computationally demanding, stereo-based, robust CAS solution for small UAVs is designed, assuming flights primarily in an outdoor environment. In order to address this problem, firstly the existing dense stereo open-source algorithms are reviewed based on their suitability for obstacle avoidance and their computational complexity. Based on a semantic evaluation and profiling, the review concludes that BM should be preferred for low-cost obstacle avoidance and SGBM should be used only in highly textureless environments. Subsequently, since the imperfect accuracy of any existing stereo solution is a fact, a machine learning method is introduced in order to predict the uncertainty of the stereo measurements. This so called “uncertainty map” method assigns an uncertainty value to every image pixel. It was shown that it can successfully predict the uncertainties of BM and SGBM stereo algorithms. Furthermore, a low-cost collision avoidance method was proposed which makes use of uncertainty map in sensing filtering, collision detection and path planning. The evaluation showed that the use of uncertainty map improves
both collision detection and path planning, especially when BM is used. Last but not least, the whole CAS was implemented in an embedded system Raspberry Pi 3 model B+. Results show that real-time execution of the propsoed CAS with a runtime frequency of 4-54 Hz is possible when BM is used and 1-16 Hz when SGBM is used. ... Read more

We investigate how an Unmanned Air Vehicle (UAV) can detect manned aircraft with a single microphone. In particular, we create an audio data set in which UAV ego-sound and recorded aircraft sound can be mixed together, and apply convolutional neural networks to the task of air traffic detection. Due to restrictions on flying UAVs close to aircraft, the data set has to be artificially produced, so the UAV sound is captured separately from the aircraft sound. The aircraft data set is collected at Lelystad airport by capturing flyovers with a microphone array. It is mixed with UAV recordings, during which labels are given indicating whether the mixed recording contains aircraft audio or not. The mixed recordings are the input for a model that determines whether an aircraft is present or not. The model is a CNN which uses the features MFCC, spectrogram or Mel spectrogram as input. For each feature the effect of UAV/aircraft amplitude ratio, the type of labeling, the window length and the addition of third party aircraft sound database recordings is explored. The results show that the best performance is achieved using the Mel spectrogram feature. The performance increases when the UAV/aircraft amplitude ratio is decreased, when the time window is increased or when the data set is extended with aircraft audio recordings from a third party sound database. It is not desirable to train the model on distant approaches and test them on nearby approaches as the performance then drops. The results also prove that the performance increases the closer the aircraft is. Although the currently presented approach has a number of false positives and false negatives, that is still too high for real-world application, this study indicates multiple paths forward that can lead to an interesting performance. In addition, the data set is provided as open access, allowing the community to contribute to the improvement of the detection task. ... Read more