Unmanned Aerial Vehicles (UAVs) play a crucial role in various applications, including disaster response, infrastructure inspection, and search-and-rescue missions. To maximise their effectiveness, UAVs must achieve a high level of autonomy, particularly when navigating cluttered environments. This requires real-time collision avoidance algorithms that efficiently utilise on-board computational resources. Existing approaches either maintain explicit map representations, which provide memory of past observations but require significant computation, or directly process depth images, which are computationally efficient but restrict flight paths to the sensor’s field of view (FOV). This thesis presents a novel, lightweight collision avoidance algorithm that operates directly on depth images while retaining memory of past observations. By aggregating previous depth data, the algorithm enables UAVs to plan trajectories beyond the FOV of onboard sensors without the computational overhead of explicit map maintenance. Experimental results demonstrate that the proposed method outperforms existing state-of-the-art methods in terms of computational efficiency, flight speed, energy cost, and path length until collision.

The goal of this thesis was to push the boundaries of fast and agile flight for autonomous drone racing. A framework was developed for high-performance vision-based localization, sensor fusion, and control, with a focus on integration of neural networks. Additionally, a neural dynamics model was created which can can replace all accelerometer measurements by thrust&drag predictions, while running directly on the flight controller.

The estimation of optical flow, which determines the movement of objects in a visual scene, is a crucial problem in computer vision. It is essential for applications such as autonomous navigation, where precise motion estimation is critical for performance and safety.

Frame-based cameras capture sequences of still images at regular intervals, from which optical flow is traditionally extracted using optimization-based or learning-based methods. Recently, event-based cameras, which detect changes in pixel brightness asynchronously, have gained traction due to their high temporal resolution and robustness to motion blur, and many algorithms have been developed to estimate optical flow from this data. IDNet is a learning-based approach that achieves state-of-the-art performance. However, IDNet and similar models face two major challenges: they require labeled ground-truth data for training, which is scarce and difficult to collect, and they rely on recurrent neural networks (RNNs) with a fixed number of refinement iterations. This fixed iteration scheme does not adapt to scene complexity, limiting accuracy for complex flows and increasing computational effort for simpler patterns.

The aim of this project is to explore, implement, and evaluate potential methods to address these two mentioned limitations and enhance the capabilities of models like IDNet.

To remove the need for ground-truth data, a self-supervised learning paradigm was implemented by introducing a novel contrast maximization loss that assesses the blur present when accumulating raw events for a certain time interval and compensating it with the predicted flow. To assess the effectiveness of this method, models were trained on the benchmark MVSEC dataset, showing improved results over previous methods with up to 15% on some sequences and an 8% improvement on average. Based on these experiments and results, further research directions were proposed.

As for the problem of the current fixed iteration scheme, Deep Equilibrium Models were found to provide a promising pathway to solving it. These novel models reformulate their iterative structure into a root-finding problem and utilize traditional solvers to find a solution based on some tolerance, providing a trade-off between speed and accuracy. Moreover, they allow for direct differentiation through the network using only their final estimate, compared to previous methods that keep track of their state through all iterations, leading to an O(1) memory consumption. Implementing these and some additional ideas, the trained DEQ IDNet model reached competitive performance on the DSEC dataset while consuming 15% less memory. Yet, further work is needed to close the gap and achieve state-of-the-art performance.

Olfactory learning in Drosophila larvae exemplifies efficient neural processing in a small-scale network with minimal power consumption. This system enables larvae to anticipate important outcomes based on new and familiar odor stimuli, a process crucial for survival and adaptation. Central to this learning mechanism is the olfactory pathway model, which embodies the principles of synaptic plasticity and associative learning through prediction error coding mediated by specific neuromodulating neurons in the mushroom body, like dopaminergic neurons. There is a pressing need to develop novel computational frameworks that capture the spatio-temporal processes while remaining compatible with the constraints of small-scale neural networks. These frameworks should draw inspiration from the biophysical properties of neurons within the olfactory pathway model, enabling accurate emulation of neural dynamics and efficient learning processes using spiking neural networks. This thesis proposes a framework based on a phenomenological conductance-based leaky integrate-and-fire (COBALIF) neuron model, inspired by the olfactory pathway model of Drosophila larvae. By first prototyping the spiking neural network in Intel's Lava Python-based framework, we validated the design on a neuron and system level for a neuromorphic hardware implementation. This was the foundation of a programmable, neuromorphic FPGA architecture capable of adaptive optimization, employed on a Zynq 7000 SoC FPGA. By implementing this architecture in a single-precision floating-point format, we model the real-time neural dynamics of the COBALIF neuron in one-tenth of a millisecond precision. Moreover, our FPGA implementation serves as a feasible prototype for deploying such biologically inspired neurons and their spatio-temporal dependencies in digital design, paving the way for scaling up to small-scale networks.

The effects of climate change put increasing strain on rainforests and their inhabitants, highlighting the demand for technological developments to aid in biodiversity monitoring and conservation efforts. This research work proposes a novel frame work for the autonomous placement of acoustic sensor networks on top of the rainforest canopy, using a quadrotor platform. In order to tackle the challenge posed by multirotors’ limited autonomy, an initial canopy exploration mission is considered and an exploration planner is developed to detect suitable locations on top of the rainforest canopy for sensor node deployment. Specifically, green detection and pointcloud projection is performed online and combined with our proposed sampling method to accomplish targeted exploration, towards the estimated location of detected green components. Flight experiments are conducted across multiple scenarios that mimic distinct rainforest features. The results validate the system architecture and demonstrate the effectiveness of our smart sampling method, laying the foundation for future autonomous sensor network exploration missions at larger-scale.

Neuromorphic sensors, like for example event cameras, detect incremental changes in the sensed quantity and communicate these via a stream of events. Desired properties of these signals such as high temporal resolution and asynchrony are not always fully exploited by algorithms that process these signals. Spiking neural networks (SNNs) have emerged as the algorithms that promise to maximally attain these characteristics and are likely the key to achieving a fully neuromorphic computing pipeline. But, this means that if the SNN is to take full advantage, the event stream must be sent directly and unaltered to the SNN, which in turn implies that all temporal integration should occur inside the SNN. Therefore, it is interesting to investigate the mechanisms that achieve this. This thesis does so through evaluating and comparing the performance of different memory mechanisms in SNNs found in the literature, as well as through an in depth analysis of the inner workings of these mechanisms. The mechanisms include spiking neural dynamics (leaks and thresholds), explicit recurrent connections, and propagation delays. We demonstrate our concepts on two small scale generated 1D moving pixel tasks in preliminary experiments first. After that, we extend our research to compare the memory mechanisms on a real-world neuromorphic vision processing task, in which the networks regress angular velocity given event based input. We find that both explicit recurrency and delays improve the prediction accuracy of the SNN, compared to having just spiking neuronal dynamics. Analysis of the inner workings of the networks shows that the threshold and reset mechanism of spiking neurons play an important role in allowing longer neuron timescales (lower membrane leak). Forgetting (at the right time) turns out to play an important role in memory. Additionally, it becomes apparent that optimizing an SNN with explicit recurrent connections or learnable delays does not lead to the formation of robust spiking neuronal dynamics. In fact, spiking neuronal dynamics are largely ignored, as after optimization virtually no input current is integrated onto the membrane potential in these cases. Instead, we consistently find that a recurrent SNN prefers to build a state solely with the explicit recurrent connections, while an SNN with delays prefers to just use the delays. Therefore, our SNNs with explicit recurrent connections and delays are in fact better described as binary activated RNNs and ANNs, respectively.

Aerial manipulators, characterized by their ability to actively engage with the environment, are gaining popularity for their versatility in performing diverse tasks.
This research focuses on augmenting the capabilities of aerial manipulators through the integration of tactile feedback, specifically employing a compliant bio-inspired three-fingered manipulator equipped with tactile capacitive sensors on each finger. The manipulator is affixed to a drone, enabling tactile-guided navigation for precise object localization, subsequent grasping, and perching. Additionally, a grasp evaluator assesses grasp quality, allowing the system to adapt by suggesting alternative grasp locations after an initial attempt is unsuccessful. A comparative analysis between the system’s performance using tactile feedback and open-loop perching/grasping in perching scenarios demonstrates that the grasp evaluator improves the perching success rate by 55%-point and increases the allowable object uncertainty by 0.14 [m]. These findings highlight the efficacy of this approach in advancing aerial manipulator capabilities.

Flapping wing micro aerial vehicles (FWMAVs) are known for their flight agility and maneuverability. However, their in-gust flight performance and stability is still inferior to their biological counterparts. To this end, a simplified in-gust dynamic model, which could capture the main gust effects on FWMAVs, has been identified with real in-gust flights' data of a FWMAV, the Flapper Drone. Based on this model, an adaptive position and velocity controller was proposed with gain scheduling and implemented for in-gust flights under gust speeds up to 2.4 m/s. With this airflow-sensing based adaptive controller, the in-gust hovering stability of the Flapper Drone has been improved when the gust's intensity and frequency changes, comparing with the original fixed-gain cascaded PID controller case.

In an effort to develop a new relative sensing method for drone swarms, the suitability of event cameras is assessed for propeller detection. Benchmark tests were conducted for different propellers under different lighting and background conditions, varying the observation distance and spinning frequency. The different tests were evaluated on event count, frequency, and clustering, as these are the most characteristic properties of the propeller-generated signal. A propeller detection metric was derived as a fuzzy classifier to assess detectability. It was observed that the sensor employed is limiting the detection range due to low resolution, with a maximum detection range of 75 cm. While at low spinning frequencies it is possible to detect the propeller at such distance, for higher frequences (6000 to 8000 RPMs) the range decreases to 45 cm for the tests with highest blade to background contrast and two-blade propellers. It was observed that lower contrasts reduce the successful detections only to low frequencies, and three-blade propellers become completely indetectable due to the static overlap between the blades. Therefore, it is concluded that, at this stage of the technology, the use case of event cameras for relative sensing is constrained to close distances with high contrast.

Flow sensing exists widely in nature to help animals perform certain tasks. It has also been widely adopted in engineering applications with different types of sensing instrumentation. In particular, in the field of aerospace engineering, airflow sensing is crucial to vehicle state evaluation and flight control. This project surveys the key mechanisms from biological features in nature that enable flow sensing and expands towards the application motivation to identify a suitable airflow sensor that can be equipped to a flapping wing micro air vehicle (FWMAV) for onboard airflow sensing.

The selection of sensors is first narrowed down to three major types of airflow sensors from the state of art that have the most potential to be integrated onboard a flapping wing MAV, considering the sensor performance need, size, weight and power (SWaP) restrictions. Two thermal-based commercially available low-cost airflow sensors RevP and RevC from Modern Device have been selected after the trade-off analysis.

A full workflow of calibrating and evaluating the two airflow sensors' directional sensitivity has been carried out through two wind tunnel campaigns. Their performance under grid-generated turbulence is compared with a constant temperature hot-wire anemometer. This series of tests leads to the conclusion that the RevP airflow sensor has better performance and is therefore chosen to be placed onboard a flapping wing MAV Delfly Nimble.

Both mounted tests and tethered hovering tests with the Delfly Nimble are performed to further examine the airflow sensor RevP's measurement performance under different influence factors such as MAV throttle levels, MAV body pitch angles and freestream speeds. In the end, it is concluded that as a proof of concept, the RevP sensor is capable of performing effective measurements for low flow speeds less than 4 m/s, within the pitching angle range of -30 to 30 degrees. Although this is the first achieved tethered hover flight with onboard airflow sensing for a flapping wing MAV, its limited payload and onboard power supply demands an even smaller and less power consuming design of airflow sensors to enable further applications such as autonomous reactive control under wind disturbances.

Aircraft with disruptive designs have no high-fidelity and accurate flight models. At the same time, developing models for stochastic phenomena for traditional aircraft configurations are costly, and classical control methods cannot operate beyond the predefined operation points or adapt to unexpected changes to the aircraft. The Proximal Policy Option Critic (PPOC) is an end-to-end hierarchical reinforcement learning method that alleviates the need for a high-fidelity flight model and allows for adaptive flight control. This research contributes to the development and analysis of online adaptive flight control by comparing PPOC against a non-hierarchical method called Proximal Policy Optimization (PPO) and PPOC with a single Option (PPOC-1). The methods are tested on an extendable mass-spring-damper system and aircraft model. Subsequently, the agents are evaluated by their sample efficiency, reference tracking capability and adaptivity. The results show, unexpectedly, that PPO and PPOC-1 are more sample efficient than PPOC. Furthermore, both PPOC agents are able to successfully track the height profile, though the agents learn a policy that results in noisy actuator inputs. Finally, PPOC with multiple learned Options has the best adaptivity, as it is able to adapt to structural failure of the horizontal tailplane, sign change of pitch damping, and generalize to different aircraft.

An idea was proposed to allow an autonomous drone to have indefinite flight times over the ocean by applying renewable energy technologies and theory to generate electricity in flight. This is considered less as a way to save energy, but to permit the use of such a drone from a ship not capable of safely retrieving it. One novel component of this idea is to use the wind updraft created by the motion of a ship or natural air currents as the wind source for an on-board turbine generator. The second component is to use the existing drive system as the on-board turbine in a 'hybrid rotor' design to reduce the need for extra parts and complexity. This report analyzes the potential for such a system compared to a more intuitive airborne solar system, and to the combination of both concepts. While indefinite flight time is paramount, the goal is to maximize the "mission" time to charge/idle time ratio. The process for determining fitness is a simulation of the aircraft flying on its mission and charging when needed (and if possible) for a full year for varying designs of aircraft and rotor. The results of all the tests show that the main idea is infeasible because not enough energy can be generated from the inefficient propeller and the updrafts are insufficient and inconsistent. The alternatives of solar and combined power systems function better but are still subject to high failure rates. The most promising system is to use a separate turbine and propeller and also include solar panels to achieve the most effectiveness both when in powered flight and while charging. This constitutes a compromise on the 'hybrid rotor' part of the idea. The conclusion of this report is that further improvements to the design and control of the most successful configuration are possible could result in a fully functional system.

Micro robotic airships offer significant advantages in terms of safety, mobility, and extended flight times. However, their highly restrictive weight constraints pose a major challenge regarding the available computational power to perform the required control tasks. Thus, spiking neural networks (SNNs) are a promising research direction. By mimicking the biological process for transferring information between neurons using spikes or impulses, they allow for low power consumption and asynchronous event-driven processing. In this work, we propose an evolved altitude controller based on a SNN for an airship which relies solely on the sensory feedback provided by an airborne radar sensor. Starting from the design of a a lightweight, low-cost, open-source airship, we also present a low-control-effort SNN architecture, an evolutionary framework for training the network in a simulated environment, and a control scheme for ameliorating the performance of the system in real-world scenarios. The system's performance is evaluated through real-world experiments, demonstrating the advantages of our approach by comparing it with an artificial neural network (ANN) and a linear controller (PID). The results show an accurate tracking of the altitude command while ensuring efficient management of the control effort. The main contributions of this work are presented in the scientific paper, corresponding to Part I of the document. Besides the research on altitude control based on SNNs and their comparison with an ANN and a PID, this thesis includes an in-depth review of the relevant literate on the main topics covered, in Part II. Finally, a detailed explanation of the methodologies used, the conclusions and recommendations for future work are proposed in Part III.

Current state-of-the-art airline planning models are required to decrease models either in size or complexity due to computational limitations, limiting the
operational applicability to problems of representative sizes. Models return suboptimal solutions, especially when confronted with factors of uncertainty. Considering the growing interest in the application of Machine Learning techniques in the Operations Research domain, and the proven success in other fields such as robotics, this research investigates the applicability of these techniques for airline planning. An Advantage Actor-Critic (A2C) Reinforcement Learning agent is applied to the airline fleet planning problem. Because of the increased computational efficiency of using an A2C agent, the problem is increased in size and the highly volatile uncertainty in fuel price is implemented.
Conversion was achieved, and when evaluating the quality of the solutions compared to a deterministic model, the performance was very satisfactory. The A2C agent was able to outperform the deterministic model, with an increasing performance as more complexity was added to the problem. It was found that
the introduction of additional uncertainty has a major effect on the optimal actions, which the agent was able to adapt to adequately.

The application of Unmanned Aerial Vehicles (UAVs) is increasing, much like the performance of these aircraft. A tailsitter is a type of UAV which is capable of performing vertical take-offs and landings (VTOL) and long endurance flights. During hover, the yaw control is limited due to the dynamics of these tailsitters. The generally used quaternion feedback for the attitude does not compensate for this as it describes a singular rotation. Tilt-twist is a solution to the problem as it splits the tilt (pitch and roll) from the twist (yaw). The axis of the yaw rotation is body fixed. When hovering with a pitch and/or roll angle the twist axis will be aligned with the body z-axis, instead of the desired gravitational force vector (for position control). Previous tilt-twist methods used a PID controller. This paper describes an improvement over previous tilt-twist approaches, the dynamic tilt-twist in combination with INDI. The INDI controller is designed for nonlinear systems. The dynamic tilt-twist compensates for the problem with the normal tilt-twist as test results will demonstrate. Tests are performed in a simulation and a real life test with the NederDrone hybrid tailsitter is done.

Flapping wing micro air vehicles (FWMAV's) are a subcategory of unmanned aerial vehicle which use flapping wings for thrust generation. The high agility and maneuverability of FWMAV's are very favorable attributes, making them more applicable in cluttered spaces. A tailless FWMAV called the Delfly Nimble has been developed at the Delft University of Technology. Due to the inherent instability of the tailless design an active controller is required to ensure safe and stable flight of the drone. In previous research, models have been developed for the longitudinal dynamics of the Delfly Nimble. In this paper, a grey-box state-space model of the lateral body dynamics in hover conditions is identified using system identification techniques. The parameters which needed to be estimated were stability and control derivatives, and they were obtained with a least-squares approach. Free-flight experiments were performed to generate the identification and validation data. A doublet train was used in the identification experiments, with the gains of the controller adjusted in such a way that maximum excitation was acquired. The identified model has been validated with various maneuvers. These included doublets, 112-maneuvers, maneuvers using coupled inputs, and maneuvers with sideways flight. The resulting model is able to predict the state derivatives of most maneuver accurately, reaching accuracies of over 90% for maneuvers close to hover. Moreover, in closed-loop configuration it is able to simulate the state response accurately, with accuracies of over 85% for maneuvers close to hover, and remains stable, making it applicable for controller design and stability analysis. Finally, based on the model the inherent instability of the lateral body dynamics was also confirmed, for there are eigenvalues with positive real parts.

Passive acoustic sensing utilizes the ability of sound to travel beyond the line-of-sight to understand the surroundings. This provides an advantage over the currently used sensors in Intelligent Vehicles that can sense obstacles within their line-of-sight only. Recently, a localization based approach has been implemented to take advantage of this sensing modality to predict approaching vehicles behind the blind corner in an urban scenario. While this approach shows a lot of promise, there is a difficulty in integrating the multi-microphone system. Additionally, the system would be unable to differentiate between the nature of two sound sources. This motivates the exploration of a classification based approach which uses audio data from only a single microphone to identify the sound sources present in them. This thesis investigates the possibility of having such a system on the Intelligent Vehicle to predict approaching vehicles from behind the blind corners. A review of the literature revealed that techniques categorized under Sound Event Detection (SED) are suitable to implement a classification based approach. The prediction of the vehicle is treated as a binary classification problem and a Convolutional Recurrent Neural Network (CRNN) is used as the acoustic model to detect the presence of an approaching car in the audio sample represented by Log Mel Spectrogram features. Additionally, domain adaptation techniques were implemented to explore the possibility
of improving the system performance with limited data collected while the ego-vehicle is driving. Experiments carried out indicate that when the ego-vehicle is static, the system performs well with the approaching vehicle predicted 1.4s before it is in line-of-sight and a balanced accuracy of 86.9% achieved for the classification task. However, the system achieved an accuracy of 68% on the samples recorded while the ego-vehicle was driving. Further experiments indicate that the acoustic model cannot generalize well to unseen situations in most cases and experiment with domain adaptation did not show
any improvement in performance.

Increasing endurance is a major challenge for battery-powered aerial vehicles. A method is presented which makes use of an updraft around obstacles to decrease the power consumption of a fixed-wing, unmanned aerial vehicle. Simulatory results have shown the conditions that the flight controller can fly in.
The effect of a change in wind velocity, wind direction and updraft has been analysed. The simulations showed that an increase in either updraft or absolute wind direction decrease the throttle consumption.
A change in wind velocity results in a shift of the flight controller’s boundaries. The simulations achieved sustained flight at 0 per cent throttle. The practical, autonomous tests reduced the average throttle down to 4.5 per cent in front of the boat. The unfavourable wind conditions and inaccuracies explain this minor
throttle requirement during the final experiment.

Online Reinforcement Learning is a possible solution for adaptive nonlinear flight control. In this research an Adaptive Critic Design (ACD) based on Dual Heuristic Dynamic Programming (DHP) is developed and implemented on a simulated Cessna Citation 550 aircraft. Using an online identified system model approximation, the method is independent of prior model knowledge. The agent consists of two Artificial Neural Networks (ANNs) which form the Adaptive Critic Design and is supplemented with a Recursive Least Squares (RLS) online model estimation. The implemented agent is demonstrated to learn a near optimal control policy for different operating points, which is capable of tracking pitch and roll rate while actively minimizing the sideslip angle in a faster than real-time simulation. Providing limited model knowledge is shown to increase the learning, performance and robustness of the controller.

Vision based control allows Micro Air Vehicles (MAV) to move autonomously in GPS-denied environments, for example in indoor applications. An open issue in this field is landing on an unknown platform. The difficulty in visual control w.r.t. such an unknown platform, is a lack of scale. Without knowledge of the scale of offsets and object sizes (without height knowledge from GPS) it is difficult to determine an appropriate response from the controller. A control algorithm is designed to fit these requirements using an adaptation of an optical flow divergence based landing scheme, combined with an Image Based Visual Servoing approach applied to features in the Virtual Camera. The approach leads to satisfactory behavior in Gazebo simulations. It results in a robust controller for a range of starting heights and divergence settings.