Ensuring the reliability of machine learning models in safety-critical space missions remains a significant challenge, especially when ground-truth data is unavailable for real-time validation. While machine learning can augment pipelines by extracting transmission spectra from complex light curves, vulnerabilities to instrument anomalies and domain shifts introduce risks. This study evaluates a modular safety cage architecture operating as a parallel monitoring layer to assess prediction validity without modifying the underlying estimator. By monitoring runtime indicators, including uncertainty quantification, out-of-domain detection, and influence functions, the framework constrains the operational domain to a verified region. The results demonstrate that model failure is multifaceted, requiring indicator fusion strategies, and that applying safety-driven rejection allows for a small reduction in data coverage leading to a significant reduction in prediction error. In addition, the framework is extended with an error-aware model specialisation pipeline that partitions the parameter space to deploy specialised local experts. ... Read more

There is an increasing interest in Event-based cameras for computer vision purposes. These cameras provide benefits such as high temporal resolution and high dynamic range by capturing the change in brightness instead of the absolute brightness signal. In order to train learning-based behaviours with event-based cameras, it is essential to be able to simulate the output of these cameras in a virtual world. Unlike their traditional frame-based counterparts, there are no real-time event-based camera rendering engines yet. All existing methods of generating events from virtual scenes impose a significant computational burden, simulating events by generating frames at a very high temporal resolution and converting these frames to events. This thesis proposes a novel method of event generation, by continuously interpolating between frames at a low temporal resolution, using auxiliary data from the rendering engine, such as depth and optical flow. In this method, the backwards pixel sampling problem is solved using a triangle grid created between all pixels, that is used to model the continuous response of each pixel between rendered frames. Other than previous methods that used VFI to temporally upsample LFR frames for event generation, this method does not create intermediate frames, instead simulating events directly from the continuous model. This method is implemented on the CPU, where it is shown to correctly interpolate the signal between frames. It is also implemented in Unreal Engine, where it is used to generate events in near real-time. ... Read more

Deep neural networks are increasingly used for ego-motion estimation. Often, in self-supervised ego-motion networks, it is decoded from a depth network and until now has not been decoded from an optical flow network. This is surprising given the tight relationship between optical flow and ego-motion. While both representations are widely used in learning-based approaches, the extent to which their latent space encodes motion information remains poorly understood. This paper presents a controlled analysis of how well depth-based and optical flow-based neural networks encode ego-motion. Using supervised depth, flow and pose networks trained on TartanAirV2, we probe motion information by attaching identical minimal pose decoders to frozen encoders. Representational space analysis through Centered Kernel Analysis (CKA) and feature space analysis through Principal Component Analysis (PCA) are used to examine how motion information is structured across network hierarchies. This work shows that optical flow representations encode ego-motion information more explicitly, in a lower dimensional, linearly accessible structure as opposed to depth representations, which exhibit weak alignment with pose. These findings suggest that in a self-supervised setting, ego-motion estimation can best be decoded from an optical flow network as opposed to a depth network. ... Read more

Vision is central to autonomous flight. From crop monitoring and infrastructure inspection to search-and-rescue and defense, the ability to perceive and navigate a complex environment in real time determines what a drone can do and where it can go. Current computer vision achieves impressive results, but at considerable cost: deep neural networks on graphics processors consume tens of watts and add hundreds of grams. For larger drones, this limits flight time and on-board intelligence. For the smallest platforms, i.e., those light enough to safely operate near people, navigate collapsed buildings, or monitor ecosystems in massive swarms, it is prohibitive. Biology suggests this trade-off is not inevitable: insects navigate complex environments with brains consuming microwatts, relying on sparse, event-driven neural computation rather than dense, synchronous processing.
This dissertation investigates how event-based vision can enable effective autonomous behavior on flying robots. Event cameras, inspired by the biological retina, respond asynchronously to brightness changes rather than capturing full frames, offering microsecond temporal resolution and high dynamic range at milliwatts of power. Neuromorphic processors extend this paradigm to computation through spiking neural networks that communicate via discrete spikes. Together, they promise a fully event-driven vision pipeline whose efficiency gains would benefit robots of any size. This dissertation targets the hardest case, asking: how can a flying robot learn to navigate autonomously when it only has event-based vision? Four research questions decompose this problem into a systematic progression from learning to deployment, adaptation, and sensor minimalism.
Chapter 2 establishes that spiking neural networks can learn complex visual tasks, presenting the first deep spiking networks to solve dense optical flow estimation from events in a self-supervised manner, moving beyond simple classification to real-world vision problems relevant for robotics. Chapter 3 takes the next step by deploying a fully neuromorphic vision-to-control pipeline on Intel’s Loihi neuromorphic processor on board a flying quadrotor, achieving the first autonomous drone flight with a fully neuromorphic system, running inference at 200 Hz consuming only 7–12 mW. Chapter 4 brings learning on board by optimizing the self-supervised contrast maximization framework for computational efficiency, achieving a 100-fold runtime reduction that enables online depth learning during flight; just two minutes of on-device adaptation improved obstacle avoidance by about 30% over pre-training. Finally, Chapter 5 pushes toward ultimate sensor minimalism by demonstrating that vision can replace inertial measurement units entirely: a recurrent neural network learns to estimate attitude and rotation rates from events alone, enabling the first stable vision-only and IMU-free quadrotor flight.
Together, these contributions chart a path from theoretical promise to practical deployment for event-based robot vision, demonstrating that it is not merely a compelling concept but a viable foundation for autonomous flight. As the technology matures, it will enable a new generation of autonomous drones that perceive and navigate the world through efficient, brain-inspired vision. ... Read more

Hybrid unmanned aerial vehicles (UAVs) with vertical take-off and landing (VTOL) capability combine efficient forward flight with hovering, making them ideal for missions requiring both high-speed flight and precise maneuvering. Among various hybrid UAVs, tailsitters offer a compact and mechanically efficient solution for applications such as search-and-rescue and environmental monitoring. However, operation across the full flight envelope remains challenging due to highly nonlinear aerodynamics at high angles of attack (AoA)and limited control authority caused by actuator saturation. This work addresses these challenges through a systematic integration of wind tunnel–based aerodynamic analysis, control law development, and flight test validations, culminating in an integrated design and control framework enabling agile, robust, and fully autonomous tailsitter flight.

Firstly, wind tunnel tests were performed on a tailsitter wing under varying propeller thrust, elevon deflection, and airspeed across the full range of angles of attack, resulting in the first publicly available aerodynamic dataset of its kind. The results reveal nonlinear aerodynamic behavior, including during stall, post-stall, and reverse flow. In reversed flow, elevon-induced pitch moments act oppositely to normal flow, though this can be mitigated by increasing throttle. Elevon deflection proves effective at low angles of attack and high airspeeds, but its influence degrades at high angles and low speeds. These findings underscore the need for alternative or supplemental actuation to maintain control authority, especially in vertical or descending flight where traditional surfaces lose effectiveness.

Secondly, in response to the limited pitch control authority observed in conventional elevon-only tailsitters (E-tailsitters), new control strategies are necessitated to achieve full-envelope autonomous flight without actuator saturation. A tailsitter equipped with dual tilt rotors (TR-tailsitter) is introduced, which relies exclusively on thrust vectoring for control moment generation. While thrust vectoring provides ample pitch control authority in hover and vertical flight, it lacks sufficient roll control during forward flight due to wing-propeller interaction. To address this limitation, a TRE-tailsitter is proposed, integrating tilting rotors with conventional elevons. This combined actuation setup provides complementary control, with tilt rotors primarily handling low-speed and vertical flight phases, while elevons dominate during highspeed cruise. To achieve full-envelope autonomous flight, a cascaded Incremental Nonlinear Dynamic Inversion (INDI) controller is implemented, with Weighted Least Squares (WLS) control allocation, which dynamically coordinates actuator allocation between rotor tilt and elevon deflection across different flight regimes, avoiding actuator saturation and ensuring seamless transitions.

Thirdly, to enable fully autonomous field deployment, a pivoting takeoff and landing controller is developed for robust VTOL operation under windy and uneven terrain conditions. By exploiting rotor tilt, the vehicle initiates liftoff from a horizontal ground posture through a controlled pivoting motion around its tail, eliminating the need for landing gear and enabling deployment on uneven terrains. Indoor and outdoor flight tests validate the stability and robustness of the proposed approach in the presence of wind disturbances.

Fourthly, the agility of the tilt-rotor tailsitter UAV is examined through high-speed sharp turn scenarios, where maximizing lift is essential for minimizing turning radius. Wind tunnel data covering various actuator combinations are used to develop empirical models of axial force, lift and pitch moment w.r.t thrust, rotor tilt, elevon deflection, AoA, and airspeed, capturing wing–propeller interaction effects. The derived models and pitching moment trim tests reveal that upward rotor tilt combined with downward elevon deflection enhances lift while maintaining pitch equilibrium. Furthermore, a theoretical minimum turning radius of 8.01𝑚 at 18𝑚/𝑠 coordinated sharp turn is computed, confirming that coordinated actuation enables aggressive maneuvers without compromising pitch stability or speed.

Overall, this dissertation develops a tilt-rotor tailsitter UAV capable of robust, autonomous and agile operation across the full flight envelope.The proposed framework advances the understanding of tailsitter aerodynamics and control, and provides a pathway toward field-deployable UAVs for demanding missions requiring both maneuverability and autonomy.
... Read more

Autonomous drones are increasingly used in cluttered, GPS-denied environments where safe and agile navigation depends on reliable visual obstacle avoidance. However, current approaches face three key challenges: the lack of a unified evaluation framework, the trade-off between safety and agility, and the gap between simulation-trained policies and real-world deployment.

This dissertation addresses these issues by developing learning-based methods and evaluation tools for onboard navigation. First, it introduces AvoidBench, a high-fidelity benchmarking suite with standardized environments and metrics to systematically evaluate obstacle avoidance performance.

Second, it presents MAVRL, a reinforcement learning algorithm that adapts flight speed to environmental complexity, achieving an improved balance between safety and agility. Third, it proposes Depth Transfer, a sim-to-real method that bridges differences in dynamics and perception, enabling robust deployment of trained policies on real drones.

Finally, a bio-inspired hierarchical architecture is introduced, separating high-level planning from low-level control to improve training efficiency and robustness.

Together, these contributions advance learning-based drone navigation by enabling reliable evaluation, adaptive behaviour, efficient training, and successful real-world deployment in complex environments. ... Read more

Micro aerial vehicles have shown promising use to further automate food production in greenhouses recently. Compared to conventional multirotor drones, flapping-wing drones offer safe and robust operation around plants due to their soft, slowly-moving wings. Their limited sensing and computational capabilities, however, prohibit the use of map-based navigation methods. To compensate for individual shortcomings, swarming ensures scalability and redundancy. This work proposes a hardware setup combining time-of-flight (ToF) and ultra-wideband (UWB) sensing and explores the artificial evolution of behaviour trees as a reactive planning strategy. Genetic programming, paired with CMAES fine-tuning was able to improve a human-designed exploration strategy by 50%. Neuroevolution has been investigated to encourage emergent swarming behaviours, but requires further experimentation in combination with behaviour trees. The solution obtained in simulation can be readily ported to hardware, but a reality gap in performance persists. These findings contribute to the development of lightweight, scalable aerial systems for autonomous greenhouse monitoring. ... Read more

Neuromorphic hardware and spiking neural networks (SNNs) offer a bio-inspired path to low-latency, energy-efficient computation by emulating the brain’s asynchronous spike-based processing, particularly attractive for real-time optical flow estimation on resource-constrained micro aerial vehicles (MAVs). We leverage a SynSense SPECK™ system-on-chip, which integrates a Dynamic Vision Sensor (DVS) with a neuromorphic processor, to realize live, onboard flow-based attitude and thrust control from dense event-based optical flow. Our inference architecture combines spiking and analog layers in a hybrid SNN-ANN framework, enabling the use of SPECK™ for regression task in a closed-loop drone control, an application not previously demonstrated on the chip. Despite the chip’s compact form factor, the system produces dense flow in real time and achieves stable indoor hover using flow-based control. The hybrid pipeline runs ~2× faster than an ANN-only baseline at identical power. These results highlight the promise of neuromorphic sensing and processing for ultra-efficient, autonomous flight in real-world scenarios. ... Read more

In recent years, flapping wing micro aerial vehicles (FWMAVs) have garnered significant attention due to their agility in cluttered environments and the safety advantages offered by their soft wings during close-proximity operations. These vehicles are subject to numerous constraints: limited power supply, restricted payload capacity, high drag and vibration from the flapping mechanism, and susceptibility to environmental disturbances such as wind gusts—making indoor operation preferable. However, these vehicles have not achieved autonomous navigation yet. The platform used in this study, the Flapper Nimble+, is attitude-stable, meaning it lacks access to egomotion or positional information and is only capable of controlling its thrust, pitch, roll, and yaw. Given the strict SWaP (Size, Power and Weight) constraints, lightweight Time-of-Flight (ToF) sensors are among the most practical sensing solutions. This thesis investigates the question: How can obstacle avoidance be achieved in an attitude-stable flapping wing air vehicle equipped with Time-of-Flight sensors? Two control approaches were implemented and tested in the IsaacGym simulator: a simple PID controller with confidence-based yaw adjustment, and a reinforcement learning (RL) policy trained using Proximal Policy Optimization. Two sensor configurations were considered: a minimal two-sensor setup (front and downward) and a richer five-sensor array for broader perception. The PID controller successfully avoided collisions in all environments, demonstrating high reliability but limited exploration, averaging 35% coverage. In contrast, the RL policy with five sensors achieved greater spatial exploration—averaging 49.5% coverage—at the cost of an average of 3.5 collisions per episode. When reduced to two sensors, the RL agent’s performance declined significantly, with average coverage dropping to 29% and collision counts increasing to 6.6 (excluding failed runs). These results show that autonomous navigation is achievable for attitude-stable flapping wing air vehicles using only ToF sensors. While PID control offers reliable, conservative navigation under minimal sensing, RL enables more exploratory behaviors and adaptability in dynamic environments. ... Read more

he real-world application of Micro Air Vehicles (MAVs) is often constrained by their limited flight range and endurance, primarily due to battery limitations. One way to overcome this challenge is by leveraging naturally occurring vertical air currents, a technique inspired by soaring birds. This paper introduces Osprey Simulator, a framework designed to simulate and test energy-efficient soaring flight strategies for fixed-wing drones. Built on Isaac Sim, Osprey Simulator enables the creation of both randomly generated urban environments and real-world environments, such as the TU Delft campus. These environments are paired with accurate wind field simulations using OpenFOAM, providing a robust platform for studying aerodynamic interactions in diverse settings. This functionality allows for the generation of scalable synthetic datasets, including depth images and wind field information, enabling comprehensive exploration of soaring potential across various structural geometries and wind conditions. Using generated synthetic data, a neural network was trained to predict optimal soaring regions by analyzing depth images and wind field information. The network demonstrates the ability to identify updraft and downdraft regions, enabling more efficient path planning for drones in urban environments. By integrating realistic simulations and advanced predictive models, Osprey Simulator serves as a powerful tool for advancing autonomous soaring and extending the operational range of fixed-wing MAVs. ... Read more

Event-based cameras provide high temporal resolution, robustness to lighting conditions and low power consumption, but their sparse, temporal data require models that reason over time. In supervised settings, this is increasingly handled with recurrent architectures. In contrast, most self-supervised learning (SSL) methods still adapt non-recurrent RGB techniques, with masking-based objectives that favor spatial reconstruction over temporal understanding. We introduce SPICE: Self-supervised Predictive Coding on Events, an SSL framework tailored to event data that processes longer sequences recurrently and learns by predicting future latent representations rather than reconstructing masked inputs, promoting a more natural objective focyused on anticipating what comes next. SPICE further incorporates an event-specific contrastive loss only operating on active regions. SPICE pre-training improves downstream performance on semantic segmentation, depth estimation and optical flow estimation. Low-dimensional projections confirm that the learned representations are meaningful and avoid collapse, while also revealing limitations in temporal stability and semantic organization, indicating clear directions for future event-specific SSL research. Code is available upon request. ... Read more

PATS-X is a greenhouse pest suppression system that uses a depth camera with an autonomous micro air vehicle (MAV) to detect, track, and physically intercept flying insects. This study targets guidance and control for reliable aerial-to-aerial interception. Reinforcement learning (RL) is used to learn policies from insect flight recordings. We evaluate control policies at increasing levels of abstraction: direct motor commands, collective thrust and body rates (CTBR), and acceleration. In simulation, lower abstraction levels yield better interception performance; moving from acceleration to motor command reduces the median time to first interception by about 41%. A systematic variation of the observation space reveals that the most effective observations are body frame relative position and velocity, and short temporal histories add no benefit beyond noise filtering. Compared with a state-of-the-art classical benchmark, Fast Response Proportional Navigation (FRPN), the best motor level RL policy in simulation achieves a median first interception time of 0.85 [0.76--1.07]s with 99.1% interception rate, compared with FRPN at 1.90 [1.04--2.80]s and 95.6%. To address the reality gap, we compare how well the different control abstractions transfer to hardware. CTBR policies deploy on hardware with the least performance loss relative to simulation. Motor-level policies also transfer when trained with modest domain randomization (DR) plus an action-difference penalty that limits command jitter and thermal load. Acceleration-level policies did not transfer. In a PATS-X proof of concept, an RL controller deployed on the actual system reached a 95.6% interception rate of virtual moths versus 80.0% for the existing controller. Moreover, the RL controller shortened time-to-first-interception by 0.70s, indicating the potential of RL-based guidance for the PATS-X system. ... Read more

To safely and efficiently solve motion planning problems in multi-agent settings, most approaches attempt to solve a joint optimization that explicitly accounts for the responses triggered in other agents. This often results in solutions with an exponential computational complexity, making these methods intractable for complex scenarios with many agents. While sequential predict-and-plan approaches are more scalable, they tend to perform poorly in highly interactive environments. This paper proposes a method to improve the interactive capabilities of sequential predict-and-plan methods in multi-agent navigation problems by introducing predictability as an optimization objective. We interpret predictability through the use of general prediction models, by allowing agents to predict themselves and estimate how they align with these external predictions. We formally introduce this behavior through the free-energy of the system, which reduces (under appropriate bounds) to the Kullback-Leibler divergence between plan and prediction, and use this as a penalty for unpredictable trajectories. The proposed interpretation of predictability allows agents to more robustly leverage prediction models, and fosters a ‘soft social convention' that accelerates agreement on coordination strategies without the need of explicit high level control or communication. We show how this predictability-aware planning leads to lower-cost trajectories and reduces planning effort in a set of multi-robot problems, including autonomous driving experiments with human driver data, where we show that the benefits of considering predictability apply even when only the ego-agent uses this strategy. ... Read more

There exists a wide array of possible applications for small, safe, cost-effective, and energy-efficient drones.
However, their development is hampered by limited payload capacity, which restricts both computational power and flight time.
Traditional control systems and sensor processing algorithms are ill-suited for these resource-constrained platforms since they typically rely on power-hungry processors and complex numerical methods.

This thesis investigates neuromorphic approaches to both state estimation and control for small drones.
Inspired by the energy-efficient and highly parallel processing of biological neural systems, neuromorphic computing leverages spiking neural networks (SNNs) that operate via discrete spikes, offering real-time, low-power processing capabilities for micro aerial vehicles (MAVs).
While previous work has applied neuromorphic methods to high-level perception tasks, their application to fundamental flight control -- such as precise attitude estimation and low-level control -- remains largely unexplored.

Following a review of the current state of neuromorphic computing, the research first explores its application to state estimation.
A recurrent SNN is designed to estimate the drone’s attitude from inertial measurement unit (IMU) data, achieving performance comparable to conventional methods like the complementary filter, despite employing a minimal network architecture.
The study then investigates event-based vision sensors by processing data from a downward-facing event camera to estimate the attitude and angular rates, enabling a quadrotor to achieve flight without inertial sensing -- a pioneering demonstration in the field.

Transitioning from estimation to control, the thesis uses neuromorphic algorithms to perform low-level control tasks.
A spiking PID controller is developed using a fixed network architecture, demonstrating altitude control using Intel's Loihi neuromorphic processor.
To address the challenge of precise integration inherent in spiking systems, the Input-Weighted Threshold Adaptation (IWTA) mechanism is introduced.
This innovative approach allows for precise integration of incoming signals and was used as the integral component of a neuromorphic PID controller, mitigating steady-state errors and compensating for sensor biases.

Ultimately, the work unifies estimation and control into a single end-to-end neuromorphic system deployed on a tiny 27g Crazyflie quadrotor. Trained via imitation learning on real flight data, the integrated network maps raw inertial sensor inputs directly to motor commands at a control frequency of 500Hz, achieving attitude tracking performance comparable to traditional controllers.

Overall, this thesis demonstrates that neuromorphic computing is a promising approach for low-level state estimation and control in flying drones, while also addressing the challenges of implementing such systems in real-world environments with sensor biases and persistent disturbances. ... Read more

Achieving precise and reliable autonomous landings on moving ship decks is a critical challenge for hybrid Unmanned Aerial vehicles (UAVs), particularly under harsh maritime conditions. This dissertation addresses this challenge by developing a nonlinear control framework for a novel dual-axis tilting rotor quad-plane, designed specifically to enhance stability, maneuverability, and precision during landing operations on moving platforms. Operating in such harsh environments requires effective handling of varying wind conditions, ship motion, and rapid position changes.

The dual-axis tilting rotor concept offers unique capabilities that make it particularly suited for precision landing tasks on moving ship decks. Unlike conventional hybrid under-actuated UAV designs, the overactuated nature of the dual-axis tilting rotor quad-plane allows it to maintains full 6 Degrees Of Freedom (DOF) control authority during low airspeed flight. Moreover, its rapid thrust vectoring capability provides exceptional wind rejection performance, enabling precise control even under turbulent maritime conditions. This versatility is essential for maintaining stability and accuracy during complex landing maneuvers where environmental disturbances and ship motion are constantly changing. However, the novel propulsion system also presents significant control challenges due to its non-affine in the input dynamics. Developing advanced control strategies capable of handling these complexities is critical for achieving reliable and precise autonomous landings on moving platforms.

First, a nonlinear programming-based control allocation algorithm is developed to overcome the limitations of state-of-the-art methods that rely on linearized control effectiveness. This assumption may be too restrictive for vehicles with highly nonlinear effector dynamics, such as the dual-axis tilting rotor quad-plane. The proposed control allocation algorithm effectively manages the complex dynamic interactions inherent to the novel propulsion system, enabling smooth and precise control within the low-airspeed flight regime.

Second, the controller is extended to operate across the entire flight envelope, from hovering to forward flight. Unlike traditional hybrid UAV controllers that rely on distinct control strategies for each flight mode, the proposed framework enables seamless transitions by dynamically reallocating control objectives across the available actuators, including adjustments to vehicle attitude as airspeed varies.

Third, this dissertation introduces a real-time actuator state feedback system to enhance the controller’s robustness and adaptability in challenging maritime environments. By eliminating reliance on predefined actuator models, this improvement provides continuous situational awareness of the propulsion system’s health, allowing the controller to dynamically adjust to varying operational conditions. The enhanced hardware architecture enables direct control of motor RPM and angular rotor tilt, significantly improving the system’s resilience against environmental disturbances, actuator degradation, and battery voltage fluctuation. This capability forms the foundation for developing a robust fault-tolerant control framework essential for reliable autonomous landings on moving platforms.

Fourth, leveraging the improved hardware architecture, a fault-tolerant control framework is developed to ensure reliable operation under various actuator failure conditions. The dual-axis tilting rotor quad-plane’s over-actuated design allows the controller to reallocate control efforts dynamically among functional actuators, maintaining stability and control authority even under severe actuator failures. Extensive flight tests validate the fault-tolerant framework’s effectiveness, demonstrating resilience in challenging scenarios.

Finally, a real-time trajectory planning algorithm is developed to enable autonomous landing on moving ship decks. Using a Long Short-Term Memory (LSTM) neural network model, a prediction framework is created to estimate ship motion over a 7-second horizon. This prediction enables the generation of feasible landing trajectories that are continuously reassessed to account for prediction uncertainties and tracking errors. While simulation results demonstrated the approach’s potential, future work will involve testing the algorithm in real-world conditions.

The proposed control framework, combined with the novel dual-axis tilting rotor quad-plane design, offers a robust and adaptable solution for achieving reliable autonomous landings on moving platforms. The methodologies developed in this thesis can be further extended to various hybrid UAV configurations, providing valuable insights for broader applications in complex operational environments. ... Read more

Insects like honeybees exhibit remarkable navigational abilities despite their simple nervous systems, showcasing expertise in tasks such as long-distance travel, landmark recognition, and spatial memory. These skills are crucial for efficient foraging and homing. In robotics, one of the main challenges is to navigate in GPS-denied environments with limited sensors and processors onboard. In this study, we propose a novel navigation strategy that uses learning flights during which a robot directly maps images to nest location vectors, inspired by honeybees. In our previous research, the learning phase was modeled using compact convolutional neural networks (CNNs) and demonstrated successful learning and control in simulation. In this work, we combine this homing model with odometry and implement both in a real-world quadrotor. More specifically, we examine how this model can compensate for the drone’s odometric drift to reach home after it performs a long-distance outbound flight. Real-world experiments demonstrate the proof of concept of the proposed navigation strategy in both indoor and outdoor flights. ... Read more

This paper presents an encoder-decoder-style convolutional neural network (CNN) for the purpose of improving monocular and stereo depth estimation (SDE) estimates, by combining them with the corresponding monocular estimates through a fusion network, assisted by prior information to provide context for the fusion. Video cameras are commonly used for depth perception in robotics, especially weight-sensitive applications, such as on Micro Aerial Vehicles (MAV). The two primary paradigms for vision-based depth perception are monocular and stereo depth or disparity estimation, each having their own strengths and weaknesses. These strengths and weaknesses seem to be complementary, and thus a fusion of the two may result in more accurate predictions. In this paper, we investigate this fusion by training a CNN that combines stereo and monocular depth or disparity estimates. The fusion network is agnostic to the choice of the input networks, providing great flexibility. It was found that such a fusion network, while increasing the computational complexity of the depth perception pipeline, indeed improves the accuracy of the estimates. The number of outlier predictions has been significantly decreased, while also limiting some fundamental limitations of both stereo and monocular methods, such as errors arising from occluded regions. ... Read more

Spiking neural networks implemented for sensing and control of robots have the potential to achieve lower latency and power consumption by processing information sparsely and asynchronously. They have been used on neuromorphic devices to estimate optical flow for micro air vehicles navigation, however robotic implementations have been limited to hardware setups with sensing and processing as separate systems. This article investigates a new approach for training a spiking neural network for optical flow to be deployed on the speck2e device from Synsense. The method takes into account the restrictions of the speck2e in terms of network architecture, neuron model, and number of synaptic operations and it involves training a recurrent neural network with ReLU activation functions, which is subsequently converted into a spiking network. A system of weight rescaling is applied after conversion, to ensure optimal information flow between the layers. Our study shows that it is possible to estimate optical flow with Integrate-and-Fire neurons. However, currently, the optical flow estimation performance is still hampered by the number of synaptic operations. As a result, the network presented in this work is able to estimate optical flow in a range of [-4, 1] pixel/s. ... Read more

New insights into the landing behavior of bumblebees show an adaptive strategy where the optical flow expansion of the landing target is step-wise regulated. In this article, the potential benefits of this approach are studied by replicating the landing experiment with a quadrotor. To this end, an open-loop switching method is developed, enabling fast steps in divergence. An adaptive control law is used to deal with non-linear system dynamics, where the control gain is scheduled based on the control effectiveness of the actuator inputs during the steps. It is demonstrated that the quadrotor can reliably land on the target from varying initial positions, and the switching strategy shows a slight reduction in landing time compared to a constant divergence strategy with the same average divergence over distance. This strategy also reduces the maximum velocity during the landing. ... Read more