Flapping wing Micro Air Vehicles (FWMAVs) hold great potential for real-world applications but are currently still hard to model. In this article, a simplified analysis of the equilibrium state of a tailless FWMAV in forward flight is presented. The definition of the equilibrium state complements previous dynamic and stability analysis, adding new information about the flight behavior of FWMAVs. A new aerodynamic decoupled model has been used for the analysis, considering separately the thrust force generated by the flapping movement and the lift and drag caused by the forward velocity. The aerodynamic forces are included in a dynamic model of the FWMAV, and the equilibrium state is derived. The formulation obtained is explicit in terms of the pitch actuator deflection, thus allowing its use for control corrections, and provides an estimation of the flight velocity. The thrust needed to maintain height is also formulated, demonstrating that forward flight is more efficient than hovering. The results are validated experimentally for the pitch angle, showing good agreement with the analytical results. Then, the dynamics of the FWMAV are simulated, comparing the results with experiments where the FWMAV goes from hovering to a specific pitch reference while maintaining its height. Additional simulations are performed with basic control considerations, showing how considering the equilibrium state for a feed-forward control significantly improves the flight behavior compared to PI and PID controllers, reducing the convergence time.

Tailsitter aircraft attract considerable interest due to their capabilities of both agile hover and high speed forward flight. However, traditional tailsitters that use aerodynamic control surfaces face the challenge of limited control effectiveness and associated actuator saturation during vertical flight and transitions. Conversely, tailsitters relying solely on tilting rotors have the drawback of insufficient roll control authority in forward flight. This letter proposes a tilt-rotor tailsitter aircraft with both elevons and tilting rotors as a promising solution. By implementing a cascaded weighted least squares (WLS) based incremental nonlinear dynamic inversion (INDI) controller, the drone successfully achieved autonomous waypoint tracking in outdoor experiments at a cruise airspeed of 16 m/s, including transitions between forward flight and hover without actuator saturation. Wind tunnel experiments confirm improved roll control compared to tilt-rotor-only configurations, while comparative outdoor flight tests highlight the vehicle's superior control over elevon-only designs during critical phases such as vertical descent and transitions. Finally, we also show that the tilt-rotors allow for an autonomous takeoff and landing with a unique pivoting capability that demonstrates stability and robustness under wind disturbances.

Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it difficult to accurately measure technological advancements, compare performance with conventional methods, and identify promising future research directions. This article presents NeuroBench, a benchmark framework for neuromorphic algorithms and systems, which is collaboratively designed from an open community of researchers across industry and academia. NeuroBench introduces a common set of tools and systematic methodology for inclusive benchmark measurement, delivering an objective reference framework for quantifying neuromorphic approaches in both hardware-independent and hardware-dependent settings. For latest project updates, visit the project website (neurobench.ai).

This study covers three aspects of acoustic localisation of drones using a microphone array. First, it assesses a grid-free approach, using differential evolution, to estimate the three-dimensional position of a drone. It is found that this is indeed possible for the drone in the near-field. For larger distances, it still provides the angular position of the drone. Second, the study emphasizes the essence of localisation over small frequency bands with the bands jointly spanning a large frequency range to reveal the presence of multiple sound sources and maximise the drone localisation range. Third, it addresses the localisation ranges for six different drones.

Autonomous flight in unknown, cluttered environments is still a major challenge in robotics. Existing obstacle avoidance algorithms typically adopt a fixed flight velocity, overlooking the crucial balance between safety and agility. We propose a reinforcement learning algorithm to learn an adaptive flight speed policy tailored to varying environment complexities, enhancing obstacle avoidance safety. A downside of learning-based obstacle avoidance algorithms is that the lack of a mapping module can lead to the drone getting stuck in complex scenarios. To address this, we introduce a novel training setup for the latent space that retains memory of previous depth map observations. The latent space is explicitly trained to predict both past and current depth maps. Our findings confirm that varying speed leads to a superior balance of success rate and agility in cluttered environments. Additionally, our memory-augmented latent representation outperforms the latent representation commonly used in reinforcement learning. Furthermore, an extensive comparison of our method with the existing state-of-the-art approaches Agile-autonomy and Ego-planner shows the superior performance of our approach, especially in highly cluttered environments. Finally, after minimal fine-tuning, we successfully deployed our network on a real drone for enhanced obstacle avoidance.

Spiking neural networks (SNNs) for event-based optical flow are claimed to be computationally more efficient than their artificial neural networks (ANNs) counterparts, but a fair comparison is missing in the literature. In this work, we propose an event-based optical flow solution based on activation sparsification and a neuromorphic processor, SENECA. SENECA has an event-driven processing mechanism that can exploit the sparsity in ANN activations and SNN spikes to accelerate the inference of both types of neural networks. The ANN and the SNN for comparison have similar low activation/spike density (∼5%) thanks to our novel sparsification-aware training. In the hardware-in-loop experiments designed to deduce the average time and energy consumption, the SNN consumes 44.9ms and 927.0μJ, which are 62.5% and 75.2% of the ANN's consumption, respectively. We find that SNN's higher efficiency is attributed to its lower pixel-wise spike density (43.5% vs. 66.5%) that requires fewer memory access operations for neuron states.

Lightweight aerial swarms have potential applications in scenarios where larger drones fail to operate efficiently. The primary foundation for lightweight aerial swarms is efficient relative localization, which enables cooperation and collision avoidance. Computing the real-time position is challenging due to extreme resource constraints. This letter presents an autonomous relative localization technique for lightweight aerial swarms without infrastructure by fusing ultra-wideband wireless distance measurements and the shared state information (e.g., velocity, yaw rate, height) from neighbors. This is the first fully autonomous, tiny, fast, and accurate relative localization scheme implemented on a team of 13 lightweight (33 grams) and resource-constrained (168 MHz MCU with 192 KB memory) aerial vehicles. The proposed resource-constrained swarm ranging protocol is scalable, and a surprising theoretical result is discovered: the unobservability poses no issues because the state drift leads to control actions that make the state observable again. By experiment, less than 0.2 m position error is achieved at the frequency of 16 Hz for as many as 13 drones. The code is open-sourced, and the proposed technique is relevant not only for tiny drones but can be readily applied to many other resource-restricted robots.

Learning-based visual ego-motion estimation is promising yet not ready for navigating agile mobile robots in the real world. In this article, we propose CUAHN-VIO, a robust and efficient monocular visual-inertial odometry (VIO) designed for micro aerial vehicles (MAVs) equipped with a downward-facing camera. The vision frontend is a content-and-uncertainty-aware homography network (CUAHN). Content awareness measures the robustness of the network toward non-homography image content, e.g. 3-dimensional objects lying on a planar surface. Uncertainty awareness refers that the network not only predicts the homography transformation but also estimates the prediction uncertainty. The training requires no ground truth that is often difficult to obtain. The network has good generalization that enables “plug-and-play” deployment in new environments without fine-tuning. A lightweight extended Kalman filter (EKF) serves as the VIO backend and utilizes the mean prediction and variance estimation from the network for visual measurement updates. CUAHN-VIO is evaluated on a high-speed public dataset and shows rivaling accuracy to state-of-the-art (SOTA) VIO approaches. Thanks to the robustness to motion blur, low network inference time (∼23 ms), and stable processing latency (∼26 ms), CUAHN-VIO successfully runs onboard an Nvidia Jetson TX2 embedded processor to navigate a fast autonomous MAV.

In high-speed quadcopter racing, finding a single controller that works well across different platforms remains challenging. This work presents the first neural network controller for drone racing that generalizes across physically distinct quadcopters. We demonstrate that a single network, trained with domain randomization, can robustly control various types of quadcopters. The network relies solely on the current state to directly compute motor commands. The effectiveness of this generalized controller is validated through real-world tests on two substantially different crafts (3-inch and 5-inch race quadcopters). We further compare the performance of this generalized controller with controllers specifically trained for the 3-inch and 5-inch drone, using their identified model parameters with varying levels of domain randomization (0%, 10%, 20%, 30%). While the generalized controller shows slightly slower speeds compared to the fine-tuned models, it excels in adaptability across different platforms. Our results show that no randomization fails sim-to-real transfer while increasing randomization improves robustness but reduces speed. Despite this trade-off, our findings highlight the potential of domain randomization for generalizing controllers, paving the way for universal AI controllers that can adapt to any platform.

Since every flight ends in a landing and every landing is a potential crash, deceleration during landing is one of the most critical flying maneuvers. Here we implement a recently-discovered insect visual-guided landing strategy in which the divergence of optical flow is regulated in a step-wise fashion onboard a quadrotor for the task of visual servoing. This approach was shown to be a powerful tool for understanding challenges encountered by visually-guided flying systems. We found that landing on a relatively small target requires mitigation of the noise with adaptive low-pass filtering, while compensation for the delays introduced by this filter requires open-loop forward accelerations to switch from divergence setpoint. Both implemented solutions are consistent with insect physiological properties. Our study evaluates the challenges of visual-based navigation for flying insects. It highlights the benefits and feasibility of the switching divergence strategy that allows for faster and safer landings in the context of robotics.

This paper analyses the methods and technologies involved in flapping-wing flying robots (FWFRs), where the actuation of the flapping wing produces thrust and lift force that mimics birds’ and insects’ flight. The focus is on the evolution of the flapping-wing technology and the challenges in prototyping, modeling, navigation, and control. The mechanism for flapping production, frequency control of the flapping, and wing/tail control for positioning the robot are important topics for successful prototyping. The article includes the study of the dynamics and aerodynamics of the FWFR. Using the combination of flapping and gliding has led researchers to seek more energy savings through this hybrid-in-nature dynamic system, which benefits from the wind, a natural and free energy source. The paper reviews the dynamics, design, and categorization of flapping-wing systems; it also includes control and onboard intelligent functionalities, particularly environment perception for positioning and guidance, as well as obstacle detection and avoidance.

The real-world application of small drones is mostly hampered by energy limitations. Neuromorphic computing promises extremely energy-efficient AI for autonomous flight but is still challenging to train and deploy on real robots. To reap the maximal benefits from neuromorphic computing, it is necessary to perform all autonomy functions end-to-end on a single neuromorphic chip, from low-level attitude control to high-level navigation. This research presents the first neuromorphic control system using a spiking neural network (SNN) to effectively map a drone's raw sensory input directly to motor commands. We apply this method to low-level attitude estimation and control for a quadrotor, deploying the SNN on a tiny Crazyflie. We propose a modular SNN, separately training and then merging estimation and control sub-networks. The SNN is trained with imitation learning, using a flight dataset of sensory-motor pairs. Post-training, the network is deployed on the Crazyflie, issuing control commands from sensor inputs at 500Hz. Furthermore, for the training procedure we augmented training data by flying a controller with additional excitation and time-shifting the target data to enhance the predictive capabilities of the SNN. On the real drone, the perception-to-control SNN tracks attitude commands with an average error of 3.0 degrees, compared to 2.7 degrees for the regular flight stack. We also show the benefits of the proposed learning modifications for reducing the average tracking error and reducing oscillations. Our work shows the feasibility of performing neuromorphic end-to-end control, laying the basis for highly energy-efficient and low-latency neuromorphic autopilots.

Sim-to-real transfer is a fundamental challenge in robot learning. Discrepancies between simulation and reality can significantly impair policy performance, especially if it receives high-dimensional inputs such as dense depth estimates from vision. We propose a novel depth transfer method based on domain adaptation to bridge the visual gap between simulated and real-world depth data. A Variational Autoencoder (VAE) is first trained to encode ground-truth depth images from simulation into a latent space, which serves as input to a reinforcement learning (RL) policy. During deployment, the encoder is refined to align stereo depth images with this latent space, enabling direct policy transfer without fine-tuning. We apply our method to the task of autonomous drone navigation through cluttered environments. Experiments in IsaacGym show that our method nearly doubles the obstacle avoidance success rate when switching from ground-truth to stereo depth input. Furthermore, we demonstrate successful transfer to the photo-realistic simulator AvoidBench using only IsaacGym-generated stereo data, achieving superior performance compared to state-of-the-art baselines. Real-world evaluations in both indoor and outdoor environments confirm the effectiveness of our approach, enabling robust and generalizable depth-based navigation across diverse domains.

Accurate trajectory tracking with quadrotors is a challenging task that requires a trade-off between accuracy and complexity to run onboard. Stateof- the-art adaptive controllers achieve impressive trajectory tracking results with slight performance degradation in varying winds or payloads but at the cost of computational complexity. This work proposes a lightweight combination of adaptive and neural control and shows its performance when flying with propeller damage. The neural architecture consists of offline learning of a condition-invariant representation of the aerodynamic forces through Deep Neural Networks. The second part consists of fast online adaptation using a composite adaptation law. We deploy this flight controller fully onboard the flight controller of the Parrot Bebop 1, showcasing its computational efficiency. The adaptive neural controller improves tracking performance by ≈ 60% over the nonlinear baseline, with minimal performance degradation of just ≈ 20% with increasing propeller damage.

This Review discusses the main results obtained in training end-to-end neural architectures for guidance and control of interplanetary transfers, planetary landings, and close-proximity operations, highlighting the successful learning of optimality principles by the underlying neural models. Spacecraft and drones aimed at exploring our solar system are designed to operate in conditions where the smart use of onboard resources is vital to the success or failure of the mission. Sensorimotor actions are thus often derived from high-level, quantifiable, optimality principles assigned to each task, using consolidated tools in optimal control theory. The planned actions are derived on the ground and transferred on board, where controllers have the task of tracking the uploaded guidance profile. Here, we review recent trends based on the use of end-to-end networks, called guidance and control networks (G&CNets), which allow spacecraft to depart from such an architecture and to embrace the onboard computation of optimal actions. In this way, the sensor information is transformed in real time into optimal plans, thus increasing mission autonomy and robustness. We then analyze drone racing as an ideal gym environment to test these architectures on real robotic platforms and thus increase confidence in their use in future space exploration missions. Drone racing not only shares with spacecraft missions both limited onboard computational capabilities and similar control structures induced from the optimality principle sought but also entails different levels of uncertainties and unmodeled effects and a very different dynamical timescale.

Heading estimation is vital for the autonomous flight of unmanned aerial vehicles. Magnetometers are typically used for this purpose, but they are not robust to electro-magnetic interferences. As a promising alternative, we investigate the insect-inspired solution of skylight polarization sensing. In particular, we develop a robust polarization compass for azimuth estimation. Two datasets are created - one based on a Mie scattering simulation, and one containing real-world pictures captured with a polarization camera under a variety of weather conditions. We employ the ResNet-18 model, which is trained and tested on both datasets separately. The trained model is robust to different weather conditions, and is able to directly analyze maps in the instrumental plane. The median error on the (mostly cloudy) real-world images of 4.30 degrees makes it a promising new method for the navigational toolkit of UAVs. We publish the real-world polarization dataset as open access data, in order to facilitate improvements by the community.

Aggressive time-optimal control of quadcopters poses a significant challenge in the field of robotics. The state-of-the-art approach leverages reinforcement learning (RL) to train optimal neural policies. However, a critical hurdle is the sim-to-real gap, often addressed by employing a robust inner loop controller-an abstraction that, in theory, constrains the optimality of the trained controller, necessitating margins to counter potential disturbances. In contrast, our novel approach introduces high-speed quadcopter control using end-to-end RL (E2E) that gives direct motor commands. To bridge the reality gap, we incorporate a learned residual model and an adaptive method that can compensate for modeling errors in thrust and moments. We compare our E2E approach against a state-of-the-art network that commands thrust and body rates to an INDI inner loop controller, both in simulated and real-world flight. E2E showcases a significant 1.39-second advantage in simulation and a 0.17-second edge in real-world testing, highlighting end-to-end reinforcement learning's potential. The performance drop observed from simulation to reality shows potential for further improvement, including refining strategies to address the reality gap or exploring offline reinforcement learning with real flight data.

Navigation is an essential capability for autonomous robots. In particular, visual navigation has been a major research topic in robotics because cameras are lightweight, power-efficient sensors that provide rich information on the environment. However, the main challenge of visual navigation is that it requires substantial computational power and memory for visual processing and storage of the results. As of yet, this has precluded its use on small, extremely resource-constrained robots such as lightweight drones. Inspired by the parsimony of natural intelligence, we propose an insect-inspired approach toward visual navigation that is specifically aimed at extremely resource-restricted robots. It is a route-following approach in which a robot's outbound trajectory is stored as a collection of highly compressed panoramic images together with their spatial relationships as measured with odometry. During the inbound journey, the robot uses a combination of odometry and visual homing to return to the stored locations, with visual homing preventing the buildup of odometric drift. A main advancement of the proposed strategy is that the number of stored compressed images is minimized by spacing them apart as far as the accuracy of odometry allows. To demonstrate the suitability for small systems, we implemented the strategy on a tiny 56-gram drone. The drone could successfully follow routes up to 100 meters with a trajectory representation that consumed less than 20 bytes per meter. The presented method forms a substantial step toward the autonomous visual navigation of tiny robots, facilitating their more widespread application.

Insects have long been recognized for their ability to navigate and return home using visual cues from their nest's environment. However, the precise mechanism underlying this remarkable homing skill remains a subject of ongoing investigation. Drawing inspiration from the learning flights of honey bees and wasps, we propose a robot navigation method that directly learns the home vector direction from visual percepts during a learning flight in the vicinity of the nest. After learning, the robot will travel away from the nest, come back by means of odometry, and eliminate the resultant drift by inferring the home vector orientation from the currently experienced view. Using a compact convolutional neural network, we demonstrate successful learning in both simulated and real forest environments, as well as successful homing control of a simulated quadrotor. The average errors of the inferred home vectors in general stay well below the 90° required for successful homing, and below 24° if all images contain sufficient texture and illumination. Moreover, we show that the trajectory followed during the initial learning flight has a pronounced impact on the network's performance. A higher density of sample points in proximity to the nest results in a more consistent return. Code and data are available at https://mavlab.tudelft.nl/learning_to_home.

Inspired by frame-based methods, state-of-the-art event-based optical flow networks rely on the explicit construction of correlation volumes, which are expensive to compute and store, rendering them unsuitable for robotic applications with limited compute and energy budget. Moreover, correlation volumes scale poorly with resolution, prohibiting them from estimating high-resolution flow. We observe that the spatiotemporally continuous traces of events provide a natural search direction for seeking pixel correspondences, obviating the need to rely on gradients of explicit correlation volumes as such search directions. We introduce IDNet (Iterative Deblurring Network), a lightweight yet high-performing event-based optical flow network directly estimating flow from event traces without using correlation volumes. We further propose two iterative update schemes: "ID"which iterates over the same batch of events, and "TID"which iterates over time with streaming events in an online fashion. Our top-performing model (ID) sets a new state of the art on DSEC benchmark. Meanwhile, the base model (TID) is competitive with prior arts while using 80% fewer parameters, consuming 20x less memory footprint and running 40% faster on the NVidia Jetson Xavier NX. Furthermore, the TID scheme is even more efficient offering an additional 5x faster inference speed and 8 ms ultra-low latency at the cost of only a 9% performance drop, making it the only model among current literature capable of real-time operation while maintaining decent performance.Code: https://github.com/tudelft/idnet.