Neuromorphic Autopilot for Drone Flight

Abstract

There exists a wide array of possible applications for small, safe, cost-effective, and energy-efficient drones.
However, their development is hampered by limited payload capacity, which restricts both computational power and flight time.
Traditional control systems and sensor processing algorithms are ill-suited for these resource-constrained platforms since they typically rely on power-hungry processors and complex numerical methods.

This thesis investigates neuromorphic approaches to both state estimation and control for small drones.
Inspired by the energy-efficient and highly parallel processing of biological neural systems, neuromorphic computing leverages spiking neural networks (SNNs) that operate via discrete spikes, offering real-time, low-power processing capabilities for micro aerial vehicles (MAVs).
While previous work has applied neuromorphic methods to high-level perception tasks, their application to fundamental flight control -- such as precise attitude estimation and low-level control -- remains largely unexplored.

Following a review of the current state of neuromorphic computing, the research first explores its application to state estimation.
A recurrent SNN is designed to estimate the drone’s attitude from inertial measurement unit (IMU) data, achieving performance comparable to conventional methods like the complementary filter, despite employing a minimal network architecture.
The study then investigates event-based vision sensors by processing data from a downward-facing event camera to estimate the attitude and angular rates, enabling a quadrotor to achieve flight without inertial sensing -- a pioneering demonstration in the field.

Transitioning from estimation to control, the thesis uses neuromorphic algorithms to perform low-level control tasks.
A spiking PID controller is developed using a fixed network architecture, demonstrating altitude control using Intel's Loihi neuromorphic processor.
To address the challenge of precise integration inherent in spiking systems, the Input-Weighted Threshold Adaptation (IWTA) mechanism is introduced.
This innovative approach allows for precise integration of incoming signals and was used as the integral component of a neuromorphic PID controller, mitigating steady-state errors and compensating for sensor biases.

Ultimately, the work unifies estimation and control into a single end-to-end neuromorphic system deployed on a tiny 27g Crazyflie quadrotor. Trained via imitation learning on real flight data, the integrated network maps raw inertial sensor inputs directly to motor commands at a control frequency of 500Hz, achieving attitude tracking performance comparable to traditional controllers.

Overall, this thesis demonstrates that neuromorphic computing is a promising approach for low-level state estimation and control in flying drones, while also addressing the challenges of implementing such systems in real-world environments with sensor biases and persistent disturbances.