Print Email Facebook Twitter Unreal Success: Vision-Based UAV Fault Detection and Diagnosis Framework Title Unreal Success: Vision-Based UAV Fault Detection and Diagnosis Framework Author de Alvear Cardenas, J.I. (San José State University) de Visser, C.C. (TU Delft Control & Simulation) Date 2024 Abstract Online fault detection and diagnosis (FDD) enables Unmanned Aerial Vehicles (UAVs) to take informed decisions upon actuator failure during flight, adapting their control strategy or deploying emergency systems. Despite the camera being a ubiquitous sensor on-board of most commercial UAVs, it has not been used within FDD systems before, mainly due to the nonexistence of UAV multi-sensor datasets that include actuator failure scenarios. This paper presents a knowledge-based FDD framework based on a lightweight LSTM network and a single layer neural network classifier that fuses camera and Inertial Measurement Unit (IMU) information. Camera data are pre-processed by first computing its optical flow with RAFT-S, a state-of-the-art deep learning model, and then extracting features with the backbone of MobileNetV3-S. Short-Time Fourier Transform is applied on the IMU data for obtaining their time-frequency information. For training and assessing the proposed framework, UUFOSim was developed: an Unreal Engine-based simulator built on AirSim that allows the collection of high-fidelity photo-realistic camera and sensor information, and the injection of actuator failures during flight. Data were collected in simulation for the Bebop 2 UAV with 16 failure cases. Results demonstrate the added value of the camera and the complementary nature of both sensors with failure detection and diagnosis accuracies of 99.98% and 98.86%, respectively. To reference this document use: http://resolver.tudelft.nl/uuid:19af5ad2-e69d-435e-9539-c89dd885a27f DOI https://doi.org/10.2514/6.2024-0760 Publisher American Institute of Aeronautics and Astronautics Inc. (AIAA) ISBN 978-1-62410-711-5 Source Proceedings of the AIAA SCITECH 2024 Forum Event AIAA SCITECH 2024 Forum, 2024-01-08 → 2024-01-12, Orlando, United States Part of collection Institutional Repository Document type conference paper Rights © 2024 J.I. de Alvear Cardenas, C.C. de Visser Files PDF de_alvear_c_rdenas_de_vis ... mework.pdf 9.65 MB Close viewer /islandora/object/uuid:19af5ad2-e69d-435e-9539-c89dd885a27f/datastream/OBJ/view