Using convolutional neural networks to automate aircraft maintenance visual inspection

Using convolutional neural networks to automate aircraft maintenance visual inspection

Doğru, Anil (Özyeğin University)
Bouarfa, Soufiane (Abu Dhabi Polytechnic, Al Ain; Delft Aviation)
Arizar, Ridwan (Singular Solutions B.V. Rotterdam)
Aydoğan, Reyhan (TU Delft Interactive Intelligence; Özyeğin University)

2020

Convolutional Neural Networks combined with autonomous drones are increasingly seen as enablers of partially automating the aircraft maintenance visual inspection process. Such an innovative concept can have a significant impact on aircraft operations. Though supporting aircraft maintenance engineers detect and classify a wide range of defects, the time spent on inspection can significantly be reduced. Examples of defects that can be automatically detected include aircraft dents, paint defects, cracks and holes, and lightning strike damage. Additionally, this concept could also increase the accuracy of damage detection and reduce the number of aircraft inspection incidents related to human factors like fatigue and time pressure. In our previous work, we have applied a recent Convolutional Neural Network architecture known by MASK R-CNN to detect aircraft dents. MASK-RCNN was chosen because it enables the detection of multiple objects in an image while simultaneously generating a segmentation mask for each instance. The previously obtained F₁ and F₂ scores were 62.67% and 59.35%, respectively. This paper extends the previous work by applying different techniques to improve and evaluate prediction performance experimentally. The approach uses include (1) Balancing the original dataset by adding images without dents; (2) Increasing data homogeneity by focusing on wing images only; (3) Exploring the potential of three augmentation techniques in improving model performance namely flipping, rotating, and blurring; and (4) using a pre-classifier in combination with MASK R-CNN. The results show that a hybrid approach combining MASK R-CNN and augmentation techniques leads to an improved performance with an F₁ score of (67.50%) and F₂ score of (66.37%).

Aircraft maintenance inspection
Anomaly detection
Convolutional neural networks
Defect inspection
Generative adversarial networks
Image augmentation
Mask R-CNN

http://resolver.tudelft.nl/uuid:42b51411-155c-46db-be55-a4d2da8fe067

https://doi.org/10.3390/aerospace7120171

2226-4310

Aerospace — Open Access Aeronautics and Astronautics Journal, 7 (12), 1-22

Institutional Repository

journal article

© 2020 Anil Doğru, Soufiane Bouarfa, Ridwan Arizar, Reyhan Aydoğan