Assessing the Impact of Metrics on the Choice of Prognostic Methodologies
M.T. Bieber (TU Delft - Air Transport & Operations)
W.J.C. Verhagen (Royal Melbourne Institute of Technology University)
B. F. Lopes dos Santos (TU Delft - Air Transport & Operations)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Over the past years, advanced prognostic models and approaches have been developed. Most existing approaches are tailored to one specific system and cannot adaptively be used on different systems. This can lead to years of research and expertise being put into implementing prognostic models without the capacity to predict system failures, either because of a lack of data or data quality or because failure behavior cannot be captured by data-driven models. In addition, prognostic models are often evaluated using metrics only related to the correctness of predictions, preventing meaningful evaluation of operational performance. This paper makes use of a framework that can automatically choose prognostic settings based on specific system data. It simultaneously optimizes the choice of methodologies using metrics that capture multiple aspects of prediction quality. We apply this framework to both a simulated data set and a real aircraft data set to characterize the impact of metrics on the choice of prognostic methodologies. The results show that the choice of optimization metric greatly impacts the output of the generic prognostic framework and the overall performance. In addition, a definition for data suitability is provided and assessed on the aircraft system data sets.