Engine Shop Visit Scheduling: A Reinforcement Learning Optimization Approach

Abstract

The scheduling of engine shop visits quickly becomes a complex problem to solve as the number of aircraft and engines increases. In recent times, different approaches have been used to tackle this problem and optimize schedules, reducing costs and increasing revenue. This paper formulates the ESV scheduling problem as a Markov Decision Process and develops a reinforcement learning model that includes parameters such as engine performance and life limited parts status, maintenance constraints, and temporal factors. A prioritization algorithm is presented to optimize the learning process and allow for the scheduling of larger fleets. The results show a slight better performance in comparison to a greedy policy when evaluating aircraft availability, flexibility to the initial parameters and reduction in use of spare engines. On the other hand, the reinforcement learning provided lower scores and higher number of removals of aircraft from operations. In conclusion, the methodology proved that reinforcement learning is a viably way to optimize the ESV scheduling process, however a fine tuning of parameters might be necessary to approximate scores to a real revenue and cost relation, and reduce the number of aircraft interventions.