Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Andriotis, C. (TU Delft Structural Design & Mechanics)
Papakonstantinou, K. G. (The Pennsylvania State University)

2021

Determination of inspection and maintenance policies for minimizing long-term risks and costs in deteriorating engineering environments constitutes a complex optimization problem. Major computational challenges include the (i) curse of dimensionality, due to exponential scaling of state/action set cardinalities with the number of components; (ii) curse of history, related to exponentially growing decision-trees with the number of decision-steps; (iii) presence of state uncertainties, induced by inherent environment stochasticity and variability of inspection/monitoring measurements; (iv) presence of constraints, pertaining to stochastic long-term limitations, due to resource scarcity and other infeasible/undesirable system responses. In this work, these challenges are addressed within a joint framework of constrained Partially Observable Markov Decision Processes (POMDP) and multi-agent Deep Reinforcement Learning (DRL). POMDPs optimally tackle (ii)-(iii), combining stochastic dynamic programming with Bayesian inference principles. Multi-agent DRL addresses (i), through deep function parametrizations and decentralized control assumptions. Challenge (iv) is herein handled through proper state augmentation and Lagrangian relaxation, with emphasis on life-cycle risk-based constraints and budget limitations. The underlying algorithmic steps are provided, and the proposed framework is found to outperform well-established policy baselines and facilitate adept prescription of inspection and intervention actions, in cases where decisions must be made in the most resource- and risk-aware manner.

Constrained stochastic optimization
Decentralized multi-agent control
Deep reinforcement learning
Inspection and maintenance planning
Partially observable Markov decision processes
System risk and reliability

http://resolver.tudelft.nl/uuid:0713e10e-31f1-499d-89dc-940e50faef3a

https://doi.org/10.1016/j.ress.2021.107551

2021-09-11

0951-8320

Reliability Engineering & System Safety, 212

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2021 C. Andriotis, K. G. Papakonstantinou