QRES-MARL

Abstract

This thesis focuses on using MARL as a decision tool for post-earthquake repair scheduling of interdependent infrastructure. MARL is a multi-agent ML paradigm which combines traditional ML research and game-theoretical approaches. Given the relative increase in natural disaster frequency and the lack of available post-disaster tools such tools are crucial in increasing the climate resilience of cities. Given the stochastic nature of earthquake events and subsequent losses, MARL can be helpful in navigating this uncertainty and finding preferable joint policies.The methodology involves multi-scenario-based seismic hazard assessment, stochastic fragility modelling and prediction of several direct and indirect losses to aggregate them into a holistic community resilience metric. This is then used to compute the instantaneous and cumulative recovery resilience loss values. The tested approach uses two custom built test-beds of 4 and 30 components, and MARL is compared against baseline solvers, including random and importance-based policies. Value Decomposition Network with Parameter Sharing (𝑉𝐷𝑁 − 𝑃𝑆), Q-Learning with Mixer Network and Parameter Sharing (𝑄𝑀𝐼𝑋 − 𝑃𝑆), Deep Centralised Multi-Agent Actor Critic (𝐷𝐶𝑀𝐴𝐶) are the algorithms tested. 𝑉𝐷𝑁 and 𝑄𝑀𝐼𝑋 are shown to perform similarly to each other and sub-optimally relative to 𝐷𝐶𝑀𝐴𝐶. 𝐷𝐶𝑀𝐴𝐶 is shown to match importance-based policies when considering full recovery, but convincingly outperforms all other 𝐷𝑅𝐿 methods and importance-based policies when considering partial recovery. This shows that 𝐷𝐶𝑀𝐴𝐶 and 𝐷𝑅𝐿 more generally is effective at swift early recovery by prioritising components that contribute most to community functionality. ii