A Hybrid Reinforcement Learning and Tree Search Approach for Network Topology Control

Abstract

The growing demand for electricity, driven by widespread adoption of heat pumps, electric vehicles, and industrial electrification, strains power grids and introduces challenges for a reliable and secure supply amidst intermittent renewable energy integration. Network topology control offers flexibility, altering connections to redirect power flows and mitigate transmission line overloads. This thesis aims to investigate an ML and AI approach to overcome the computational complexity. The proposed approach merges a curriculum-trained machine learning agent with a Monte Carlo Tree Search (MCTS) to enhance power network action security. The MCTS guides the simulation of potential actions, considering future outcomes for improved long-term performance identification. A curriculum-based ML approach is used to pre-train an agent to propose grid actions. MCTS is then used to secure these actions, leveraging outcomes in the training algorithm for enhanced sample efficiency and reduced training times. The approach uses MCTS-verified, simulation-tested actions for immediate training feedback, eliminating the need to wait for scenario completion, enhancing sample efficiency. An electrically distance-guided search in the MCTS improves convergence by prioritising actions closer to overflows, often found to be most influential in reducing violations.