A Hybrid Reinforcement Learning and Tree Search Approach for Network Topology Control

A Hybrid Reinforcement Learning and Tree Search Approach for Network Topology Control

Meppelink, Geert jan (TU Delft Electrical Engineering, Mathematics and Computer Science)

Cremer, Jochen (mentor)
Rajaei, A. (mentor)

Delft University of Technology
Norwegian University of Science and Technology (NTNU)

European Wind Energy Masters (EWEM) | Rotor Design Track

2023-12-22

The growing demand for electricity, driven by widespread adoption of heat pumps, electric vehicles, and industrial electrification, strains power grids and introduces challenges for a reliable and secure supply amidst intermittent renewable energy integration. Network topology control offers flexibility, altering connections to redirect power flows and mitigate transmission line overloads. This thesis aims to investigate an ML and AI approach to overcome the computational complexity. The proposed approach merges a curriculum-trained machine learning agent with a Monte Carlo Tree Search (MCTS) to enhance power network action security. The MCTS guides the simulation of potential actions, considering future outcomes for improved long-term performance identification. A curriculum-based ML approach is used to pre-train an agent to propose grid actions. MCTS is then used to secure these actions, leveraging outcomes in the training algorithm for enhanced sample efficiency and reduced training times. The approach uses MCTS-verified, simulation-tested actions for immediate training feedback, eliminating the need to wait for scenario completion, enhancing sample efficiency. An electrically distance-guided search in the MCTS improves convergence by prioritising actions closer to overflows, often found to be most influential in reducing violations.

Machine Learning
Reinforcement Learning
Network Topology Control
Monte-Carlo Tree Search
Transmission Control
L2RPN

http://resolver.tudelft.nl/uuid:3f2b53e9-fc73-49e0-8b7d-f9e62bb2a419

Student theses

master thesis

© 2023 Geert jan Meppelink