Optimising Grid Topology Reconfiguration using Reinforcement Learning

Abstract

The de-carbonisation of the energy system, more commonly known as the 'Energy Transition' has a vital role to play in the pursuit of mitigating the climate emergency’s impact. There is a global trend of moving toward making power systems more future-proof and this largely affects the roles and activities of a Transmission System Operator (TSO) such as TenneT.

The particular branch of power grid control and operation is one that is undergoing a massive overhaul. Control room operators are highly-skilled and need to be aware of all the physical processes involved in the power system in order to manually intervene when necessary. They are constantly monitoring the grid and are tasked with making quick decisions with a high frequency which rely on many factors such as the expected active power of conventional generator units, the expected demand and also the renewable energy forecasts for a given time instant. One important action that is often considered by the control room operators is reconfiguring the network topology. Making topological changes to the network offers a flexibility that is an under-exploited and low-cost alternative to maintain network security. These operational tasks are getting more complex and nuanced with the addition of newer technologies.

There is specifically an increasing reliance on Information, Communication and Technology (ICT) and newer smart grid technology which can assist in many ways such as higher levels of automation, increase in computation speed and so on. One particular field of study under the umbrella of ICT is the application of Artificial Intelligence (AI) solutions. These solutions have the potential to pave the path to better cyber-physical systems with which large strides can be made to cope with many challenges introduced by the 'Energy Transition'.

Various initiatives are being undertaken in this realm. One such initiative being Réseau de Transport d’Électricité (RTE)’s initiative of  "Learning To Run a Power Network (L2PRN)" competition. This competition was conducted with the primary goal of introducing and recognising the potential of AI and machine learning-based tools in order to support control rooms and assist in making optimal decisions. One particular branch of AI that has shown great promise in the field of decision support is Reinforcement Learning.

This competition acts as a great starting point to bring together these two almost exclusive research communities. The research conducted in this thesis uses this competition as a stepping stone along with the tool chain developed for it, to investigate the use of an AI-based solution and test the behaviour of the agent, not just from a machine learning point of view, but also from a power system perspective. This thesis addresses the potential of machine learning as a decision support tool for power system control rooms by implementing a reinforcement learning algorithm to represent an artificial control room operator and assessing its performance on a particular IEEE test network. This thesis also hopes to provide some groundwork for TenneT and to contribute toward a ‘Control Room of the Future’ initiative which can incorporate such an AI-based decision support tool to assist grid operators in taking well-informed actions during the operation of the power systems.