Reinforcement Learning Methodology for Electricity Market Simulation
C.R. Whitman (TU Delft - Electrical Engineering, Mathematics and Computer Science)
L.J. De Vries – Mentor (TU Delft - Energy and Industry)
Jochen L. Cremer – Mentor (TU Delft - Intelligent Electrical Power Grids)
V. Zobernig – Mentor (TU Delft - Energy and Industry)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
This work seeks to resolve an outstanding problem in the use of reinforcement-learning methods for the simulation of economically-rational agents. We discuss the problem of non-stationarity, and how this subsequently limits market simulation capabilities. After explicating and isolating the source of the problem for a day-ahead electricity market, we demonstrate the application of methods which resolve this problem in simple test-cases, and prove conditions under which similar methods will work in general. Subsequently, we illustrate how these techniques can be used to solve a restricted market-design problem, in the process introducing a framework for discussing adversarial market-design for electricity markets in general. It is hoped that, insofar as they provide a new feedback-loop for market-design, these results will facilitate the design of more complex electricity-markets suitable for the energy transition.