Model-based reinforcement learning for predictive control and optimisation
F. Airaldi (TU Delft - Team Azita Dabiri)
B. De Schutter – Promotor (TU Delft - Delft Center for Systems and Control)
A. Dabiri – Copromotor (TU Delft - Team Azita Dabiri)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
In the current age of emerging autonomous and artificial-intelligence-driven machines, sequential decision making constitutes one of the theoretical fundaments at the core of intelligent agency. As these systems are increasingly deployed in real-world engineering applications (e.g., autonomous vehicles and drones, as well as smart energy grids and greenhouses), there is a growing need for the control architectures governing these agents to meet, aside from traditional performance requirements, also interpretability and safety criteria, while encouraging adaptability and scalability. Classical model-based methodologies, such as Model Predictive Control (MPC), can in general provide rigorous frameworks that integrate a priori knowledge (e.g., via explicit, though often approximate, prediction models) and can handle constraints to enforce safety, yet their performance is tightly coupled with the accuracy of the underlying model and expert manual tuning of its parameters. Conversely, purely model-free approaches, such as deep Reinforcement Learning (RL), offer remarkable data-driven adaptation, but often lack interpretability and reliable constraint handling required to provide formal guarantees.
This work expands on the current state-of-the-art results that combine these two distinct approaches into a single framework. While not always straightforward, it is well known that endowing these decision-making processes with model-based knowledge can not only enhance their performance but also benefit their interpretability and analysis: model-based RL, also known as Approximate Dynamic Programming (ADP), is perhaps the most renowned machine learning paradigm to craft these intelligent predictive agents. This dissertation aims to look at RL from a different perspective. Instead of as an alternative to model-based control, RL is used as a performance-enhancing mechanism operating within rigorously defined safety requirements. Concurrently, this thesis establishes MPC as a unifying and scalable foundation block for learning-based control and optimisation for constrained, uncertain, and distributed decision-making systems......