Approximate Dynamic Programming for Constrained Piecewise Affine Systems With Stability and Safety Guarantees

Journal Article (2025)
Author(s)

K. He (TU Delft - Team Bart De Schutter)

S. Shi (Massachusetts Institute of Technology)

Ton Van Den Boom (TU Delft - Team Ton van den Boom)

B De Schutter (TU Delft - Delft Center for Systems and Control)

Research Group
Team Bart De Schutter
DOI related publication
https://doi.org/10.1109/TSMC.2024.3515645
More Info
expand_more
Publication Year
2025
Language
English
Research Group
Team Bart De Schutter
Issue number
3
Volume number
55
Pages (from-to)
1722-1734
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Infinite-horizon optimal control of constrained piecewise affine (PWA) systems has been approximately addressed by hybrid model predictive control (MPC), which, however, has computational limitations, both in offline design and online implementation. In this article, we consider an alternative approach based on approximate dynamic programming (ADP), an important class of methods in reinforcement learning. We accommodate nonconvex union-of-polyhedra state constraints and linear input constraints into ADP by designing PWA penalty functions. PWA function approximation is used, which allows for a mixed-integer encoding to implement ADP. The main advantage of the proposed ADP method is its online computational efficiency. Particularly, we propose two control policies, which lead to solving a smaller-scale mixed-integer linear program than conventional hybrid MPC, or a single convex quadratic program, depending on whether the policy is implicitly determined online or explicitly computed offline. We characterize the stability and safety properties of the closed-loop systems, as well as the suboptimality of the proposed policies, by quantifying the approximation errors of value functions and policies. We also develop an offline mixed-integer-linear-programming-based method to certify the reliability of the proposed method. Simulation results on an inverted pendulum with elastic walls and on an adaptive cruise control problem validate the control performance in terms of constraint satisfaction and CPU time.