This paper presents a novel distributed model predictive control (MPC) formulation without terminal cost and a corresponding distributed synthesis approach for distributed linear discrete-time systems with coupled constraints. The proposed control scheme introduces an explicit stability condition as an additional constraint based on relaxed dynamic programming. As a result, contrary to other related approaches, system stability with the developed controller does not rely on designing a terminal cost. A distributed synthesis approach is then introduced to handle the stability constraint locally within each local agent. To solve the underlying optimization problem for distributed MPC, a violation-free distributed optimization approach is developed, using constraint tightening to ensure feasibility throughout iterations. A numerical example demonstrates that the proposed distributed MPC approach ensures closed-loop stability for each feasible control sequence, with each agent computing its control input in parallel.

Uncertainty in the behavior of other traffic participants is a crucial factor in collision avoidance for automated driving; here, stochastic metrics could avoid overly conservative decisions. This article introduces a stochastic model predictive control (SMPC) planner for emergency collision avoidance in highway scenarios to proactively minimize collision risk while ensuring safety through chance constraints. To guarantee that the emergency trajectory can be attained, we incorporate nonlinear tire dynamics in the prediction model of the ego vehicle. Further, we exploit max-min-plus-scaling (MMPS) approximations of the nonlinearities to avoid conservatism, enforce proactive collision avoidance, and improve computational efficiency in terms of performance and speed. Consequently, our contributions include integrating a dynamic ego vehicle model into the SMPC planner, introducing the MMPS approximation for real-time implementation in emergency scenarios, and integrating SMPC with hybridized chance constraints and risk minimization. We evaluate our SMPC formulation in terms of proactivity and efficiency in various hazardous scenarios. Moreover, we demonstrate the effectiveness of our proposed approach by comparing it with a state-of-the-art SMPC planner and we validate that the generated trajectories can be attained using a high-fidelity vehicle model in IPG CarMaker.

Greenhouse climate control is concerned with maximizing performance in terms of crop yield and resource efficiency. One promising approach is model predictive control (MPC), which leverages a model of the system to optimize the control inputs, while enforcing physical constraints. However, prediction models for greenhouse systems are inherently inaccurate due to the complexity of the real system and the uncertainty in predicted weather profiles. For model-based control approaches such as MPC, this can degrade performance and lead to constraint violations. Existing approaches address uncertainty in the prediction model with robust or stochastic MPC methodology; however, these necessarily reduce crop yield due to conservatism and often bear higher computational loads. In contrast, learning-based control approaches, such as reinforcement learning (RL), can handle uncertainty naturally by leveraging data to improve performance. This work proposes an MPC-based RL control framework to optimize the climate control performance in the presence of prediction uncertainty. The approach employs a parametrized MPC scheme that learns directly from data, in an online fashion, the parametrization of the constraints, prediction model, and optimization cost that minimizes constraint violations and maximizes climate control performance. Simulations show that the approach can learn an MPC controller that significantly outperforms the current state-of-the-art in terms of constraint violations and efficient crop growth.

Learning-based control, in particularReinforcement Learning (RL) reinforcementReinforcement learning, and optimization-based control, in particular model predictive control, each have their advantages and disadvantages for online, real-timeOptimal control optimal controlOptimal control of systems with complex dynamicsDynamic. However, both approaches are highly complementary and therefore there is an increased interest in combining their advantages in an integrated approach. In this chapter, we provide an overview of recent results, challenges, and opportunities on an integrated learning-based and optimization-based control approach. We focus in particular on piecewise affine systems as they are an extension of linear systemsLinear systems that can model or approximate hybridHybrid or nonlinearNonlinearbehaviorBehavior and as they still allow for effective numerical solutionSolution approaches.

Co-optimization of both vehicle speed and gear position via model predictive control (MPC) has been shown to offer benefits for fuel-efficient autonomous driving. However, optimizing both the vehicle’s continuous dynamics and discrete gear positions may be too computationally intensive for a real-time implementation. This work proposes a learning-based MPC scheme to address this issue. A policy is trained to select and fix the gear positions across the prediction horizon of the MPC controller, leaving a significantly simpler continuous optimization problem to be solved online. In simulation, the proposed approach is shown to have a significantly lower computation burden and a comparable performance, with respect to pure MPC-based co-optimization.

In the backdrop of an increasingly pressing need for effective urban and highway transportation systems, this work explores the synergy between model-based and learning-based strategies to enhance traffic flow management by use of an innovative approach to the problem of ramp metering control that embeds Reinforcement Learning (RL) techniques within the Model Predictive Control (MPC) framework. The control problem is formulated as an RL task by crafting a suitable stage cost function that is representative of the traffic conditions, variability in the control action, and violations of the constraint on the maximum number of vehicles in queue. An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp and satisfy its constraints despite uncertainties in the system model and variable demands. Simulations are performed on a benchmark small-scale highway network to compare the proposed methodology against other state-of-the-art control approaches. Results show that, starting from an MPC controller that has an imprecise model and is poorly tuned, the proposed methodology is able to effectively learn to improve the control policy such that congestion in the network is reduced and constraints are satisfied, yielding an improved performance that is superior to the other controllers.

This paper presents a novel approach for distributed model predictive control (MPC) for piecewise affine (PWA) systems. Existing approaches rely on solving mixed-integer optimization problems, requiring significant computation power or time. We propose a distributed MPC scheme that requires solving only convex optimization problems. The key contribution is a novel method, based on the alternating direction method of multipliers, for solving the non-convex optimal control problem that arises due to the PWA dynamics. We present a distributed MPC scheme, leveraging this method, that explicitly accounts for the coupling between subsystems by reaching agreement on the values of coupled states. Stability and recursive feasibility are shown under additional assumptions on the underlying system. Two numerical examples are provided, in which the proposed controller is shown to significantly improve the CPU time and closed-loop performance over existing state-of-the-art approaches.

Virtual coupling is regarded as an efficient way to improve the line capacity of rail transportation systems by reducing the spacing between consecutive trains. This paper is the first to compare and assess different distributed model predictive control (MPC) approaches, i.e., cooperative distributed MPC, serial distributed MPC, and decentralized MPC, for virtually coupled trains with a nonlinear train dynamic model. To make a balanced trade-off between computational complexity and efficiency, we also propose and assess convex approximations of the above control approaches. Furthermore, we are the first to introduce the relaxed dynamic programming approach to analyze the stability of the MPC-based nonlinear train control problem. By using the relaxed dynamic programming approach, a distributed stopping criterion with a stability guarantee is developed for the cooperative distributed MPC approach. In real life, masses of trains are different and can change at stations due to changes in passenger loads. This change in mass can significantly affect the dynamics and control of the virtually coupled trains when not taken into account in the control design. Therefore, we explicitly consider heterogeneous train masses when designing MPC approaches. We evaluate the different distributed MPC approaches through case studies based on the data of the Beijing Yizhuang Line. Simulation results indicate that the cooperative distributed MPC approach has the best tracking performance, while the serial distributed MPC approach can reduce communication requirements and computation capabilities with sacrifices of tracking performance.

This paper presents a novel approach to multi-agent reinforcement learning (RL) for linear systems with convex polytopic constraints. Existing work on RL has demonstrated the use of model predictive control (MPC) as a function approximator for the policy and value functions. The current paper is the first work to extend this idea to the multi-agent setting. We propose the use of a distributed MPC scheme as a function approximator, with a structure allowing for distributed learning and deployment. We then show that Q-learning updates can be performed distributively without introducing nonstationarity, by reconstructing a centralized learning update. The effectiveness of the approach is demonstrated on a numerical example.

The railway timetable rescheduling problem is regarded as an efficient way to handle disturbances. Typically, it is tackled using a mixed integer linear programming (MILP) formulation. In this paper, an algorithm that combines both reinforcement learning and optimization is proposed to solve the railway timetable rescheduling problem. Specifically, a value-based reinforcement learning algorithm is implemented to determine the independent integer variables of the MILP problem. Then, the values of all the integer variables can be derived from these independent integer variables. With the solution for the integer variables, the MILP problem can be transformed into a linear programming problem, which can be solved efficiently. The simulation results show that the proposed method can reduce passenger delays compared with the baseline, while also reducing the solution time.

Real-time timetable scheduling is an effective way to improve passenger satisfaction and to reduce operational costs in urban rail transit networks. In this paper, a novel passenger-oriented network model is developed for real-time timetable scheduling that can model time-dependent passenger origin-destination demands with consideration of a balanced trade-off between model accuracy and computation speed. Then, a model predictive control (MPC) approach is proposed for the timetable scheduling problem based on the developed model. The resulting MPC optimization problem is a nonlinear non-convex problem. In this context, the online computational complexity becomes the main issue for the real-time feasibility of MPC. To reduce the online computational complexity, the MPC optimization problem is therefore reformulated into a mixed-integer linear programming (MILP) problem. The resulting MILP problem is exactly equivalent to the original MPC optimization problem and can be solved very efficiently by existing MILP solvers, so that we can obtain the solution very fast and realize real-time timetable scheduling. Numerical experiments based on a part of Beijing subway network show the effectiveness and efficiency of the developed model and the MILP-based MPC method.

This paper deals with the train scheduling problem for metro networks taking into account time-dependent passenger origin–destination demands and train speed profiles. The aim is to adjust train schedules online according to time-dependent passenger demands so that passenger satisfaction and operational costs are jointly optimized. An extended passenger absorption model that explicitly includes time-dependent passenger origin–destination demands is developed, where the term “absorption” refers to passengers boarding trains. Then, the passenger absorption model is extended to a bi-level framework, where passenger demands and rolling stock availability are considered at the higher level, and detailed timetables and train speed profiles are included at the lower level. A bi-level model predictive control (MPC) approach is developed for the integrated problem. The optimization problems of both levels of the bi-level MPC approach can be converted into mixed-integer linear programming (MILP) problems, which enables us to solve them with existing MILP solvers. We then show that the recursive feasibility of both the higher-level and the lower-level optimization problems can be guaranteed. In this way, we can achieve real-time train scheduling for the metro system. Numerical experiments, based on real-life data from the Beijing metro network, illustrate the effectiveness of the extended passenger absorption model and the proposed bi-level MPC approach.

This paper proposes a method to encourage safety in Model Predictive Control (MPC)-based Reinforcement Learning (RL) via Gaussian Process (GP) regression. The framework consists of 1) a parametric MPC scheme that is employed as model-based controller with approximate knowledge on the real system's dynamics, 2) an episodic RL algorithm tasked with adjusting the MPC parametrization in order to increase its performance, and 3) GP regressors used to estimate, directly from data, constraints on the MPC parameters capable of predicting, up to some probability, whether the parametrization is likely to yield a safe or unsafe policy. These constraints are then enforced onto the RL updates in an effort to enhance the learning method with a probabilistic safety mechanism. Compared to other recent publications combining safe RL with MPC, our method does not require further assumptions on, e.g., the prediction model in order to retain computational tractability. We illustrate the results of our method in a numerical example on the control of a quadrotor drone in a safety-critical environment.

Timetables determine the service quality for passengers and the energy consumption of trains in metro systems. In metro networks, a timetable can be made by designing train departure frequencies for different periods of the day, which is typically formulated as a mixed-integer linear programming (MILP) problem. In this paper, we first apply Benders decomposition to optimize the departure frequencies considering time-varying passenger origin-destination demands in metro networks. An ϵ -optimal Benders decomposition approach is subsequently used to reduce the solution time further. The performance of both methods is illustrated in a simulation-based case study using a grid metro network. The results show that both the classical Benders decomposition approach and the ϵ -optimal Benders decomposition approach can significantly reduce the computation time for the optimization of train departure frequencies in metro networks. In addition, the ϵ -optimal Benders decomposition approach can further reduce the solution time compared to the classical Benders decomposition approach when the problem scale increases while maintaining an acceptable level of performance.

Real-time train scheduling is essential for passenger satisfaction in urban rail transit networks. This paper focuses on real-time train scheduling for urban rail transit networks considering uncertain time-dependent passenger origin-destination demands. First, a macroscopic passenger flow model we proposed before is extended to include rolling stock availability. Then, a distributed-knowledgeable-reduced-horizon (DKRH) algorithm is developed to deal with the computational burden and the communication restrictions of the train scheduling problem in urban rail transit networks. For the DKRH algorithm, a cost-to-go function is designed to reduce the prediction horizon of the original model predictive control approach while taking into account the control performance. By applying a scenario reduction approach, a scenario-based distributed-knowledgeable-reduced-horizon (S-DKRH) algorithm is proposed to handle the uncertain passenger flows with an acceptable increase in computation time. Numerical experiments are conducted to evaluate the effectiveness of the developed DKRH and S-DKRH algorithms based on real-life data from the Beijing urban rail transit network. The simulation results indicate that DKRH can be used to achieve real-time train scheduling for the urban rail transit network, while S-DKRH can handle the uncertainty in the passenger flows with an acceptable sacrifice in computation time.