Multi-modal transport is getting more popular due to the emergence of new traffic
modes. The increase of modes also adds complexity for the transport researchers.
This paper proposes an augmented link-based super-network approach for modeling
multi-modal transport networks, addressing the scalability and versatility issues of
conventional methods. This approach is used to calculate the user equilibrium for
urban transport networks traffic assignment with multiple traffic modes, a difficult
problem due to the intractable enumeration of feasible paths between origindestination
pairs and restricted transfers between different traffic modes. In the supernetwork
representation of multi-modal transport networks, the travel cost of any
feasible route between the origin and destination is formulated as the sum of cost
functions of the augmented links, thus avoiding the enumeration of feasible paths.
Additionally, restrictions on traffic mode transfers can be embedded in the link-based
model by excluding infeasible transfer links or adding penalties for undesired transfers.
The user equilibrium of the augmented link-based super-network model is formulated
as a variational inequality problem, solved using the extra-gradient algorithm. A multimodal
transport network is considered in the case study. Simulation results validate the
effectiveness of the proposed model, demonstrating its scalability and versatility in
addressing complex multi-modal transport networks with diverse traffic modes.
We anticipate that our method can serve as an efficient modeling approach for more
general and complex multi-modal transport networks, facilitating traffic management
and network design.

The increasing urbanization, combined with shrinking space for transport infrastructure and private parking, significantly challenges urban accessibility. Moreover, the rising number of vehicles exacerbates congestion in city centers, leading to longer commute times, increased noise levels, and greater air pollution. These issues underscore the urgent need for creating low-car urban zones. One promising approach is an integrated traffic management system that considers various modes of transportation—such as cycling, walking, shared mobility, and public transport. However, multi-modal traffic management typically involves diverse stakeholders with potentially conflicting interests, which necessitates a balance of these interests through multi-objective optimization. Traditional approaches often employ a weighted sum method to transform multiple objectives into a single objective. This method significantly constrains the solution space and complicates the assignment of appropriate weights to different objectives. Therefore, generating a Pareto front for multi-modal traffic management could provide decision-makers with a set of efficient solutions, enabling them to select the most suitable option. The ε-constraint method is recognized for its ability to generate a Pareto front. The question we discuss here is whether this method can be effectively applied to managing multi-objective, multi-modal traffic networks. In this study, we answer this question by proposing an augmented ε-constraint-based optimization framework for multi-objective multi-modal traffic management. This framework is bi-level and can accommodate various traffic models and objectives that reflect the diverse interests of multiple stakeholders. Thus the multi-modal traffic management problem can be formulated as a multi-objective nonlinear optimization problem. The augmented ε-constraint method (Mavrotas, 2009) is employed to efficiently address the multiple objectives, and the multi-start sequential quadratic programming method is used to solve the nonlinear optimization problems, such that the Pareto front is obtained. We validate the effectiveness of our framework through a case study, whose preliminary results show that our method improves the traffic performance and provides insights into the trade-off among different objectives.

Parameterized model predictive control (PMPC) is one of the many approaches that have been developed to alleviate the high computational requirement of model predictive control (MPC), and it has been shown to significantly reduce the computational complexity while providing comparable control performance with conventional MPC. However, PMPC methods still require a sufficiently accurate model to guarantee the control performance. To deal with model mismatches caused by the changing environment and by disturbances, this paper first proposes a novel framework that uses reinforcement learning (RL) to adapt all components of the PMPC scheme in an online way. More specifically, the novel framework integrates various strategies to adjust different components of PMPC (e.g., objective function, state-feedback control function, optimization settings, and system model), which results in a synthesis framework for RL-based adaptive PMPC. We show that existing adaptive (P)MPC approaches can also be embedded in this synthesis framework. The resulting combined RL-PMPC framework provides a solution for an efficient MPC approach that can deal with model mismatches. A case study is performed in which the framework is applied to freeway traffic control. Simulation results show that for the given case study the RL-based adaptive PMPC approach reduces computational complexity by 98% on average compared to conventional MPC while achieving better control performance than the other controllers, in the presence of model mismatches and disturbances.

Model predictive control (MPC) and deep reinforcement learning (DRL) have been developed extensively as two independent techniques for traffic management. Although the features of MPC and DRL complement each other very well, few of the current studies consider combining these two methods for application in the field of freeway traffic control. This paper proposes a novel framework for integrating MPC and DRL methods for freeway traffic control that is different from existing MPC-(D)RL methods. Specifically, the proposed framework adopts a hierarchical structure, where a high-level efficient MPC component works at a low frequency to provide a baseline control input, while the DRL component works at a high frequency to modify online the output generated by MPC. The control framework, therefore, needs only limited online computational resources and is able to handle uncertainties and external disturbances after proper learning with enough training data. The proposed framework is implemented on a benchmark freeway network in order to coordinate ramp metering and variable speed limits, and the performance is compared with standard MPC and DRL approaches. The simulation results show that the proposed framework outperforms standalone MPC and DRL methods in terms of total time spent (TTS) and constraint satisfaction, despite model uncertainties and external disturbances.

The railway timetable rescheduling problem is regarded as an efficient way to handle disturbances. Typically, it is tackled using a mixed integer linear programming (MILP) formulation. In this paper, an algorithm that combines both reinforcement learning and optimization is proposed to solve the railway timetable rescheduling problem. Specifically, a value-based reinforcement learning algorithm is implemented to determine the independent integer variables of the MILP problem. Then, the values of all the integer variables can be derived from these independent integer variables. With the solution for the integer variables, the MILP problem can be transformed into a linear programming problem, which can be solved efficiently. The simulation results show that the proposed method can reduce passenger delays compared with the baseline, while also reducing the solution time.

Distributed model predictive control (DMPC) has become a focus in the energy management of shipboard power systems due to its capabilities for privacy preservation, robustness, and distributing computing burdens to local processors. DMPC determines control actions in a distributed manner based on the predictions of system statuses. However, the performance of DMPC is affected by inaccurate predictions resulting from uncertain parameters in nominal prediction models. Particularly, these inaccuracies in predicting propulsion loads and solar panel generation powers can lead to power imbalances when implementing the control actions determined by DMPC. To address this challenge, this paper proposed a novel reinforcement learning compensated DMPC (RL-C-DMPC) to distributively compensate for the control actions determined by DMPC baseline control, thereby rectifying the power imbalances caused by uncertain parameters in nominal prediction models. A value-decomposition-network-based training and distributed testing mechanism is designed for our proposed RL-C-DMPC. Furthermore, a method for range selection of compensation rate is specifically proposed for the energy management of shipboard power systems. To validate the effectiveness of our proposed RL-C-DMPC, we conduct a comprehensive case study utilizing real-life voyage data and historical solar power generation data in the area of the voyage to build the environment for training and testing. By comparing power imbalances between DMPC and RL-C-DMPC, our results indicate significant reductions in power imbalances so that frequency stability can be better ensured. Furthermore, via the case study, we also evaluate the communication robustness of RL-C-DMPC.

Model predictive control (MPC) has been widely used for traffic management, such as for minimizing the total time spent or the total emissions of vehicles. When long-term green urban mobility is considered including e.g. a constraint on the total yearly emissions, the optimization horizon of the MPC problem is significantly larger than the control sampling time, and thus the number of the variables that should be optimized per control time step becomes very large. For systems with dynamics that involve nonlinear, non-convex, and non-smooth functions, including urban traffic networks, this results in optimization problems that are computationally intractable in real time. In this paper, we propose a novel bi-level temporal distribution of such complex MPC optimization problems, and we develop two mathematically linked short-term and long-term MPC formulations with small and large control sampling times that will be solved together instead of the original complex optimization problem. The resulting bi-level control architecture is used to solve the two MPC formulations online for real-time control of urban traffic networks with the objective of long-term green mobility. In order to assess the performance of the bi-level control architecture, we perform a case study where a rough version of the model of the urban traffic flow, S-model, is used by the long-term MPC level to estimate the states of the urban traffic networks, and a detailed version of the model is used by the short-term MPC level. The results of the simulations prove the effectiveness (with respect to the objective of control, as well as computational efficiency) of the proposed bi-level MPC approach, compared to state-of-the-art control approaches.

This thesis focuses on management and control of traffic networks, including urban networks and freeway networks, in which we aim to reduce traffic congestion by minimizing the total time spent of all the vehicles in the network, and also consider green mobility by minimizing the total emissions produced by the vehicles. In this thesis, we have addressed the challenges of model predictive control (MPC) for traffic management in terms of computational complexity and model mismatches by developing several novel MPC-based control frameworks for urban and freeway traffic networks. More specifically, several multi-level and learning-based MPC control frameworks are proposed. First, a novel bi-level temporally-distributed MPC framework is proposed to deal with the green urban mobility issue that usually involves long-term (e.g., one year) emission constraints, and is thus computationally intractable due to the large window of the problem. Second, we employ a grammatical evolution method to generate parameterized control laws for parameterized MPC (PMPC) with application to urban traffic signal control. Third, we develop a novel combined MPC- deep reinforcement learning (DRL) multi-level control framework, in which the MPC module provides a basic control performance at a lower frequency based on a prediction model, and the DRL module works at a higher frequency to compensate for the model mismatches and external disturbances through learning. Forth, we propose a synthesis framework of reinforcement learning (RL)-based adaptive PMPC. In this framework, all components of the PMPC scheme, such as the cost function, the prediction model, the control law, the constraint set, and the terminal set, can be parameterized and adjusted by a high-level RL agent.

While Model Predictive Control (MPC) is a promising approach for network-wide control of urban traffic, the computational complexity of the, often nonlinear, online optimization procedure is too high for real-time implementations. In order to make MPC computationally efficient, this paper introduces a parameterized MPC (PMPC) approach for urban traffic networks that uses Grammatical Evolution to construct continuous parameterized control laws using an effective simulation-based training framework. Furthermore, a projection-based method is proposed to remove the nonlinear constraints that are imposed on the parameters of the parameterized control laws and to guarantee the feasibility of the solution of the MPC optimization problem. The performance and computational efficiency of the constructed parameterized control laws are compared to those of a conventional MPC controller in an extensive simulation-based case study. The results show that the parameterized control laws, which are automatically constructed using Grammatical Evolution, decrease the computational complexity of the online optimization problem by more than 80% with a decrease in performance by less than 10%.

We propose a novel method to improve the convergence performance of model predictive control (MPC) for setpoint tracking, by introducing sub-references within a multilevel MPC structure. In some cases, MPC is implemented with a short prediction horizon due to limited on-line computation capacity, which could lead to deteriorated dynamic performance. The introduced multi-level optimization method can generate proper sub-references for the MPC setpoint tracking problem, and efficiently improve the dynamic performance. In the higher level a specific performance criterion is taken as the objective, while explicit MPC is utilized in the lower level to represent the control input. The generated sub-references are then used in MPC for the real system with prediction horizon restrictions. Setpoint-tracking MPC for linear systems is used to illustrate the approach throughout the paper. Numerical simulations show that MPC with sub-references significantly improves the convergence performance compared with regular MPC with the same prediction horizon. Thus, it can be concluded that MPC with sub-references has a high potential to tackle more complicated control problems with limited computation capacity.

Traffic control is essential to reduce congestion in both urban and freeway traffic networks. These control measures include ramp metering and variable speed limits for freeways, and traffic signal control for urban traffic. However, current traffic control methods are either too simple to respond to complex traffic environment, or too sophisticated for real-life implementation. In this paper, we propose an adaptive parameterized control method for traffic management by using reinforcement learning algorithms. This method takes advantage of the simple structure of parameterized state-feedback controllers for traffic; meanwhile, a reinforcement learning agent is employed to adjust the parameters of the controllers on-line to react to the varying environment. Therefore, the proposed method requires limited real-time computational efforts, and is adaptive to external disturbances. Furthermore, the reinforcement learning agent can coordinate multiple local traffic controllers when adjusting their parameters. The method is validated by a numerical case study on a freeway network. Results show that the proposed method outperforms conventional controllers when the system is exposed to a changing environment.

In general, the performance of model-based controllers cannot be guaranteed under model uncertainties or disturbances, while learning-based controllers require an extensively sufficient training process to perform well. These issues especially hold for large-scale nonlinear systems such as urban traffic networks. In this paper, a new framework is proposed by combining model predictive control (MPC) and reinforcement learning (RL) to provide desired performance for urban traffic networks even during the learning process, despite model uncertainties and disturbances. MPC and RL complement each other very well, since MPC provides a sub-optimal and constraint-satisfying control input while RL provides adaptive control laws and can handle uncertainties and disturbances. The resulting combined framework is applied for traffic signal control (TSC) of an urban traffic network. A case study is carried out to compare the performance of the proposed framework and other baseline controllers. Results show that the proposed combined framework outperforms conventional control methods under system uncertainties, in terms of reducing traffic congestion.