Circular Image

Y. Zhu

info

Please Note

20 records found

A graph neural network based deep reinforcement learning method for the train platforming and rescheduling problem

Journal article (2026) - Hongxiang Zhang, Andrea D’Ariano, Yongqiu Zhu, Yaoxin Wu, Liuyang Hu, Gongyuan Lu
The train platforming schedule is the crucial plan for guiding trains to travel through a railway station without spatial and temporal conflicts. When trains are delayed in arriving at the station due to disturbances or disruptions, it raises the Train Platforming and Rescheduling Problem (TPRP), one of the hot topics in railway traffic management. It focuses on allocating platforms and time slots for trains to reduce delays and ensure operational efficiency in a station. This paper introduces a novel graph neural network based deep reinforcement learning method to address this problem, named Learning to Reschedule Platforms (L2RP). We formulate the solving process of TPRP as a customized Markov decision process. Meanwhile, we integrate a microscopic discrete-event train operation simulation model to serve as the agent exploration environment, which provides states, executes actions, and completes transitions. Then, we design a hybrid graph neural network based policy network to derive high-quality actions under each graph encoded state.The policy network is trained with the reward function designed to minimize total train knock-on delays and platform changes. The experiments on real-world instances show that the proposed L2RP method can produce high-quality solutions for instances of various scenarios within stably short solving times. ...
Journal article (2026) - Pu Zhang, Lingyun Meng, Yongqiu Zhu, Jianrui Miao, Xiaojie Luan, Zhengwen Liao
This paper proposes a value-based deep reinforcement learning approach that is capable of handling train timetable rescheduling under both disturbed and disrupted situations. A railway environment is constructed to simulate the problem as a Markov decision process, where the optimization objective is integrated into the reward module and various constraints are incorporated into the conflict detection and avoidance module. To address the challenges of sparse rewards and large action space with limited legal actions, a value-based algorithm framework is proposed to efficiently select and effectively evaluate actions. Through the designed simulation and training procedures, the proposed approach is tested on several disturbance and disruption cases based on a real-world instance (i.e. a Chinese high-speed railway corridor). Experimental results show that the proposed method can obtain high-quality solutions within a reasonable computing time, and also outperforms handcrafted rules in terms of the optimality of solutions. Furthermore, the proposed method exhibits promising generalization capabilities in homogeneous perturbation scenarios (disturbance scenarios and disruption scenarios that share either the same affected location and start time or the same affected location and disrupted duration). ...
Railway systems suffer from disturbances in operations, such as extended section running times caused by temporary speed restrictions and prolonged dwell times at stations due to unexpected passenger volumes. These disturbances cause deviations from the original timetable and negatively impact service reliability and passenger experience. Effective and timely rescheduling measures are crucial in reducing the impact of these disturbances. Existing timetable rescheduling models that rely on optimization-based methods often struggle with computational inefficiency, especially when dealing with scenarios involving a large number of train services. To address these challenges, we propose a learning-based timetable rescheduling framework that considers scalability in its formulation to reduce the growing computational burden associated with an increasing number of train services. The proposed framework decomposes the complex rescheduling problem into multiple subtasks, facilitating a systematic approach to managing extensive railway networks with numerous stations and train services. A high-level agent, functioning as a centralized traffic controller, is responsible for decomposing the overall deviation reduction task into subtasks at a low level and assigning them to individual train services with the primary objective of minimizing the time required to restore the original timetable. Low-level agents, acting as distributed train dispatchers, are tasked with rescheduling the timetables of their assigned trains. These low-level agents employ various dispatching strategies, facilitated by inter-train communication, to search for optimal rescheduling solutions while adhering to operational constraints such as minimum headway requirements. The lowlevel agents utilize an actor-critic architecture to generate continuous control decisions for dwell and running times, enabling them to learn and optimize their performance. Knowledge-sharing mechanisms amongst the low-level agents enable faster and more robust learning. Furthermore, advanced exploration methods are integrated to enhance the efficiency of the agents' training process. ...

A heterogeneous multi-agent reinforcement learning framework

Journal article (2025) - Enze Liu, Shuguang Zhan, Yongqiu Zhu, Zhiyuan Lin, Dian Wang
With growing demand straining urban transit systems’ resilience in managing outburst passenger flows, existing approaches focused on offline and single-modal evacuations remain limited. This study proposes an online multi-modal evacuation framework that coordinates on-duty taxis, buses, and metros while minimizing impact on their regular services. We develop a data-driven agent-based environment to update multi-modal transit data and stranded passenger information in real time. Two coordination strategies are introduced: (1) an independent strategy using a decentralized training and distributed execution algorithm, and (2) a collaborative strategy using a hybrid centralized training and distributed execution algorithm. To dynamically assess evacuation effectiveness, we design a resilience framework with three metrics: robustness, rapidity, and resourcefulness. These metrics are transformed into demand-responsive feedback at each time step, enabling agents to proactively generate resilient evacuation plans. In a real-world case study triggered by a railway disruption, our approach outperforms genetic algorithms and multi-agent deep deterministic policy gradient algorithms in computation time and solution quality under offline conditions. Simulated new environments further validate its online applicability, demonstrating its potential for real-world deployment. ...

A Deep Reinforcement Learning Method for the Train Platforming and Rescheduling Problem

Abstract (2025) - Hongxiang Zhang, Yongqiu Zhu, Liuyang Hu, Andrea D’Ariano, Yaoxin Wu, Gongyuan Lu
This paper proposes the Learning to Platforming (L2P) method, a novel graph neural network based deep reinforcement learning method, to solve the Train Platforming and Rescheduling Problem (TPRP). We customize a Markov decision process (MDP) to formulate the solving process of TPRP, utilizing a graph structure to represent states of trains, routes, and berthing tracks from a microscopic perspective. Then, we design a hybrid graph neural network named hAI-GNN to learn informative node embeddings on the graph encoded state. These embeddings are utilized to derive an effective action from the lightweight action space of MDP, which is associated with the decision object train under the state. A discrete-event simulation model is employed to serve as the environment of MDP and implement state transition mechanism. The hAI-GNN based policy network is trained by the Proximal Policy Optimization (PPO) algorithm with the reward function designed to minimize total knock-on delay trains and platform changes. The experiments on real-world instances show that the proposed L2P method can obtain high-quality solutions for both small and large scale instances within very short solving times. ...
Metro networks face operational challenges due to increasing ridership and system growth, particularly in managing delay propagation. Epidemiology models have recently been an interesting method in transportation research for studying delays. This study, therefore, aims to investigate if the Susceptible-infectious-susceptible (SIS) model is suitable to help model delay propagation in a metro network through its ability to reproduce the vulnerability of metro stations for specific instances. Using data from the Washington Metro Network, two groups of delay propagation instances were selected and used for model training and testing using a differential evolution algorithm. The results indicate that the vulnerability values as calculated from the reallife data do not follow the expected trend. Still, our model can capture this variation with good vulnerability estimation accuracy for both groups. Also, the predicted vulnerability values for the first group are more accurate than for the second group. However, limitations such as underestimation and overestimation of station vulnerabilities, and sensitivity to training data were observed. These challenges stemmed from the dynamics between specific parameters and the lack of additional factors. ...
Recent research in Energy-Efficient Train Control (EETC) and Energy-Efficient Train Timetabling (EETT) has uncovered various strategies that can be utilized to reduce railway energy consumption without placing additional demands on the capacity or compromising the robustness of operations. Several railway undertakings have already integrated aspects of these methodologies both in daily operations by the implementation of Driver Advisory Systems (DAS) and in the timetable design process. The major passenger railway operator in the Netherlands, Nederlandse Spoorwegen (NS), utilizes a tablet-based DAS that provides coasting advice to train drivers, while also displaying the route, timetable, temporary speed restrictions and blocks occupied by preceding traffic. Despite the implementation of this system, historical trajectory data from real world operations in the Netherlands indicate variances in the extent of energy-efficient train driving application. These variations could lessen the energy-savings of EETC and increase operational costs. Hence, the main aim of this poster is to evaluate the application of the EETC strategy in real world operations under varying environmental and operational conditions based on historical timetable and train trajectory data, while identifying the causes leading to the observed differences. Subsequently, a literature review of train trajectory optimization techniques is conducted to examine the extent to which these causes are addressed. Finally, the real world applicability of these methods is discussed and future research directions are provided. ...

A review of the literature and future research directions

Journal article (2024) - Shuguang Zhan, Jiemin Xie, S. C. Wong, Yongqiu Zhu, Francesco Corman
External and internal factors can cause disturbances or disruptions in daily train operations, leading to deviations from official timetables and passenger delays. As a result, efficient train timetable rescheduling (TTR) methods are necessary to restore disrupted train services. Although TTR has been a popular research topic in recent years, the uncertain characteristics of railways have not been sufficiently addressed. This review first identifies the primary uncertainties of TTR and examines their impacts on both TTR and passenger routing during disturbances or disruptions. It finds that only a few uncertainties have been investigated, and the existing solution methods do not adequately meet practical requirements, such as considering the dynamic nature of disturbances or disruptions, which is crucial for real-world applications. Therefore, the review highlights problems associated with TTR uncertainties that need urgent attention and suggests promising methodologies that could effectively address these issues as future research directions. This review aims to help practitioners develop improved automatic train-dispatching systems with better train-rescheduling performance under disturbances or disruptions compared to current systems. ...
Journal article (2021) - Yongqiu Zhu, Rob M.P. Goverde
Unexpected disruptions occur in the railways on a daily basis, which are typically handled manually by experienced traffic controllers with the support of predefined contingency plans. When several disruptions occur simultaneously, it is rather hard for traffic controllers to make rescheduling decisions, because (1) the predefined contingency plans corresponding to these disruptions may conflict with each other and (2) no predefined contingency plan considering the combined effects of multiple disruptions is available. This paper proposes a Mixed Integer Linear Programming (MILP) model to reschedule the timetable in case of multiple disruptions that occur at different geographic locations but have overlapping periods and are pairwise connected by at least one train line. The dispatching measures of retiming, reordering, cancelling, adding stops and flexible short-turning are formulated in the MILP model that also considers the rolling stock circulations at terminal stations and platform capacity. We develop two approaches for rescheduling the timetable in a dynamic environment: the sequential approach and the combined approach. In the sequential approach, a single-disruption rescheduling model is applied to handle each new disruption with the last solution as reference. In the combined approach, the multiple-disruption rescheduling model is applied every time an extra disruption occurs by considering all ongoing disruptions. A rolling-horizon solution method to the multiple-disruption model has been developed to handle long multiple connected disruptions in a more efficient way. The sequential and combined approaches have been tested on real-life instances on a subnetwork of the Dutch railways with 38 stations and 10 train lines operating half-hourly in each direction. In a few cases, the sequential approach did not find feasible solutions, while the combined approach obtained the solutions for all considered cases. Besides, the combined approach was able to find solutions with less cancelled train services and/or train delays than the sequential approach. For long disruptions, the proposed rolling-horizon method was able to generate high-quality rescheduling solutions in an acceptable time. ...
Journal article (2020) - Y. Zhu, R.M.P. Goverde
During railway disruptions, most passengers may not be able to find preferred alternative train services due to the current way of handling disruptions that does not take passenger responses into account. To offer better alternatives to passengers, this paper proposes a novel passenger-oriented timetable rescheduling model, which integrates timetable rescheduling and passenger reassignment into a Mixed Integer Linear Programming model with the objective of minimizing generalized travel times: in-vehicle times, waiting times at origin/transfer stations and the number of transfers. The model applies the dispatching measures of re-timing, re-ordering, cancelling, flexible stopping and flexible short-turning trains, handles rolling stock circulations at both short-turning and terminal stations of trains, and takes station capacity into account. To solve the model efficiently, an Adapted Fix-and-Optimize (AFaO) algorithm is developed. Numerical experiments were carried out to a part of the Dutch railways. The results show that the proposed passenger-oriented timetable rescheduling model is able to shorten generalized travel times significantly compared to an operator-oriented timetable rescheduling model that does not consider passenger responses. By allowing only 10 min more train delay than an optimal operator-oriented rescheduling solution, the passenger-oriented model is able to shorten the generalized travel times over all passengers by thousands of minutes in all considered disruption scenarios. With a passenger-oriented rescheduled timetable, more passengers continue their train travels after a disruption started, compared to a rescheduled timetable from the operator-oriented model. The AFaO algorithm obtains high-quality solutions to the passenger-oriented model in up to 300 s. ...
Conference paper (2020) - Yongqiu Zhu, Hongrui Wang, Rob Goverde
Real-time railway traffic management is important for the daily operations of railway systems. It predicts and resolves operational conflicts caused by events like excessive passenger boardings/alightings. Traditional optimization methods for this problem are restricted by the size of the problem instances. Therefore, this paper proposes a reinforcement learning-based timetable rescheduling method. Our method learns how to reschedule a timetable off-line and then can be applied online to make an optimal dispatching decision immediately by sensing the current state of the railway environment. Experiments show that the rescheduling solution obtained by the proposed reinforcement learning method is affected by the state representation of the railway environment. The proposed method was tested to a part of the Dutch railways considering scenarios with single initial train delays and multiple initial train delays. In both cases, our method found high-quality rescheduling solutions within limited training episodes. ...
Journal article (2020) - Yongqiu Zhu, Rob Goverde
Unexpected disruptions occur frequently in the railways, during which many train services cannot run as scheduled. This paper deals with timetable rescheduling during such disruptions, particularly in the case where all tracks between two stations are blocked for hours. In practice, a disruption may become shorter or longer than predicted. To take the uncertainty of the disruption duration into account, this paper formulates the timetable rescheduling as a rolling horizon two-stage stochastic programming problem in deterministic equivalent form. The random disruption duration is assumed to have a finite number of possible realizations, called scenarios, with given probabilities. Every time a prediction about the range of the disruption end time is updated, new scenarios are defined, and a two-stage stochastic model computes the optimal rescheduling solution to all these scenarios. The stochastic method was tested on a part of the Dutch railways, and compared to a deterministic rolling-horizon method. The results showed that compared to the deterministic method, the stochastic method is more likely to generate better rescheduling solutions for uncertain disruptions by less train cancellations and/or delays, while the solution robustness can be affected by the predicted range regarding the disruption end time. ...
Doctoral thesis (2019) - Yongqiu Zhu, Rob Goverde
Railway systems are vulnerable to unexpected disruptions, which usually result in track blockages for a few hours. In practice, disruptions are handled manually and the resulting impact to passengers is rarely considered. To enable disruption management more efficiently, operator-friendly and passenger-friendly, this thesis develops mathematical models and solution methods for dynamic passenger assignment, timetable rescheduling, and the integrated passenger assignment with timetable rescheduling during disruptions. ...
Journal article (2019) - Yongqiu Zhu, Rob Goverde
Railway operations are vulnerable to unexpected disruptions that should be handled in an efficient and passenger-friendly way. To this end, we propose a timetable rescheduling model where flexible stopping (i.e. skipping stops and adding stops) and flexible short-turning (i.e. full choice of short-turn stations) are innovatively integrated with three other dispatching measures: retiming, reordering, and cancelling. The Mixed Integer Linear Programming model also ensures that each train serving a station is ensured with a platform track. To consider the rescheduling impact on passengers, the weight of each decision is estimated individually according to the time-dependent passenger demand. The objective is minimizing passenger delays. A case study is carried out for hundreds of disruption scenarios on a subnetwork of the Dutch railways. It is found that (1) applying a mix of flexible stopping and flexible short-turning results in less passenger delays; (2) shortening the recovery duration mitigates the post-disruption consequence by less delay propagation but is at the expense of more cancelled train services during the disruption; and (3) the optimal rescheduling solution is sensitive to the disruption duration, but some steady behaviour is observed when the disruption duration increases by the timetable cycle time. ...
Conference paper (2019) - Yongqiu Zhu, Rob Goverde
Unexpected disruptions occur frequently in railway systems, during which many train services cannot run as scheduled. This paper deals with timetable rescheduling during such disruptions, particularly in the case where all tracks between two stations are blocked for a few hours. In practice, the disruption length is uncertain, and a disruption may become shorter or longer than predicted. Thus, it is necessary to take the uncertainty of the disruption duration into account. This paper formulates the robust timetable rescheduling as a rolling horizon two-stage stochastic programming problem in deterministic equivalent form. The random disruption duration is assumed to have a finite number of possible realizations, called scenarios, with given probabilities. Every time a prediction about the range of the disruption end time is updated, new scenarios are defined, and the model computes the optimal rescheduling solution for an extended control horizon, which is robust to all these scenarios. Based on the model, uncertain disruptions can be handled with robust solutions in a dynamic environment. The stochastic method was tested on a part of the Dutch railways, and compared to a deterministic rolling-horizon method. The results showed that compared to the deterministic method, the stochastic method is more likely to generate better rescheduling solutions for uncertain disruptions by less train cancellations and/or delays, while the solution robustness can be affected by the predicted range regarding the disruption end time. ...
Journal article (2019) - Yongqiu Zhu, Rob Goverde
Passenger assignment models for major disruptions that require trains to be
cancelled/short-turned in railway systems are rarely considered in literature, although these models could make a significant contribution to passenger-oriented disruption timetable design/rescheduling. This paper proposes a dynamic passenger assignment model, where the passengers who start travelling before, during and after the disruption are all considered. The model ensures that on-board passengers are given priority over waiting passengers, and waiting passengers are boarding under the first-come-first-serve rule. Moreover, the model allows information interventions by publishing information about service variations and train congestion at different locations with the aim of distributing passengers wisely to achieve less travel time increase due to the disruption. Discrete event simulation is adopted to implement the model, where
loading/unloading procedures are realized and passengers re-plan their paths based on the information they receive. The model tracks individual travels, which helps to evaluate a disruption timetable in a passenger-oriented way. ...
Conference paper (2019) - Fengbo Liu, Yongqiu Zhu, Nikola Bešinović, Rob Goverde, Ruihua Xu
With high frequency and unavoidable disruptions, metro systems are nowadays undertaking great emphasis on disruption management. This paper proposes a mixed integer programming model for train rescheduling in high-frequency metro systems during partial blockages. Several train rescheduling strategies are formulated into the model that considers station capacity and rolling stock circulation. The model is applied to a busy line of the Shanghai metro network. The computation time meets the real-time application requirement. The case study presents different influences of various disruption scenarios and emergency train constraints on the optimal solution. ...
Conference paper (2018) - Yongqiu Zhu, Rob Goverde, Egidio Quaglietta
Unexpected disruptions occur in the railways on a daily basis, which are typically handled manually by experienced trac controllers with the support of predened contingency plans. When several disruptions occur simultaneously, it is rather hard for trac controllers to make trac management decisions, because 1) the predened contingency plans corresponding to these disruptions may con ictwith each other and 2) no predened contingency plan considering the combined eects of all disruptions is available. This paper proposes a Mixed Integer Linear Programming (MILP) model that reschedules the timetable automatically in case of multiple simultaneous disruptions occurring at dierent geographic locations. This multiple-disruptions rescheduling model considers the interactions between service adjustments made for dierent disruptions. The combined multiple disruptions rescheduling model is applied every time an extra disruption occurs by considering all ongoing disruptions. Also, a sequential single-disruption rescheduling model is considered to handle each new disruption with the last solution as reference. A case study is performed by assuming two simultaneous disruptions occurring in part of the Dutch railways with 38 stations and 10 train lines operating half-hourly in each direction. By setting 3 minutes as the computation time limit in the considered disruption scenario, the combined approach resulted in less cancelled train services and train delays compared to the sequential approach. ...
Conference paper (2017) - Yongqiu Zhu, Rob Goverde
The issue of disruptions in railway systems attracts a growing attention due to its severity to the society. Mitigation strategies are proposed from different perspectives to reduce the system vulnerability resulting by disruptions. However, due to financial limitations, it is not realistic to realize all strategies in the real world. Thus, it is necessary to establish a model based on which the system vulnerability can be measured and the mitigation strategy can be evaluated. In this paper, such a model is constructed by applying a Monte Carlo simulation based on the historical disruption data of railways in the Netherlands. Based on this model, we first construct a baseline scenario that is capable of reflecting the current disruption practice and calculate the top ten vulnerable tracks with the indicator of yearly disruption duration from this scenario. Next, we propose mitigation strategies towards each of the top ten vulnerable tracks of different magnitudes, and construct the mitigation scenarios according to the strategies. Finally, comparison between the baseline scenario and each of the mitigation scenarios is performed with the purpose of strategy evaluation. ...
Conference paper (2017) - Yongqiu Zhu, Rob Goverde
Passenger-oriented rescheduling problems receive increasing attention. However, the passenger assignment models used for evaluating the rescheduling solutions are usually simplified by many assumptions. To estimate passenger inconvenience more accurately, this paper establishes a dynamic passenger assignment model during disruptions, in which the time-dependent demand, disruption-induced service variations and vehicle capacities are all taken into account. Event-based simulation is adopted to implement the model of the dynamic loading and unloading procedures of passengers. Based on the model, individual travels can be tracked, thus making the estimation of individual passenger delay possible. By aggregating individual inconvenience, the performance of a given rescheduling solution/contingency plan can be evaluated. Furthermore, recommendations such as adding train units can also be proposed, as illustrated in the case study. ...