Y. Zhu
Please Note
20 records found
1
Learning to reschedule platforms
A graph neural network based deep reinforcement learning method for the train platforming and rescheduling problem⁎
The train platforming schedule is the crucial plan for guiding trains to travel through a railway station without spatial and temporal conflicts. When trains are delayed in arriving at the station due to disturbances or disruptions, it raises the Train Platforming and Rescheduling Problem (TPRP), one of the hot topics in railway traffic management. It focuses on allocating platforms and time slots for trains to reduce delays and ensure operational efficiency in a station. This paper introduces a novel graph neural network based deep reinforcement learning method to address this problem, named Learning to Reschedule Platforms (L2RP). We formulate the solving process of TPRP as a customized Markov decision process. Meanwhile, we integrate a microscopic discrete-event train operation simulation model to serve as the agent exploration environment, which provides states, executes actions, and completes transitions. Then, we design a hybrid graph neural network based policy network to derive high-quality actions under each graph encoded state.The policy network is trained with the reward function designed to minimize total train knock-on delays and platform changes. The experiments on real-world instances show that the proposed L2RP method can produce high-quality solutions for instances of various scenarios within stably short solving times.
Reinforcement learning for train timetable rescheduling under perturbation
A general value-based approach
This paper proposes a value-based deep reinforcement learning approach that is capable of handling train timetable rescheduling under both disturbed and disrupted situations. A railway environment is constructed to simulate the problem as a Markov decision process, where the optimization objective is integrated into the reward module and various constraints are incorporated into the conflict detection and avoidance module. To address the challenges of sparse rewards and large action space with limited legal actions, a value-based algorithm framework is proposed to efficiently select and effectively evaluate actions. Through the designed simulation and training procedures, the proposed approach is tested on several disturbance and disruption cases based on a real-world instance (i.e. a Chinese high-speed railway corridor). Experimental results show that the proposed method can obtain high-quality solutions within a reasonable computing time, and also outperforms handcrafted rules in terms of the optimality of solutions. Furthermore, the proposed method exhibits promising generalization capabilities in homogeneous perturbation scenarios (disturbance scenarios and disruption scenarios that share either the same affected location and start time or the same affected location and disrupted duration).
Online multi-modal evacuation during passenger flow outburst in urban transit system
A heterogeneous multi-agent reinforcement learning framework
With growing demand straining urban transit systems’ resilience in managing outburst passenger flows, existing approaches focused on offline and single-modal evacuations remain limited. This study proposes an online multi-modal evacuation framework that coordinates on-duty taxis, buses, and metros while minimizing impact on their regular services. We develop a data-driven agent-based environment to update multi-modal transit data and stranded passenger information in real time. Two coordination strategies are introduced: (1) an independent strategy using a decentralized training and distributed execution algorithm, and (2) a collaborative strategy using a hybrid centralized training and distributed execution algorithm. To dynamically assess evacuation effectiveness, we design a resilience framework with three metrics: robustness, rapidity, and resourcefulness. These metrics are transformed into demand-responsive feedback at each time step, enabling agents to proactively generate resilient evacuation plans. In a real-world case study triggered by a railway disruption, our approach outperforms genetic algorithms and multi-agent deep deterministic policy gradient algorithms in computation time and solution quality under offline conditions. Simulated new environments further validate its online applicability, demonstrating its potential for real-world deployment.
Learning to Platforming
A Deep Reinforcement Learning Method for the Train Platforming and Rescheduling Problem
Metro networks face operational challenges due to increasing ridership and system growth, particularly in managing delay propagation. Epidemiology models have recently been an interesting method in transportation research for studying delays. This study, therefore, aims to investigate if the Susceptible-infectious-susceptible (SIS) model is suitable to help model delay propagation in a metro network through its ability to reproduce the vulnerability of metro stations for specific instances. Using data from the Washington Metro Network, two groups of delay propagation instances were selected and used for model training and testing using a differential evolution algorithm. The results indicate that the vulnerability values as calculated from the reallife data do not follow the expected trend. Still, our model can capture this variation with good vulnerability estimation accuracy for both groups. Also, the predicted vulnerability values for the first group are more accurate than for the second group. However, limitations such as underestimation and overestimation of station vulnerabilities, and sensitivity to training data were observed. These challenges stemmed from the dynamics between specific parameters and the lack of additional factors.
Handling uncertainty in train timetable rescheduling
A review of the literature and future research directions
External and internal factors can cause disturbances or disruptions in daily train operations, leading to deviations from official timetables and passenger delays. As a result, efficient train timetable rescheduling (TTR) methods are necessary to restore disrupted train services. Although TTR has been a popular research topic in recent years, the uncertain characteristics of railways have not been sufficiently addressed. This review first identifies the primary uncertainties of TTR and examines their impacts on both TTR and passenger routing during disturbances or disruptions. It finds that only a few uncertainties have been investigated, and the existing solution methods do not adequately meet practical requirements, such as considering the dynamic nature of disturbances or disruptions, which is crucial for real-world applications. Therefore, the review highlights problems associated with TTR uncertainties that need urgent attention and suggests promising methodologies that could effectively address these issues as future research directions. This review aims to help practitioners develop improved automatic train-dispatching systems with better train-rescheduling performance under disturbances or disruptions compared to current systems.
Unexpected disruptions occur in the railways on a daily basis, which are typically handled manually by experienced traffic controllers with the support of predefined contingency plans. When several disruptions occur simultaneously, it is rather hard for traffic controllers to make rescheduling decisions, because (1) the predefined contingency plans corresponding to these disruptions may conflict with each other and (2) no predefined contingency plan considering the combined effects of multiple disruptions is available. This paper proposes a Mixed Integer Linear Programming (MILP) model to reschedule the timetable in case of multiple disruptions that occur at different geographic locations but have overlapping periods and are pairwise connected by at least one train line. The dispatching measures of retiming, reordering, cancelling, adding stops and flexible short-turning are formulated in the MILP model that also considers the rolling stock circulations at terminal stations and platform capacity. We develop two approaches for rescheduling the timetable in a dynamic environment: the sequential approach and the combined approach. In the sequential approach, a single-disruption rescheduling model is applied to handle each new disruption with the last solution as reference. In the combined approach, the multiple-disruption rescheduling model is applied every time an extra disruption occurs by considering all ongoing disruptions. A rolling-horizon solution method to the multiple-disruption model has been developed to handle long multiple connected disruptions in a more efficient way. The sequential and combined approaches have been tested on real-life instances on a subnetwork of the Dutch railways with 38 stations and 10 train lines operating half-hourly in each direction. In a few cases, the sequential approach did not find feasible solutions, while the combined approach obtained the solutions for all considered cases. Besides, the combined approach was able to find solutions with less cancelled train services and/or train delays than the sequential approach. For long disruptions, the proposed rolling-horizon method was able to generate high-quality rescheduling solutions in an acceptable time.
cancelled/short-turned in railway systems are rarely considered in literature, although these models could make a significant contribution to passenger-oriented disruption timetable design/rescheduling. This paper proposes a dynamic passenger assignment model, where the passengers who start travelling before, during and after the disruption are all considered. The model ensures that on-board passengers are given priority over waiting passengers, and waiting passengers are boarding under the first-come-first-serve rule. Moreover, the model allows information interventions by publishing information about service variations and train congestion at different locations with the aim of distributing passengers wisely to achieve less travel time increase due to the disruption. Discrete event simulation is adopted to implement the model, where
loading/unloading procedures are realized and passengers re-plan their paths based on the information they receive. The model tracks individual travels, which helps to evaluate a disruption timetable in a passenger-oriented way. ...
cancelled/short-turned in railway systems are rarely considered in literature, although these models could make a significant contribution to passenger-oriented disruption timetable design/rescheduling. This paper proposes a dynamic passenger assignment model, where the passengers who start travelling before, during and after the disruption are all considered. The model ensures that on-board passengers are given priority over waiting passengers, and waiting passengers are boarding under the first-come-first-serve rule. Moreover, the model allows information interventions by publishing information about service variations and train congestion at different locations with the aim of distributing passengers wisely to achieve less travel time increase due to the disruption. Discrete event simulation is adopted to implement the model, where
loading/unloading procedures are realized and passengers re-plan their paths based on the information they receive. The model tracks individual travels, which helps to evaluate a disruption timetable in a passenger-oriented way.