document

An adaptive agent-based approach for instant delivery order dispatching: Incorporating task buffering and dynamic batching strategies

Lu, Miaojia (author), Yan, Xinyu (author), Sharif Azadeh, S. (author), Wang, P. (author)

The volume of instant delivery has witnessed a significant growth in recent years. Given the involvement of numerous heterogeneous stakeholders, instant delivery operations are inherently characterized by dynamics and uncertainties. This study introduces two order dispatching strategies, namely task buffering and dynamic batching, as...

journal article 2024

document

Analysis of the impact of traffic density on training of reinforcement learning based conflict resolution methods for drones

Groot, D.J. (author), Ellerbroek, Joost (author), Hoekstra, J.M. (author)

Conventional Air Traffic Control is still predominantly being done by human Air Traffic Controllers, however, as the traffic density increases, the workload of the controllers increases as well. Especially for the area of unmanned aviation, driven by the rise in drones, having human controllers might become unfeasible. One of the methods that...

journal article 2024

document

Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach

Tseremoglou, I. (author), Santos, Bruno F. (author)

In the Condition-Based Maintenance (CBM) context, the definition of optimal maintenance plans for an aircraft fleet depends on an efficient integration of : (i) the probabilistic predictions of the health condition of the components and (ii) the stochastic arrival of the corrective maintenance tasks, together with consideration of the...

journal article 2024

document

Generalized Model and Deep Reinforcement Learning-Based Evolutionary Method for Multitype Satellite Observation Scheduling

Song, Yanjie (author), Ou, Junwei (author), Pedrycz, Witold (author), Suganthan, Ponnuthurai Nagaratnam (author), Wang, X. (author), Xing, Lining (author), Zhang, Yue (author)

Multitype satellite observation, including optical observation satellites, synthetic aperture radar (SAR) satellites, and electromagnetic satellites, has become an important direction in integrated satellite applications due to its ability to cope with various complex situations. In the multitype satellite observation scheduling problem ...

journal article 2024

document

Cooperative lane-changing in mixed traffic: a deep reinforcement learning approach

Yao, X. (author), Du, Zhaocheng (author), Sun, Zhanbo (author), Calvert, S.C. (author), ji, Ang (author)

Deep Reinforcement Learning (DRL) has made remarkable progress in autonomous vehicle decision-making and execution control to improve traffic performance. This paper introduces a DRL-based mechanism for cooperative lane changing in mixed traffic (CLCMT) for connected and automated vehicles (CAVs). The uncertainty of human-driven vehicles (HVs...

journal article 2024

document

Synergetic-informed deep reinforcement learning for sustainable management of transportation networks with large action spaces

Lai, Li (author), Dong, You (author), Andriotis, C. (author), Wang, Aijun (author), Lei, Xiaoming (author)

Effective transportation network management systems should consider safety and sustainability objectives. Existing research on large-scale transportation network management often employs the assumption that bridges can be considered individually under these objectives. However, this simplification misses accurate system-level representations,...

journal article 2024

document

Deep Reinforcement Learning for Orchestrating Cost-Aware Reconfigurations of vRANs

Murti, Fahri Wisnu (author), Ali, Samad (author), Iosifidis, G. (author), Latva-aho, Matti (author)

Virtualized Radio Access Networks (vRANs) are fully configurable and can be implemented at a low cost over commodity platforms to enable network management flexibility. In this paper, a novel vRAN reconfiguration problem is formulated to jointly reconfigure the functional splits of the base stations (BSs), locations of the virtualized central...

journal article 2024

document

Adaptive Cruise Control Utilizing Noisy Multi-Leader Measurements: A Learning-Based Approach

Ni, Y. (author), Knoop, V.L. (author), Kooij, J.F.P. (author), van Arem, B. (author)

A substantial number of vehicles nowadays are equipped with adaptive cruise control (ACC), which adjusts the vehicle speed automatically. However, experiments have found that commercial ACC systems which only detect the direct leader amplify the propagating disturbances in the platoon. This can cause severe traffic congestion when the number...

journal article 2024

document

Learning-Based Multi-UAV Flocking Control With Limited Visual Field and Instinctive Repulsion

Bai, Chengchao (author), Yan, Peng (author), Piao, Haiyin (author), Pan, W. (author), Guo, Jifeng (author)

This article explores deep reinforcement learning (DRL) for the flocking control of unmanned aerial vehicle (UAV) swarms. The flocking control policy is trained using a centralized-learning-decentralized-execution (CTDE) paradigm, where a centralized critic network augmented with additional information about the entire UAV swarm is utilized...

journal article 2024

document

A Comparative Study of Optimization Models for Condition-Based Maintenance Scheduling of an Aircraft Fleet

Tseremoglou, I. (author), van Kessel, Paul J. (author), Santos, Bruno F. (author)

Condition-based maintenance (CBM) scheduling of an aircraft fleet in a disruptive environment while considering health prognostics for a set of systems is a very complex combinatorial problem, which is becoming more challenging in light of the uncertainty included in health prognostics. This type of problem falls under the broad category of...

journal article 2023

document

Deep reinforcement learning for cerebral anterior vessel tree extraction from 3D CTA images

Su, Jiahang (author), Li, Shuai (author), Wolff, Lennard (author), van Zwam, Wim (author), Niessen, W.J. (author), van der Lugt, Aad (author), van Walsum, T. (author)

Extracting the cerebral anterior vessel tree of patients with an intracranial large vessel occlusion (LVO) is relevant to investigate potential biomarkers that can contribute to treatment decision making. The purpose of our work is to develop a method that can achieve this from routinely acquired computed tomography angiography (CTA) and...

journal article 2023

document

A Secure Robot Learning Framework for Cyber Attack Scheduling and Countermeasure

Wu, Chengwei (author), Yao, Weiran (author), Luo, Wensheng (author), Pan, W. (author), Sun, Guanghui (author), Xie, Hui (author), Wu, Ligang (author)

The problem of learning-based control for robots has been extensively studied, whereas the security issue under malicious adversaries has not been paid much attention to. Malicious adversaries can invade intelligent devices and communication networks used in robots, causing incidents, achieving illegal objectives, and even injuring people....

journal article 2023

document

Comfort-Oriented Motion Planning for Automated Vehicles Using Deep Reinforcement Learning

Rajesh, Nishant (author), Zheng, Y. (author), Shyrokau, B. (author)

Automated vehicles promise numerous advantages to their users. The proposed benefits could however be overshadowed by a rise in the susceptibility of passengers to motion sickness due to their engagement in non-driving tasks. Increasing attention is paid to designing vehicle motion to mitigate motion sickness. In this work, the deep...

journal article 2023

document

AutoPOI: automated points of interest selection for side-channel analysis

Remmerswaal, Mick G.D. (author), Wu, L. (author), Tiran, Sébastien (author), Mentens, Nele (author)

Template attacks (TAs) are one of the most powerful side-channel analysis (SCA) attacks. The success of such attacks relies on the effectiveness of the profiling model in modeling the leakage information. A crucial step for TA is to select relevant features from the measured traces, often called points of interest (POIs), to extract the...

journal article 2023

document

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Morato, P. G. (author), Andriotis, C. (author), Papakonstantinou, K. G. (author), Rigo, P. (author)

In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I...

journal article 2023

document

Synchromodal freight transport re-planning under service time uncertainty: An online model-assisted reinforcement learning

Zhang, Y. (author), Negenborn, R.R. (author), Atasoy, B. (author)

The objective of this study is to address the issue of service time uncertainty in synchromodal freight transport, which can cause delays, inefficiencies, and reduced satisfaction for shippers. The proposed solution is an online deep Reinforcement Learning (RL) approach that takes into account the service time uncertainty, assisted by an...

journal article 2023

document

Learning to Solve Multiple-TSP With Time Window and Rejections via Deep Reinforcement Learning

Zhang, Rongkai (author), Zhang, Cong (author), Cao, Zhiguang (author), Song, Wen (author), Tan, Puay Siew (author), Zhang, Jie (author), Wen, Bihan (author), Dauwels, J.H.G. (author)

We propose a manager-worker framework (the implementation of our model is publically available at: https://github.com/zcaicaros/manager-worker-mtsptwr) based on deep reinforcement learning to tackle a hard yet nontrivial variant of Travelling Salesman Problem (TSP), i.e. multiple-vehicle TSP with time window and rejections (mTSPTWR), where...

journal article 2023

document

The first AI4TSP competition: Learning to solve stochastic routing problems

Zhang, Yingqian (author), Bliek, Laurens (author), da Costa, Paulo (author), Refaei Afshar, Reza (author), Reijnen, Robbert (author), Catshoek, T. (author), Vos, D.A. (author), Verwer, S.E. (author), Schmitt-Ulms, Fynn (author)

This paper reports on the first international competition on AI for the traveling salesman problem (TSP) at the International Joint Conference on Artificial Intelligence 2021 (IJCAI-21). The TSP is one of the classical combinatorial optimization problems, with many variants inspired by real-world applications. This first competition asked the...

journal article 2023

document

Deep reinforcement learning control approach to mitigating actuator attacks

Wu, C. (author), Pan, W. (author), Staa, Rick (author), Liu, Jianxing (author), Sun, Guanghui (author), Wu, Ligang (author)

This paper investigates the deep reinforcement learning based secure control problem for cyber–physical systems (CPS) under false data injection attacks. We describe the CPS under attacks as a Markov decision process (MDP), based on which the secure controller design for CPS under attacks is formulated as an action policy learning using data....

journal article 2023

document

Deep reinforcement learning for predictive aircraft maintenance using probabilistic Remaining-Useful-Life prognostics

Lee, J. (author), Mitici, M.A. (author)

The increasing availability of sensor monitoring data has stimulated the development of Remaining-Useful-Life (RUL) prognostics and maintenance planning models. However, existing studies focus either on RUL prognostics only, or propose maintenance planning based on simple assumptions about degradation trends. We propose a framework to integrate...

journal article 2022