document

Spiking Neural Networks Based Data Driven Control: An Illustration Using Cart-Pole Balancing Example

Liu, Yuxiang (author)

Machine learning can be effectively applied in control loops to robustly make optimal control decisions. There is increasing interest in using spiking neural networks (SNNs) as the apparatus for machine learning in control engineering, because SNNs can potentially offer high energy efficiency and new SNN-enabling neuromorphic hardwares are being...

master thesis 2023

document

Spiking Neural-Networks-Based Data-Driven Control

Liu, Y. (author), Pan, W. (author)

Machine learning can be effectively applied in control loops to make optimal control decisions robustly. There is increasing interest in using spiking neural networks (SNNs) as the apparatus for machine learning in control engineering because SNNs can potentially offer high energy efficiency, and new SNN-enabling neuromorphic hardware is being...

journal article 2023

document

Hierarchical Motion Planning and Tracking for Autonomous Vehicles Using Global Heuristic Based Potential Field and Reinforcement Learning Based Predictive Control

Du, Guodong (author), Zou, Yuan (author), Zhang, Xudong (author), Li, Z. (author), Liu, Qi (author)

The autonomous vehicle is widely applied in various ground operations, in which motion planning and tracking control are becoming the key technologies to achieve autonomous driving. In order to further improve the performance of motion planning and tracking control, an efficient hierarchical framework containing motion planning and tracking...

journal article 2023

document

Deep reinforcement learning control approach to mitigating actuator attacks

Wu, C. (author), Pan, W. (author), Staa, Rick (author), Liu, Jianxing (author), Sun, Guanghui (author), Wu, Ligang (author)

This paper investigates the deep reinforcement learning based secure control problem for cyber–physical systems (CPS) under false data injection attacks. We describe the CPS under attacks as a Markov decision process (MDP), based on which the secure controller design for CPS under attacks is formulated as an action policy learning using data....

journal article 2023

document

Benchmarking Robustness and Generalization in Multi-Agent Systems: A Case Study on Neural MMO

Chen, Yangkun (author), Yu, Chenghui (author), Zhu, Hengman (author), Liu, Shuai (author), Zhang, Yibing (author), Suarez, Joseph (author), Zhao, Liang (author), He, J. (author), Chen, Jiaxin (author)

We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-agent systems: participants train teams of agents to complete a multi-task objective against opponents not seen during training. We summarize the competition design...

journal article 2023

document

Eligibility traces and forgetting factor in recursive least-squares-based temporal difference

Baldi, S. (author), Zhang, Z. (author), Liu, Di (author)

We propose a new reinforcement learning method in the framework of Recursive Least Squares-Temporal Difference (RLS-TD). Instead of using the standard mechanism of eligibility traces (resulting in RLS-TD((Formula presented.))), we propose to use the forgetting factor commonly used in gradient-based or least-square estimation, and we show that...

journal article 2022

document

Generalized Single-Vehicle-Based Graph Reinforcement Learning for Decision-Making in Autonomous Driving

Yang, Fan (author), Li, Xueyuan (author), Liu, Qi (author), Li, Z. (author), Gao, Xin (author)

In the autonomous driving process, the decision-making system is mainly used to provide macro-control instructions based on the information captured by the sensing system. Learning-based algorithms have apparent advantages in information processing and understanding for an increasingly complex driving environment. To incorporate the...

journal article 2022

document

Coupled effects of corrosion damage and sustained loading on the flexural behavior of RC beams strengthened with CFRP anchorage system

Pan, Tanbo (author), Zheng, Yonglai (author), Zhou, Yujue (author), Liu, Yongcheng (author), Yu, Kunlong (author), Zhou, Y. (author)

This paper presents the results of an experimental study on the flexural behavior of carbon fibre reinforced polymer (CFRP) anchorage system strengthened reinforced concrete (RC) beams under the coupled effects of corrosion damage and sustained loading. The test beams were subjected to combined accelerated corrosion in 5% NaCl solution and...

journal article 2022

document

A new reinforcement learning-based variable speed limit control approach to improve traffic efficiency against freeway jam waves

Han, Yu (author), Hegyi, A. (author), Zhang, Le (author), He, Zhengbing (author), Chung, Edward (author), Liu, Pan (author)

Conventional reinforcement learning (RL) models of variable speed limit (VSL) control systems (and traffic control systems in general) cannot be trained in real traffic process because new control actions are usually explored randomly, which may result in high costs (delays) due to exploration and learning. For this reason, existing RL-based...

journal article 2022

document

Influence of different corrosion solutions on reinforced concrete by DIC and traditional strain test

Wang, XiaoX (author), Liu, JiaP (author), Zhong, PeiH (author)

When the corrosion expansion stress of the steel reinforcement exceeds the ultimate tensile strength of the concrete, the concrete will crack and eventually cause structural damage. Therefore, direct characterization methods of steel corrosion expansion stress can help predict the concrete cracking time, prevent corrosion expansion and repair...

conference paper 2021

document

Secure Control for Cyber-Physical Systems under Malicious Attacks

Wu, Chengwei (author), Yao, Weiran (author), Pan, W. (author), Sun, Guanghui (author), Liu, Jianxing (author), Wu, Ligang (author)

This article investigates the secure control problem for cyber-physical systems when the malicious data are injected into the cyber realm, which directly connects to the actuators. Based on moving target defense (MTD) and reinforcement learning, we propose a novel proactive and reactive defense control scheme. First, the system (A,B) is...

journal article 2021

document

Learning Tracking Control for Cyber-Physical Systems

Wu, C. (author), Pan, W. (author), Sun, Guanghui (author), Liu, Jianxing (author), Wu, Ligang (author)

This paper investigates the problem of optimal tracking control for cyber-physical systems (CPS) when the cyber realm is attacked by denial-of-service (DoS) attacks which can prevent the control signal transmitting to the actuator. Attention is focused on how to design the optimal tracking control scheme without using the system dynamics and...

journal article 2021

document

A Looseness Detection Method for Railway Catenary Fasteners based on Reinforcement Learning Refined Localization

Zhong, Junping (author), Liu, Zhigang (author), Wang, H. (author), Liu, Wenqiang (author), Yang, Cheng (author), Han, Zhiwei (author), Nunez, Alfredo (author)

Brace sleeve (BS) fasteners, i.e., nut and bolt, are small components but play essential roles in fixing BS and cantilever in railway catenary system. They are commonly inspected by onboard cameras using computer vision to ensure the safety of railway operation. However, most BS fasteners cannot be directly localized because they are too...

journal article 2021

document

An investigation on the fatigue performance of cracked steel plates reinforced with FRP and stop hole

Liu, Zhiping (author), Li, Z. (author), Huang, Chuanhai (author), Jiang, X. (author)

In this paper, we implement the combined method of drilling stop hole and FRP reinforcement to repair cracked steel plates subjected to cyclic tension. The crack initiation and growth are experimentally investigated. The stress distribution at the stop hole and the residual fatigue life are evaluated by the FEM. The effects on prolonging...

journal article 2021

document

Action-driven Reinforcement Learning for Improving Localization of Brace Sleeve in Railway Catenary

Zhong, J. (author), Liu, Zhigang (author), Wang, H. (author), Liu, W. (author), Yang, Cheng (author), Nunez, Alfredo (author)

Brace Sleeve (BS) plays an essential role in connecting and fixing cantilevers of railway catenary systems. It needs to be monitored to ensure the safety of railway operations. In the literature, image processing techniques that can localize BSs from inspection images are proposed. However, the boxes produced by existing methods can contain...

conference paper 2020

document

Numerical investigation on the surface crack growth in FRP-reinforced steel plates subjected to tension

Li, Z. (author), Jiang, X. (author), Hopman, J. (author), Zhu, Ling (author), Liu, Zhiping (author)

In this paper, we analyse the surface crack growth in the Fibre-Reinforced Polymer (FRP) reinforced steel plates subjected to tension by means of the finite element (FE) method. Following the experimental study, a three-dimensional FE model is developed to evaluate the Stress Intensity Factor (SIF) of the surface crack, and the crack growth...

journal article 2020

document

Experimental investigation on FRP-reinforced surface cracked steel plates subjected to cyclic tension

Li, Z. (author), Jiang, X. (author), Hopman, J.J. (author), Zhu, Ling (author), Liu, Z. (author), Tang, Weiguo (author)

Surface cracked steel plates reinforced with single-side Fiber-Reinforced Polymer (FRP) subjected to cyclic tension are experimental studied. The main purpose is to analyze the effect of FRP reinforcement on the crack growth. The failure modes and their effects are analyzed as well. Given the single-side reinforcement, reinforcing the cracked...

journal article 2020

document

Analysis on mechanical characteristics of welded joint with a new reinforced device in high-speed railway

Xiao, Hong (author), Yan, Dongwei (author), Liu, Guangpeng (author), Wang, Haoyu (author)

High-speed railways adopt continuous welded rail to maintain the smoothness and continuity of the rail surface. However, the welded joint became one of the weakest parts. In order to clear the characteristics and mechanical properties of the new reinforced device, a dynamic three-dimensional vehicle-reinforced device-track coupling model is...

journal article 2020

document

Experimental and numerical study on behaviour of square steel tube confined reinforced concrete stub columns after fire exposure

Liu, Faqi (author), Yang, Hua (author), Yan, R. (author), Wang, Wei (author)

The behaviour of square steel tube confined reinforced concrete columns after fire exposure was studied experimentally and numerically in this paper. Eighteen stub columns were first heated following the ISO 834 standard fire including both heating and cooling phases, and were subsequently loaded to failure after cooling to ambient temperature....

journal article 2019

document

Geocon bridge geopolymer concrete mixture for structural applications

Ye, G. (author), Lukovic, M. (author), Ghiassi, B. (author), Aldin, Zainab (author), Prinsse, Silke (author), Liu, Jonh (author), Nedeljković, Marija (author), Hordijk, D.A. (author), Lagendijk, P. (author), Bosman, A. (author), Blom, T.A. (author), van Leeuwen, M. (author), Huang, Zhekang (author), Celada, Ulric (author), Du, C. (author), van den Berg, J.A.M. (author), Thijssen, A. (author), Wijte, Simon (author)

The sustainability of infrastructure projects is becoming increasingly important issue in engineering practice. This means that in the future the construction materials will be selected on the basis of the contribution they can make to reach sustainability requirements. Geopolymers are materials based on by-products from industries. By using...

journal article 2019