Deep reinforcement learning control approach to mitigating actuator attacks

Deep reinforcement learning control approach to mitigating actuator attacks

Wu, C. (Harbin Institute of Technology)
Pan, W. (TU Delft Robot Dynamics; The University of Manchester)
Staa, Rick (Student TU Delft)
Liu, Jianxing (Harbin Institute of Technology)
Sun, Guanghui (Harbin Institute of Technology)
Wu, Ligang (Harbin Institute of Technology)

2023

This paper investigates the deep reinforcement learning based secure control problem for cyber–physical systems (CPS) under false data injection attacks. We describe the CPS under attacks as a Markov decision process (MDP), based on which the secure controller design for CPS under attacks is formulated as an action policy learning using data. Rendering the soft actor–critic learning algorithm, a Lyapunov-based soft actor–critic learning algorithm is proposed to offline train a secure policy for CPS under attacks. Different from the existing results, not only the convergence of the learning algorithm but the stability of the system using the learned policy is proved, which is quite important for security and stability-critical applications. Finally, both a satellite attitude control system and a robot arm system are used to show the effectiveness of the proposed scheme, and comparisons between the proposed learning algorithm and the classical PD controller are also provided to demonstrate the advantages of the control algorithm designed in this paper.

Cyber–physical systems
Deep reinforcement learning
False data injection attacks
Lyapunov stability

http://resolver.tudelft.nl/uuid:fac3153f-e987-4ae2-88ed-8460b3d54735

https://doi.org/10.1016/j.automatica.2023.110999

2023-09-17

0005-1098

Automatica, 152

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2023 C. Wu, W. Pan, Rick Staa, Jianxing Liu, Guanghui Sun, Ligang Wu