Online robot guidance and navigation in non-stationary environment with hybrid Hierarchical Reinforcement Learning

Online robot guidance and navigation in non-stationary environment with hybrid Hierarchical Reinforcement Learning

Zhou, Y. (TU Delft Control & Simulation; Universiti Sains Malaysia)
Ho, H.W. (TU Delft Control & Simulation; Universiti Sains Malaysia)

2022

Hierarchical Reinforcement Learning (HRL) provides an option to solve complex guidance and navigation problems with high-dimensional spaces, multiple objectives, and a large number of states and actions. The current HRL methods often use the same or similar reinforcement learning methods within one application so that multiple objectives can be easily combined. Since there is not a single learning method that can benefit all targets, hybrid Hierarchical Reinforcement Learning (hHRL) was proposed to use various methods to optimize the learning with different types of information and objectives in one application. The previous hHRL method, however, requires manual task-specific designs, which involves engineers’ preferences and may impede its transfer learning ability. This paper, therefore, proposes a systematic online guidance and navigation method under the framework of hHRL, which generalizes training samples with a function approximator, decomposes the state space automatically, and thus does not require task-specific designs. The simulation results indicate that the proposed method is superior to the previous hHRL method, which requires manual decomposition, in terms of the convergence rate and the learnt policy. It is also shown that this method is generally applicable to non-stationary environments changing over episodes and over time without the loss of efficiency even with noisy state information.

Function approximation
Hybrid Hierarchical Reinforcement Learning
Non-stationary environment
Online guidance and navigation
State space decomposition

http://resolver.tudelft.nl/uuid:0797bee5-0a87-4784-84d0-335d6604b4c2

https://doi.org/10.1016/j.engappai.2022.105152

2023-07-01

0952-1976

Engineering Applications of Artificial Intelligence, 114

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2022 Y. Zhou, H.W. Ho