Hierarchical Clustering-Based State Grouping Reinforcement Learning for Switching Decision of Autonomous Vehicles
Wenjie Ouyang (Hunan University)
Yiwen Jiao (Hunan University)
Yang Liu (TU Delft - Electronic Instrumentation)
Yang Li (Hunan University, TU Delft - Algorithmics)
Manjiang Hu (Hunan University)
Hongmao Qin (Hunan University, Institute of Hunan University)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Reinforcement learning (RL) has gained wide attention, but its implementation in autonomous vehicles is still limited by insufficient sample efficiency and heavy training costs. The training efficiency of RL agents is influenced by the dimension of the state space, which can be partitioned to reduce the complexity of sampling and computation. This study proposes a hierarchical clustering-based state grouping reinforcement learning (HCSG-RL) method for the switching decision of autonomous vehicles. First, we partition the base state space into groups and generate a hierarchical tree of state space groups. Then, we train multiple sub-agents for each node in the hierarchical tree. Finally, we add these trained-well sub-model into master policy. This method allows us to fully explore all state spaces and improve the training efficiency of individual agents, which handles the 'long-tail' issue and the curse of dimensionality issue. We conduct experiments in a simulation environment and results show that the proposed method has 16-72% reward improvement compared to the tree model in different road length.