Artificial Potential Fields for Safe Reinforcement Learning

Abstract

Reinforcement Learning (RL) methods have become a topic of interest for performing guidance and navigation tasks, due to potential adaptability and autonomy improvements within dynamic systems. Nevertheless, a core component of RL is an agent exploring the environment it finds itself in, resulting in an intrinsic violation of the agent's safety. A subfield of Safe Reinforcement Learning (SRL) has emerged in an attempt to address the safety issues introduced by the agent's need to explore. Artificial Potential Field (APF) methods have also been studied extensively for the purposes of guidance and navigation, but seldom have the two methods, RL \& APF, been examined when combined. This research focuses on integrating APF information within a RL controller for aerospace flight control applications. Different methods of integration are compared, by implementing APF information at varying levels of the RL algorithm- at the value function level, in the policy, and by exploration modification. Experiments show a decrease in collisions in the learning stage with a slight reduction in performance, relative to flat Reinforcement Learning.