Mobile robots that operate in human environments require the ability to safely navigate among humans and other obstacles. Existing approaches use Deep Reinforcement Learning (DRL) to obtain safe robot behavior in such environments, but do not ensure collision avoidance or trajectory feasibility. This issue is solved by methods combining DRL with model predictive control (MPC). However, they do not account for static obstacle avoidance. Moreover, DRL-based approaches train their network in multiple environments with increasing difficulty to speed up training and improve generalization ability. By sequentially training the model on new (more complex) environments, new knowledge could interfere with old knowledge, a problem known as catastrophic forgetting.
Despite performing well on the most challenging scenarios, performance on more simple scenarios is diminished. This paper introduces a continual reinforcement learning (CRL) strategy that can sequentially learn multiple navigation tasks while retaining performance on all previously learned tasks. For robot navigation, we utilize a combination of MPC with DRL, to develop an algorithm that can avoid not only dynamic interacting agents but also static obstacles. Our approach is shown to be able to safely navigate to a goal position in multiple environments. In addition, by sequentially learning multiple tasks, we improve navigation performance in terms of successful events and travel times and distances compared to other DRL approaches. ... Read more

Social Navigation is the task of robot motion planning in an environment shared with humans.This is an especially hard sub-problem of motion planning because the planner has to dealwith a dynamic, continuous and unpredictable environment. We present a local motionplanner, namely Neural Network Model Predictive Control, for autonomous ground vehiclesin highly dynamic environments. A neural network is trained to plan local trajectories basedon human behavior data. It has therefor learned to mimic how a person would behave in sucha situation. The trajectory plan of the neural network is used as guidance and initializationof a model predictive controller. This MPC creates a kinematically feasible trajectory andassures collision avoidance with the static and dynamic obstacles in the environment withinits receding horizon. This combined planner and controller is tested in simulation and showedon a real autonomous robot ... Read more

Deep Reinforcement Learning (DRL) enables us to design controllers for complex tasks with a deep learning approach. It allows us to design controllers that are otherwise cumbersome to design with conventional control methodologies. Often, an objective for RL is binary in nature. However, exploring in environments with sparse rewards is a problem in RL, and finding positive reward becomes exponentially more difficult with increased environment complexity. For this project, our objective is to design an RL based controller for the landing of a quadcopter on inclined surfaces. Landing is defined as reaching these inclined surfaces with reasonable speed, such that no damage is done to either the quadcopter or the surface to land on upon impact. We aim to use a binary reward for this task. We use methods to aid exploration in sparse reward environments, namely Hindsight Experience Replay (HER), and non-optimized demonstrations. HER can resample goals from the demonstrator data and the policy rollouts. The resampling of goals is done by considering a portion of the visited states during policy rollouts as the intended goals. The demonstrations are non-optimized in the sense that the demonstrations do not follow the same objective as ours. We consider demonstrations valid if these demonstrations are obtained from arbitrary stable policies. Our results show that the RL system does generalize to other goals when using HER and demonstrations. The demonstrations are not imitated as were to happen in pure imitation learning. HER, on the other hand, enabled us to receive reward in our complex environment, while also allowing us to experience multiple goals in one policy rollout. We found that lack of HER and demonstrations were not able to overcome the problems of exploration in sparse reward environments. We found that landing a quadcopter on inclined surfaces using an RL controller is feasible. Our trajectories clearly showed a swinging motion which in theory should be a valid control strategy for this problem. This swinging motion results in dead spots with the quadcopter being in a state with a minimal translational and rotational velocities under a relatively large angle. Further research is needed to increase the accuracy and robustness of our RL based controller. ... Read more

Motion planning for Autonomous Ground Vehicles (AGVs) in dynamic environments is an extensively studied and complex problem. State of the art methods provide approximate solutions that make conservative assumptions to provide safety and feasibility. We aim to outperform current methods by following a trajectory optimization-based approach, providing a Local Model Predictive Contouring Control framework. Our method allows AGVs to execute reactive motion while tracking a locally parametrized reference path, anticipating on the predicted evolution of the environment. Given the static environment configuration in an occupancy grid map and dynamic obstacles represented by ellipses, we formulate explicit collision avoidance constraints. Well-informed planning decisions are made through a cost function with trade-offs between competing performance variables such as tracking accuracy, maintaining the reference velocity, and clearance from obstacles.

An efficient implementation of the method is presented that satisfies the real-time constraint of online navigation tasks. Furthermore, we present an implementation of a complete navigation system to emphasize our ability to deal with real sensor data and onboard processing. We show that the general definition of the framework applies to both unicycle and bicycle kinematic models, commonly used to represent mobile robots and autonomous cars, respectively. Simulation results for a car and experimental results with a mobile robot show that our method is a feasible and scalable approach. Proposed improvements of the method include 1) considering obstacle velocities and positioning with respect to the AGV in the penalty term that creates clearance, 2) incorporating prediction uncertainty of obstacles, and 3) improving our method that deals with infeasible solutions of the optimal control problem. ... Read more