A lot research has been conducted in the field of autonomous navigation of mobile robots with focus on Robot Vision and Robot Motion Planning. However, most of the classical navigation solutions require several steps of data pre-processing and hand tuning of parameters, with separate modules for vision, localization, planning and control. All these modules work independently and make their own parameter assumptions to optimize their own performance without taking into account the effect these assumptions have on the performance of rest of the modules. Hence, even though each module in the whole system tries to achieve an optimal performance for the task it has been assigned, the lack of interdependence exhibited by these modules for decision making means that the overall performance of the whole system is sub-optimal in most of the cases. An alternating approach for addressing these issues is to train certain parts of the vision module to incorporate partial tasks from the planning module. Deep Learning architectures have achieved great success in the field of pattern recognition and object detection and as a result are usually being deployed to design such a module that jointly learns to carry out perception and path planning. This master's thesis, making use of Deep Learning, proposes an End-to-End Learning architecture that learns to directly map raw sensor readings to control commands for a ground based mobile robot. The research makes use of the simulation of Jackal UGV from Clearpath Robotics and the proposed network is able to produce collision free trajectories for the robot to navigate in it's environment.

The successful integration of autonomous vehicles (AVs) in human environments is highly dependent on their ability to navigate safely and timely through dense traffic conditions. Such conditions involve a diverse range of human behaviors, ranging from cooperative (willing to yield) to non-cooperative human drivers (unwilling to yield) that need to be identified without any explicit inter-vehicle communication. In order to maneuver through such conditions, AVs must not only compute a collision-free trajectory but also account for the effects of its actions on the surrounding agents to negotiate the navigation maneuver safely. Existing motion planning techniques fail in these environments because they suffer from one or more of the following drawbacks: suffer from ”the curse of dimensionality” due to the high number of agents (e.g., optimization-based methods); do not account for the interaction effects among the agents; do not provide any collision avoidance or trajectory feasibility guarantees (e.g., learning-based methods). In this paper, we propose a novel navigation framework combining the strengths of learning-based with optimization-based algorithms. More specifically, we employ a Soft Actor-Critic agent to learn a continuous guidance policy that provides global guidance to an optimization-based planner generating feasible and collision- free trajectories. We evaluate our method in a highly inter- active simulation environment where we compare our method with two baseline approaches, a learning-based method and an optimization-based method, and present performance results demonstrating our method significantly reduces the number of collisions and increase the success rate with fewer number of deadlocks. We also show that that our method is able to generalise and applicable to other traffic scenarios (e.g., an unprotected left turn).

Mobile robots that operate in human environments require the ability to safely navigate among humans and other obstacles. Existing approaches use Deep Reinforcement Learning (DRL) to obtain safe robot behavior in such environments, but do not ensure collision avoidance or trajectory feasibility. This issue is solved by methods combining DRL with model predictive control (MPC). However, they do not account for static obstacle avoidance. Moreover, DRL-based approaches train their network in multiple environments with increasing difficulty to speed up training and improve generalization ability. By sequentially training the model on new (more complex) environments, new knowledge could interfere with old knowledge, a problem known as catastrophic forgetting. Despite performing well on the most challenging scenarios, performance on more simple scenarios is diminished. This paper introduces a continual reinforcement learning (CRL) strategy that can sequentially learn multiple navigation tasks while retaining performance on all previously learned tasks. For robot navigation, we utilize a combination of MPC with DRL, to develop an algorithm that can avoid not only dynamic interacting agents but also static obstacles. Our approach is shown to be able to safely navigate to a goal position in multiple environments. In addition, by sequentially learning multiple tasks, we improve navigation performance in terms of successful events and travel times and distances compared to other DRL approaches.

Motion planning for Autonomous Ground Vehicles (AGVs) in dynamic environments is an extensively studied and complex problem. State of the art methods provide approximate solutions that make conservative assumptions to provide safety and feasibility. We aim to outperform current methods by following a trajectory optimization-based approach, providing a Local Model Predictive Contouring Control framework. Our method allows AGVs to execute reactive motion while tracking a locally parametrized reference path, anticipating on the predicted evolution of the environment. Given the static environment configuration in an occupancy grid map and dynamic obstacles represented by ellipses, we formulate explicit collision avoidance constraints. Well-informed planning decisions are made through a cost function with trade-offs between competing performance variables such as tracking accuracy, maintaining the reference velocity, and clearance from obstacles. An efficient implementation of the method is presented that satisfies the real-time constraint of online navigation tasks. Furthermore, we present an implementation of a complete navigation system to emphasize our ability to deal with real sensor data and onboard processing. We show that the general definition of the framework applies to both unicycle and bicycle kinematic models, commonly used to represent mobile robots and autonomous cars, respectively. Simulation results for a car and experimental results with a mobile robot show that our method is a feasible and scalable approach. Proposed improvements of the method include 1) considering obstacle velocities and positioning with respect to the AGV in the penalty term that creates clearance, 2) incorporating prediction uncertainty of obstacles, and 3) improving our method that deals with infeasible solutions of the optimal control problem.

Learning human motion prediction models online is key for autonomous navigation in unknown dynamic scenarios. Previous works focus solely on improving prediction network architectures, whilst training them offline. This paper introduces a self-supervised continual learning framework that continuously improves data-driven pedestrian trajectory prediction models online across various environments. We propose to use online streams of pedestrian data, normally available from detection and tracking pipelines. Examples are autonomously extracted from this data stream and aggregated in temporally bounded episodes, where the data of each episode is discarded as soon as the model has been adapted to it. Our framework overcomes the problem of catastrophic forgetting across episodes by selectively slowing down learning of important neurons and by rehearsing a small set of examples of constant length. Our approach is shown to significantly improve prediction performance in new and unseen environments compared standard gradient descent approaches. Finally, we present qualitative experimental results in simulation and in real environments.

Social Navigation is the task of robot motion planning in an environment shared with humans.This is an especially hard sub-problem of motion planning because the planner has to dealwith a dynamic, continuous and unpredictable environment. We present a local motionplanner, namely Neural Network Model Predictive Control, for autonomous ground vehiclesin highly dynamic environments. A neural network is trained to plan local trajectories basedon human behavior data. It has therefor learned to mimic how a person would behave in sucha situation. The trajectory plan of the neural network is used as guidance and initializationof a model predictive controller. This MPC creates a kinematically feasible trajectory andassures collision avoidance with the static and dynamic obstacles in the environment withinits receding horizon. This combined planner and controller is tested in simulation and showedon a real autonomous robot

With the performance of current motion planning methods being highly dependent on the quality of the perception system, robust 3D multi-object detection and tracking are vital for autonomous driving applications. Despite all the advancements in 2D and 3D object detectors, robust tracking of pedestrians in dense scenarios is still a challenging subject for small Automated Guided Vehicles (AGVs). Most research in the field of object detection and tracking focuses on autonomous cars, neglecting the design challenges that come with small AGVs. This thesis presents a real-time multi-modal multi-pedestrian detection and tracking pipeline for small mobile robots. The framework integrates five RGB-D cameras and a LiDAR sensor to achieve real-time pedestrian detection and tracking. The system relies on state-of-the-art 2D and 3D object detectors, a sensor fusion and filtering scheme, and a 3D object tracker. Moreover, to improve detection and tracking performance, we have collected a pedestrian dataset tailored for small AGVs. We use this dataset to train the 3D pedestrian detector and evaluate the performance of the pedestrian detectors and tracker. Evaluation of the proposed framework demonstrated the ability to robustly detect and track multiple pedestrians up to a distance of 10 meters. We open-sourced our framework at: https://github.com/bbrito/amr_navigation.

Deep Reinforcement Learning (DRL) enables us to design controllers for complex tasks with a deep learning approach. It allows us to design controllers that are otherwise cumbersome to design with conventional control methodologies. Often, an objective for RL is binary in nature. However, exploring in environments with sparse rewards is a problem in RL, and finding positive reward becomes exponentially more difficult with increased environment complexity. For this project, our objective is to design an RL based controller for the landing of a quadcopter on inclined surfaces. Landing is defined as reaching these inclined surfaces with reasonable speed, such that no damage is done to either the quadcopter or the surface to land on upon impact. We aim to use a binary reward for this task. We use methods to aid exploration in sparse reward environments, namely Hindsight Experience Replay (HER), and non-optimized demonstrations. HER can resample goals from the demonstrator data and the policy rollouts. The resampling of goals is done by considering a portion of the visited states during policy rollouts as the intended goals. The demonstrations are non-optimized in the sense that the demonstrations do not follow the same objective as ours. We consider demonstrations valid if these demonstrations are obtained from arbitrary stable policies. Our results show that the RL system does generalize to other goals when using HER and demonstrations. The demonstrations are not imitated as were to happen in pure imitation learning. HER, on the other hand, enabled us to receive reward in our complex environment, while also allowing us to experience multiple goals in one policy rollout. We found that lack of HER and demonstrations were not able to overcome the problems of exploration in sparse reward environments. We found that landing a quadcopter on inclined surfaces using an RL controller is feasible. Our trajectories clearly showed a swinging motion which in theory should be a valid control strategy for this problem. This swinging motion results in dead spots with the quadcopter being in a state with a minimal translational and rotational velocities under a relatively large angle. Further research is needed to increase the accuracy and robustness of our RL based controller.