Reinforcement Learning (RL) has been applied to teach quadcopters guidance tasks. Most applications rely on position information from an absolute reference system such as Global Positioning System (GPS). The dependence on “absolute position” information is a general limitation in the autonomous flight of Unmanned Aerial Vehicles (UAVs). Environments that have weak or no GPS signals are difficult to traverse for them. Instead of using absolute position, it is possible to sense the environment and the information contained within it in order to come up with a “relative” description of the UAV’s position. This paper presents the design of an RL agent with relative vision-based states and rewards for the teaching of a guidance task to a quadcopter. The agent is taught the task of turning towards a red marker and approaching it in simulation and in flight tests. A more complex task of travelling between a blue and a red marker is trained in simulation. This work shows that relative vision-based states and rewards can be used with RL to teach quadcopters simple guidance tasks. The performance of the trained agent is inconsistent in simulation and flight test due to the inherent partial observability in the relative description of the state. ... Read more

The use of Micro Aerial Vehicles (MAVs) in practical applications, to solve real-world problems, is growing in demand as the technology becomes more widely known and accessible. Proposed applications already span a wide berth of fields like military, search and rescue, ecology, artificial pollinators, and more. As compared to larger Unmanned Aerial Systems (UAS), MAVs are specifically desirable for applications which take advantage of their small size or light weight – whether that means being discreet, having insect-like maneuverability, operating in small spaces, or being more inherently safe with respect to injury towards people. In some cases, MAVs work under conditions where autonomy is needed. The small size of MAVs and the desire for autonomy combine to create a demanding set of challenges for the guidance, navigation, and control (GNC) of these systems. Limitations of on-board sensors, difficulties in modeling their complex and often time varying dynamics, and limited on-board computational resources, are just a few examples of the challenges facing MAV autonomy... ... Read more

Reinforcement learning is a promising framework for controlling complex vehicles with a high level of autonomy, since it does not need a dynamic model of the vehicle, and it is able to adapt to changing conditions. When learning from scratch, the performance of a reinforcement learning controller may initially be poor and -for real life applications- unsafe. In this paper the effects of using human demonstrations on the performance of reinforcement learning is investigated, using a combination of offline and online least squares policy iteration. It is found that using the human as an efficient explorer improves learning time and performance for a benchmark reinforcement learning problem. The benefit of the human demonstration is larger for problems where the human can make use of its understanding of the problem to efficiently explore the state space. Applied to a simplified quadrotor slung load drop off problem, the use of human demonstrations reduces the number of crashes during learning. As such, this paper contributes to safer and faster learning for model-free, adaptive control problems. ... Read more

This paper demonstrates a real life approach for quadrotor obstacle avoidance in indoor flight. A color-based vision approach for obstacle detection is used to good effect conjointly with an adaptive path planning algorithm. The presented task is to move about a set indoor space while avoiding randomly located obstacles and adapting a path to prevent future confrontation with the obstacles all together. The goal is to complete this task with a solution that is simple and efficient. The result is an adaptive path planning algorithm that evades obstacles when necessary and uses these interactions to find an obstaclefree path with simple logic. The whole task is implemented within Paparazzi, an open source autopilot software. Flight tests are performed in an indoor flight arena with simulated GPS from a camera tracking system. Through these flight tests, the approach proves to be reliable and efficient ... Read more