Decentralized multi-robot systems typically perform coordinated motion planning by constantly broadcasting their intentions to avoid collisions. However, the risk of collision between robots varies as they move and communication may not always be needed. This paper presents an efficient communication method that addresses the problem of “when” and “with whom” to communicate in multi-robot collision avoidance scenarios. In this approach, each robot learns to reason about other robots’ states and considers the risk of future collisions before asking for the trajectory plans of other robots. We introduce a new neural architecture for the learned communication policy which allows our method to be scalable. We evaluate and verify the proposed communication strategy in simulation with up to twelve quadrotors, and present results on the zero-shot generalization/robustness capabilities of the policy in different scenarios. We demonstrate that our policy (learned in a simulated environment) can be successfully transferred to real robots.

@en

Autonomous navigation of Micro Aerial Vehicles (MAVs) in dynamic and unknown environments is a complex and challenging task. Current works rely on assumptions to solve the problem. The MAV's pose is precisely known, the dynamic obstacles can be explicitly segmented from static ones, their number is known and fixed, or they can be modeled with given shapes. In this paper, we present a method for MAV navigation in dynamic uncertain environments without making any of these assumptions. The method employs a particle-based dynamic map to represent the local environment and predicts it to the near future. Collision risk is defined based on the predicted maps and a series of risk-aware spatio-temporal (RAST) safety corridors are constructed, which are finally used to optimize a dynamically-feasible collision-free trajectory for the MAV. We compared our method with several state-of-the-art works in 12000 simulation tests in Gazebo with the physical engine enabled. The results show that our method has the highest success rate at different uncertainty levels. Finally, we validated the proposed method in real experiments.@en

This letter addresses the problem of pushing manipulation with nonholonomic mobile robots. Pushing is a fundamental skill that enables robots to move unwieldy objects that cannot be grasped. We propose a stable pushing method that maintains stiff contact between the robot and the object to avoid consuming repositioning actions. We prove that a line contact, rather than a single point contact, is necessary for nonholonomic robots to achieve stable pushing. We also show that the stable pushing constraint and the nonholonomic constraint of the robot can be simplified as a concise linear motion constraint. Then the pushing planning problem can be formulated as a constrained optimization problem using nonlinear model predictive control (NMPC). According to the experiments, our NMPC-based planner outperforms a reactive pushing strategy in terms of efficiency, reducing the robot's traveled distance by 23.8% and time by 77.4%. Furthermore, our method requires four fewer hyperparameters and decision variables than the Linear Time-Varying (LTV) MPC approach, making it easier to implement. Real-world experiments are carried out to validate the proposed method with two differential-drive robots, Husky and Boxer, under different friction conditions.

@en

In this paper, we present a decentralized and communication-free collision avoidance approach for multi-robot systems that accounts for both robot localization and sensing uncertainties. The approach relies on the computation of an uncertainty-aware safe region for each robot to navigate among other robots and static obstacles in the environment, under the assumption of Gaussian-distributed uncertainty. In particular, at each time step, we construct a chance-constrained buffered uncertainty-aware Voronoi cell (B-UAVC) for each robot given a specified collision probability threshold. Probabilistic collision avoidance is achieved by constraining the motion of each robot to be within its corresponding B-UAVC, i.e. the collision probability between the robots and obstacles remains below the specified threshold. The proposed approach is decentralized, communication-free, scalable with the number of robots and robust to robots’ localization and sensing uncertainties. We applied the approach to single-integrator, double-integrator, differential-drive robots, and robots with general nonlinear dynamics. Extensive simulations and experiments with a team of ground vehicles, quadrotors, and heterogeneous robot teams are performed to analyze and validate the proposed approach.

@en

Planning safe motions for multi-robot systems is crucial for deploying them in real-world applications such as target tracking, environmental monitoring, and multi-view cinematography. Traditional approaches mainly solve the multi-robot motion planning problem in a deterministic manner, where the robot states and system models are perfectly known. Practically, however, many sources of uncertainty exist in real-world environments, such as noisy sensor measurements, motion disturbances, and uncertain behaviors of other decision-making agents. Reasoning about these uncertainties is of utmost importance for robust and safe navigation of multi-robot systems. To this end, this thesis aims to develop probabilistic methods for multi-robot motion planning under uncertainty. The first main contribution of this thesis is a Chance-Constrained Nonlinear Model Predictive Control (CCNMPC) method for probabilistic multi-robot motion planning. Taking into account uncertainties in robot localization, sensing, and motion disturbances, the method explicitly considers the collision probability between each robot and obstacle and formulates a model predictive control problem with chance constraints. A tight upper bound of the collision probability is developed which makes the CCNMPC formulation tractable and solvable in real time. In addition, the CCNMPC is incorporated into multi-robot motion planning using three coordination strategies: a) centralized sequential planning, b) distributed planning in which robots communicate their future planned trajectories, and c) decentralized planning in which robots predict other robots' trajectories using the constant velocity model (CVM). Performances of the three strategies are analyzed and compared. The CCNMPC method requires robots to know the future trajectories of other robots, either via communication or motion prediction using CVM. However, communication is not always available, and the CVM based motion prediction can lead to collisions among robots, especially in crowded environments. To achieve decentralized and communication-free multi-robot collision avoidance under uncertainty, this thesis then presents a method that relies on the introduced Buffered Uncertainty-Aware Voronoi Cell (B-UAVC). The B-UAVC defines a local safe region for each robot among other robots and obstacles, such that the collision probability between robots and obstacles is below a specified threshold if each robot's motion is constrained to be within its corresponding B-UAVC. An approach to constructing the B-UAVC is proposed, which leverages the techniques of computing a separating hyperplane between two Gaussian distributions and adding buffers for probabilistic collision avoidance. Based on B-UAVC, a set of reactive controllers are designed for single-integrator, double-integrator, and differential-drive robots, respectively; and a receding horizon planner is proposed for general nonlinear dynamical systems. Instead of directly generating a control action for each robot to move towards its waypoint goal as in the CCNMPC and B-UAVC methods, this thesis further presents a method that can compute a safe control input by minimally modifying a given nominal controller, which may come from a high-level task-oriented planner. The method is decentralized and relies on the Chance-Constrained Safety Barrier Certificates (CC-SBC), which defines a probabilistic safe control space for each robot in a multi-robot system considering robot localization and sensing uncertainties. The CC-SBC chance constraints are reformulated into a set of deterministic quadratic constraints, based on which a quadratically constrained quadratic program (QCQP) can be formulated. By solving the QCQP, the robot can obtain a safe control action thanks to that the CC-SBC guarantees forward invariance of the robot's safety set in a probabilistic manner. Hence, the CC-SBC method can be used as a probabilistic safety filter for multi-robot systems. While both the B-UAVC and CC-SBC methods are decentralized and communication-free, they typically lead to more conservative robot motions than the CCNMPC method with robots communicating their planner trajectories. The CCNMPC method can also be communication-free by letting each robot predict the other robots' trajectories using the constant velocity model, but it is unsafe in crowded environments. To address the issue, this thesis finally presents a novel trajectory prediction model based on Recurrent Neural Networks (RNN) that can learn multi-robot motion behaviors from demonstrated trajectories generated using a centralized motion planner. By incorporating the learned RNN-based trajectory prediction model within the MPC framework, efficient and communication-free multi-robot motion planning is achieved. The motion planning methods developed in the thesis have been extensively evaluated and validated in simulations and real-world experiments with a team of quadrotors, showing safe navigation of robots under uncertainty. @en

Autonomous mobile robots require accurate human motion predictions to safely and efficiently navigate among pedestrians, whose behavior may adapt to environmental changes. This paper introduces a self-supervised continual learning framework to improve data-driven pedestrian prediction models online across various scenarios continuously. In particular, we exploit online streams of pedestrian data, commonly available from the robot's detection and tracking pipelines, to refine the prediction model and its performance in unseen scenarios. To avoid the forgetting of previously learned concepts, a problem known as catastrophic forgetting, our framework includes a regularization loss to penalize changes of model parameters that are important for previous scenarios and retrains on a set of previous examples to retain past knowledge. Experimental results on real and simulation data show that our approach can improve prediction performance in unseen scenarios while retaining knowledge from seen scenarios when compared to naively training the prediction model online.

@en

Autonomous flight for UAVs relies on visual information for avoiding obstacles and ensuring safe collision-free flight. In addition to visual clues, safe UAVs often need connectivity with the ground station. In this article, we study the synergies between vision and communications for edge-computing-enabled UAV flight. By proposing a framework of edge computing assisted autonomous flight (ECAAF), we illustrate that vision and communications can interact with and assist each other with the aid of edge computing and offloading, and further speed up UAV mission completion. ECAAF consists of three functionalities that are discussed in detail: edge computing for 3D map acquisition, radio map construction from the 3D map, and online trajectory planning. During ECAAF, the interactions of communication capacity, video offloading, 3D map quality, and channel state of the trajectory form a positive feedback loop. Simulation results verify that the proposed method can improve mission performance by enhancing connectivity. Finally, we conclude with some future research directions.

@en

We address the problem of assigning a team of drones to autonomously capture a set desired shots of a dynamic target in the presence of obstacles. We present a two-stage planning pipeline that generates offline an assignment of drone to shots and locally optimizes online the viewpoint. Given desired shot parameters, the high-level planner uses a visibility heuristic to predict good times for capturing each shot and uses an Integer Linear Program to compute drone assignments. An online Model Predictive Control algorithm uses the assignments as reference to capture the shots. The algorithm is validated in hardware with a pair of drones and a remote controlled car.@en

This paper presents a data-driven decentralized trajectory optimization approach for multi-robot motion planning in dynamic environments. When navigating in a shared space, each robot needs accurate motion predictions of neighboring robots to achieve predictive collision avoidance. These motion predictions can be obtained among robots by sharing their future planned trajectories with each other via communication. However, such communication may not be available nor reliable in practice. In this paper, we introduce a novel trajectory prediction model based on recurrent neural networks (RNN) that can learn multi-robot motion behaviors from demonstrated trajectories generated using a centralized sequential planner. The learned model can run efficiently online for each robot and provide interaction-aware trajectory predictions of its neighbors based on observations of their history states. We then incorporate the trajectory prediction model into a decentralized model predictive control (MPC) framework for multi-robot collision avoidance. Simulation results show that our decentralized approach can achieve a comparable level of performance to a centralized planner while being communication-free and scalable to a large number of robots. We also validate our approach with a team of quadrotors in real-world experiments.

@en

This paper presents an online informative path planning approach for active information gathering on three-dimensional surfaces using aerial robots. Most existing works on surface inspection focus on planning a path offline that can provide full coverage of the surface, which inherently assumes the surface information is uniformly distributed hence ignoring potential spatial correlations of the information field. In this paper, we utilize manifold Gaussian processes (mGPs) with geodesic kernel functions for mapping surface information fields and plan informative paths online in a receding horizon manner. Our approach actively plans information-gathering paths based on recent observations that respect dynamic constraints of the vehicle and a total flight time budget. We provide planning results for simulated temperature modeling for simple and complex 3D surface geometries (a cylinder and an aircraft model). We demonstrate that our informative planning method outperforms traditional approaches such as 3D coverage planning and random exploration, both in reconstruction error and information-theoretic metrics. We also show that by taking spatial correlations of the information field into planning using mGPs, the information gathering efficiency is significantly improved.@en

Decentralized multi-robot systems typically perform coordinated motion planning by constantly broadcasting their intentions as a means to cope with the lack of a central system coordinating the efforts of all robots. Especially in complex dynamic environments, the coordination boost allowed by communication is critical to avoid collisions between cooperating robots. However, the risk of collision between a pair of robots fluctuates through their motion and communication is not always needed. Additionally, constant communication makes much of the still valuable information shared in previous time steps redundant. This paper presents an efficient communication method that solves the problem of "when"and with "whom"to communicate in multi-robot collision avoidance scenarios. In this approach, every robot learns to reason about other robots' states and considers the risk of future collisions before asking for the trajectory plans of other robots. We evaluate and verify the proposed communication strategy in simulation with four quadrotors and compare it with three baseline strategies: non-communicating, broadcasting and a distance-based method broadcasting information with quadrotors within a predefined distance.

@en

In this paper, we present an on-board vision-based approach for avoidance of moving obstacles in dynamic environments. Our approach relies on an efficient obstacle detection and tracking algorithm based on depth image pairs, which provides the estimated position, velocity and size of the obstacles. Robust collision avoidance is achieved by formulating a chance-constrained model predictive controller (CC-MPC) to ensure that the collision probability between the micro aerial vehicle (MAV) and each moving obstacle is below a specified threshold. The method takes into account MAV dynamics, state estimation and obstacle sensing uncertainties. The proposed approach is implemented on a quadrotor equipped with a stereo camera and is tested in a variety of environments, showing effective on-line collision avoidance of moving obstacles.

@en

Safe autonomous navigation of microair vehicles in cluttered dynamic environments is challenging due to the uncertainties arising from robot localization, sensing, and motion disturbances. This letter presents a probabilistic collision avoidance method for navigation among other robots and moving obstacles, such as humans. The approach explicitly considers the collision probability between each robot and obstacle and formulates a chance constrained nonlinear model predictive control problem (CCNMPC). A tight bound for approximation of collision probability is developed, which makes the CCNMPC formulation tractable and solvable in real time. For multirobot coordination, we describe three approaches, one distributed without communication (constant velocity assumption), one distributed with communication (of previous plans), and one centralized (sequential planning). We evaluate the proposed method in experiments with two quadrotors sharing the space with two humans and verify the multirobot coordination strategy in simulation with up to sixteen quadrotors.

@en

This paper presents a distributed method for splitting and merging of multi-robot formations in dynamic environments with static and moving obstacles. Splitting and merging actions rely on distributed consensus and can be performed to avoid obstacles. Our method accounts for the limited communication range and visibility radius of the robots and relies on the communication of obstacle-free convex regions and the computation of an intersection graph. In addition, our method is able to detect and recover from (permanent and temporary) communication and motion faults. Finally, we demonstrate the applicability and scalability of the proposed method in simulations with up to sixteen quadrotors and real-world experiments with a team of four quadrotors.

@en

This paper presents B-UAVC, a distributed collision avoidance method for multi-robot systems that accounts for uncertainties in robot localization. In particular, Buffered Uncertainty-Aware Voronoi Cells (B-UAVC) are employed to compute regions where the robots can safely navigate. By computing a set of chance constraints, which guarantee that the robot remains within its B-UAVC, the method can be applied to non-holonomic robots. A local trajectory for each robot is then computed by introducing these chance constraints in a receding horizon model predictive controller. The method guarantees, under the assumption of normally distributed position uncertainty, that the collision probability between the robots remains below a specified threshold. We evaluate the proposed method with a team of quadrotors in simulations and in real experiments.

@en