We propose a framework that enables an agent to plan effectively under interaction uncertainty in decentralized multi-agent task and motion planning settings for cooperative manipulation tasks. In decentralized systems, each agent computes plans locally, based on local observations and assumptions on the other agents. Decentralization leads to each agent being uncertain about the actions and behavior of the other agents in the scene. We refer to this as ”interaction uncertainty”, as it arises from the implicit interaction between agents. This stems from limitations of the perception system and the inherent ambiguity of actions, meaning that an agent is not certain about the action being performed by the other agent(s). In other words, it is not certain if the other agents’ observed actions will obtain desired outcomes. In addition, an agent cannot predict the future behavior of the other agents or how this behavior is influenced by its own actions. The proposed framework addresses these two levels of interaction uncertainty in two-agent settings by leveraging the partial observability of the other agent’s short-term intent through ego’s perception system, and modeling the behavior of the other agent through an interactive MPD. The uncertainty about the other agent’s short- and long-term intent is represented using probabilistic effects of joint
actions. The focus of the thesis is on high-level, symbolic uncertainty in action recognition and preferences of the other agent, but may be extended to its low-level, geometric counterpart using existing methods in task and motion planning. Experiments are performed for different behavior models of the unknown agent, from the perspective of the protagonist or ego agent and are compared to baselines from sequential centralized planning. For most settings, the agents are able to eventually find the symbolic goal. Although the experiments are performed for a simple goal with two agents, the consistent convergence to the symbolic goal indicates that this approach may be extended to real-world settings with more complex ambiguity between actions and more agents acting collaboratively.

We present Sadcher, a real-time task assignment framework for heterogeneous multi-robot teams that incorporates dynamic coalition formation and task precedence constraints. Sadcher is trained through Imitation Learning and combines graph attention and transformers to predict assignment rewards between robots and tasks. Based on the predicted rewards, a relaxed bipartite matching step generates high-quality schedules with feasibility guarantees. We explicitly model robot and task positions, task durations, and robots’ remaining processing times, enabling advanced temporal and spatial reasoning and generalization to environments with different spatiotemporal distributions compared to training. Trained on optimally solved small-scale instances, our method can scale to larger task sets and team sizes. Sadcher outperforms other learning-based and heuristic baselines on randomized, unseen prob-
lems for small and medium-sized teams with computation times suitable for real-time operation. Wealso explore sampling-based variants and evaluate scalability across robot and task counts. To address performance limitations identified for large-scale problem instances, we experiment with using Reinforcement Learning (RL) to fine-tune the imitation-learned model. Both discrete and continuous RL formulations are explored, leveraging Proximal Policy Optimization (PPO). This allows for training on larger problem instances, for which optimal solutions for IL are infeasible to obtain. Our RL experiments provide insights into the comparative advantages and trade-offs of IL and RL methods. In addition, we release our dataset of 250,000 optimal schedules to facilitate future research. We include a detailed description of the instance generation and the Mixed-Integer Linear Programming (MILP) formulation used to solve them optimally. Furthermore, this thesis contributes a lightweight
simulation environment with visualization tools for benchmarking different task assignment algorithms.

This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV. Unlike state-of-the-art controllers that utilize a centralized scheme, our policy does not require global states, inter-MAV communications, nor neighboring MAV information. Instead, agents communicate implicitly through load pose observations alone, which enables high scalability and flexibility. It also significantly reduces computing costs during inference time, enabling onboard deployment of the policy. In addition, we introduce a new action space design for the MAVs using linear acceleration and body rates. This choice, combined with a robust low-level controller, enables reliable sim-to-real transfer despite significant uncertainties caused by cable tension during dynamic 3D motion. We validate our method in various real-world experiments, including full-pose control under load model uncertainties, showing setpoint tracking performance comparable to the state-of-the-art centralized method. We also demonstrate cooperation amongst agents with heterogeneous control policies, and robustness to the complete in-flight loss of one MAV. Videos of experiments: https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl