Task and motion planning for mobile manipulation requires jointly reasoning about high-level action sequences and the continuous motions needed to execute them. Classical TAMP methods typically rely on hand-designed task models and search over candidate symbolic and continuous solutions, which can become computationally expensive and limit rapid replanning when the environment changes. Generative approaches offer an alternative by learning planning behaviour directly from demonstrations, but jointly learning symbolic and continuous plans is particularly challenging when demonstrations must be collected through real-world teleoperation and are therefore limited in number.

This thesis introduces a Hybrid Flow Matching planner that jointly generates a discrete symbolic task plan and continuous trajectories for the mobile base and end-effector of a mobile manipulator. The method builds on a hybrid discrete--continuous generative planning by replacing the diffusion denoising and training elements with flow matching and introducing the following additional components. Modality embeddings are introduced to distinguish the two prediction modalities; plan-suffix augmentation increases the effective number of training samples by extracting several intermediate demonstrations from the original ones and exposes the model to those intermediate task states; and remasking allows low-confidence discrete predictions to be reconsidered during inference. The symbolic plan additionally defines action boundaries at which updated observations can be incorporated for closed-loop replanning.

The method is trained on 109 human-teleoperated demonstrations of a three-cube stacking task and evaluated on 12 validation demonstrations. Hybrid Flow Matching achieves a symbolic-plan validity of 55\%, with task-completion rates of 30\% in open-loop and 41\% in closed-loop execution, and a median end-effector RMSE of \(0.255\,\mathrm{m}\). Hybrid Diffusion trained on the same suffix-augmented data achieves higher symbolic validity at 62\%, but substantially lower task-completion rates of 6\% and 9\%, respectively, with an RMSE of \(0.710\,\mathrm{m}\). A motion-only Flow Matching baseline reaches 39\% and 52\% task completion, but does not provide an explicit symbolic plan.

Real-system experiments further provide evidence that the jointly generated symbolic and continuous plans can be executed on the real mobile manipulator and replanned after execution failures. Performance nevertheless degrades for under-represented long-horizon configurations, indicating that limited generalisation remains the main challenge. Overall, the results demonstrate the feasibility of joint symbolic and continuous generation with flow matching in a data-limited mobile-manipulation setting, while highlighting the trade-off between explicit task-level planning and continuous execution performance. ... Read more

We propose a framework that enables an agent to plan effectively under interaction uncertainty in decentralized multi-agent task and motion planning settings for cooperative manipulation tasks. In decentralized systems, each agent computes plans locally, based on local observations and assumptions on the other agents. Decentralization leads to each agent being uncertain about the actions and behavior of the other agents in the scene. We refer to this as ”interaction uncertainty”, as it arises from the implicit interaction between agents. This stems from limitations of the perception system and the inherent ambiguity of actions, meaning that an agent is not certain about the action being performed by the other agent(s). In other words, it is not certain if the other agents’ observed actions will obtain desired outcomes. In addition, an agent cannot predict the future behavior of the other agents or how this behavior is influenced by its own actions. The proposed framework addresses these two levels of interaction uncertainty in two-agent settings by leveraging the partial observability of the other agent’s short-term intent through ego’s perception system, and modeling the behavior of the other agent through an interactive MPD. The uncertainty about the other agent’s short- and long-term intent is represented using probabilistic effects of joint
actions. The focus of the thesis is on high-level, symbolic uncertainty in action recognition and preferences of the other agent, but may be extended to its low-level, geometric counterpart using existing methods in task and motion planning. Experiments are performed for different behavior models of the unknown agent, from the perspective of the protagonist or ego agent and are compared to baselines from sequential centralized planning. For most settings, the agents are able to eventually find the symbolic goal. Although the experiments are performed for a simple goal with two agents, the consistent convergence to the symbolic goal indicates that this approach may be extended to real-world settings with more complex ambiguity between actions and more agents acting collaboratively. ... Read more

We present Sadcher, a real-time task assignment framework for heterogeneous multi-robot teams that incorporates dynamic coalition formation and task precedence constraints. Sadcher is trained through Imitation Learning and combines graph attention and transformers to predict assignment rewards between robots and tasks. Based on the predicted rewards, a relaxed bipartite matching step generates high-quality schedules with feasibility guarantees. We explicitly model robot and task positions, task durations, and robots’ remaining processing times, enabling advanced temporal and spatial reasoning and generalization to environments with different spatiotemporal distributions compared to training. Trained on optimally solved small-scale instances, our method can scale to larger task sets and team sizes. Sadcher outperforms other learning-based and heuristic baselines on randomized, unseen prob-
lems for small and medium-sized teams with computation times suitable for real-time operation. Wealso explore sampling-based variants and evaluate scalability across robot and task counts. To address performance limitations identified for large-scale problem instances, we experiment with using Reinforcement Learning (RL) to fine-tune the imitation-learned model. Both discrete and continuous RL formulations are explored, leveraging Proximal Policy Optimization (PPO). This allows for training on larger problem instances, for which optimal solutions for IL are infeasible to obtain. Our RL experiments provide insights into the comparative advantages and trade-offs of IL and RL methods. In addition, we release our dataset of 250,000 optimal schedules to facilitate future research. We include a detailed description of the instance generation and the Mixed-Integer Linear Programming (MILP) formulation used to solve them optimally. Furthermore, this thesis contributes a lightweight
simulation environment with visualization tools for benchmarking different task assignment algorithms. ... Read more

This paper presents the first decentralized method to enable real-world 6-DoF manipulation of a cable-suspended load using a team of Micro-Aerial Vehicles (MAVs). Our method leverages multi-agent reinforcement learning (MARL) to train an outer-loop control policy for each MAV. Unlike state-of-the-art controllers that utilize a centralized scheme, our policy does not require global states, inter-MAV communications, nor neighboring MAV information. Instead, agents communicate implicitly through load pose observations alone, which enables high scalability and flexibility. It also significantly reduces computing costs during inference time, enabling onboard deployment of the policy. In addition, we introduce a new action space design for the MAVs using linear acceleration and body rates. This choice, combined with a robust low-level controller, enables reliable sim-to-real transfer despite significant uncertainties caused by cable tension during dynamic 3D motion. We validate our method in various real-world experiments, including full-pose control under load model uncertainties, showing setpoint tracking performance comparable to the state-of-the-art centralized method. We also demonstrate cooperation amongst agents with heterogeneous control policies, and robustness to the complete in-flight loss of one MAV. Videos of experiments: https://autonomousrobots.nl/paper_websites/aerial-manipulation-marl ... Read more