Task and Motion Planning with Flow Matching for Coupled Mobile Manipulation
Z.M. Redondo Garcia (TU Delft - Mechanical Engineering)
A. Matoses Gimenez – Mentor (TU Delft - Mechanical Engineering)
Javier Alonso-Mora – Mentor (TU Delft - Mechanical Engineering)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Task and motion planning for mobile manipulation requires jointly reasoning about high-level action sequences and the continuous motions needed to execute them. Classical TAMP methods typically rely on hand-designed task models and search over candidate symbolic and continuous solutions, which can become computationally expensive and limit rapid replanning when the environment changes. Generative approaches offer an alternative by learning planning behaviour directly from demonstrations, but jointly learning symbolic and continuous plans is particularly challenging when demonstrations must be collected through real-world teleoperation and are therefore limited in number.
This thesis introduces a Hybrid Flow Matching planner that jointly generates a discrete symbolic task plan and continuous trajectories for the mobile base and end-effector of a mobile manipulator. The method builds on a hybrid discrete--continuous generative planning by replacing the diffusion denoising and training elements with flow matching and introducing the following additional components. Modality embeddings are introduced to distinguish the two prediction modalities; plan-suffix augmentation increases the effective number of training samples by extracting several intermediate demonstrations from the original ones and exposes the model to those intermediate task states; and remasking allows low-confidence discrete predictions to be reconsidered during inference. The symbolic plan additionally defines action boundaries at which updated observations can be incorporated for closed-loop replanning.
The method is trained on 109 human-teleoperated demonstrations of a three-cube stacking task and evaluated on 12 validation demonstrations. Hybrid Flow Matching achieves a symbolic-plan validity of 55\%, with task-completion rates of 30\% in open-loop and 41\% in closed-loop execution, and a median end-effector RMSE of \(0.255\,\mathrm{m}\). Hybrid Diffusion trained on the same suffix-augmented data achieves higher symbolic validity at 62\%, but substantially lower task-completion rates of 6\% and 9\%, respectively, with an RMSE of \(0.710\,\mathrm{m}\). A motion-only Flow Matching baseline reaches 39\% and 52\% task completion, but does not provide an explicit symbolic plan.
Real-system experiments further provide evidence that the jointly generated symbolic and continuous plans can be executed on the real mobile manipulator and replanned after execution failures. Performance nevertheless degrades for under-represented long-horizon configurations, indicating that limited generalisation remains the main challenge. Overall, the results demonstrate the feasibility of joint symbolic and continuous generation with flow matching in a data-limited mobile-manipulation setting, while highlighting the trade-off between explicit task-level planning and continuous execution performance.