Collaborative transportation and manipulation of cable-suspended loads by multiple UAVs offer a promising way for expanding UAVs’ role in heavy-lifting operations. Existing approaches for collaborative aerial manipulation of a payload along a reference trajectory typically rely either on centralized control architectures or on reliable inter-agent communication. In this work, we propose a novel machine learning–based method for decentralized kinodynamic planning that operates effectively under partial observability and without inter-agent communication. Our method leverages imitation learning to train a decentralized homogenous student policy for each UAV by imitating a centralized kinodynamic motion planner which has access to privileged global observations. The student policy uses physics-informed neural networks that respect the derivative relationships of motion to generate trajectory that are step-wise consistent and guaranteed to be kinematically feasible. During training, the student policies utilize the full trajectory generated by the teacher policy, leading to improved sample efficiency. Therefore, the student policy can be trained in under two hours on modest hardware. We validate our method in both simulation and real-world environments to follow an agile reference trajectory, demonstrating performance comparable to that of centralized approaches.