Sequentially Teaching Sequential Tasks (ST)²

Abstract

Learning from demonstration (LfD) has proved useful for teaching robots complex skills with high sample efficiency. However, teaching long-horizon tasks with multiple skills is challenging as deviations tend to accumulate, the distributional shift becomes more evident, and human teachers become fatigued over time, thereby increasing the likelihood of failure. To address these challenges, we introduce (ST)² a sequential method for learning long-horizon manipulation tasks that allows users to control teaching flow by specifying keypoints, enabling structured and incremental demonstrations. Using this framework, we study how users respond to two teaching paradigms: 1) a traditional monolithic approach in which users demonstrate the entire task trajectory at once, and 2) a sequential approach, in which the task is segmented and demonstrated step by step. We conducted an extensive user study on the restocking task with 16 participants in a realistic retail store environment, evaluating the user preferences and effectiveness of the methods. A user-level analysis showed superior performance for the sequential approach in most cases (10 users), compared with the monolithic approach (five users), with one tie. Our subjective results indicate that some teachers prefer sequential teaching - as it allows them to teach complicated tasks iteratively - or others prefer teaching in one go due to its simplicity.