Interactive Imitation Learning for Robotic Tomato Picking using aGeneralist Robot Policy

Abstract

Robotic manipulation in unstructured environments remains challenging due to partial observability, contact-rich interactions, and task-level distribution shift. This thesis investigates an uncertainty-aware interactive imitation learning framework for robotic manipulation built on top of a pretrained generalist robot policy. Epistemic uncertainty, estimated using Monte Carlo Dropout, is used as a signal for when expert demonstrations may be requested during policy execution.

When uncertainty exceeds a predefined threshold, corrective demonstrations are selectively acquired and aggregated to further adapt the policy through offline imitation learning. The goal of this uncertainty-guided dataset aggregation strategy is to focus supervision on challenging parts of the task while reducing unnecessary expert input.

The framework is evaluated in simulation on robotic tomato harvesting as a representative task from biological agriculture. Additional experiments examine uncertainty behaviour under controlled perceptual degradation and analyse the influence of model capacity on imitation learning performance.

The results indicate that epistemic uncertainty increases under task-level distribution shift and decreases as task-specific data are aggregated. They further suggest that uncertainty-guided interaction can provide a useful signal for targeted data collection, although the observed gains over uninformed data collection remain limited and not yet conclusive in terms of task success.

Overall, the thesis presents a simulation-based proof of concept for uncertainty-aware adaptation of generalist robot policies to high-variability manipulation tasks, highlighting both its potential and its current limitations.