Manual tuning of performance-critical controller parameters can be tedious and sub-optimal. Bayesian Optimization (BO) is an increasingly popular practical alternative to automatically optimize controller parameters from few experiments. Standard BO practice is to evaluate the closed-loop performance of parameters proposed during optimization on an episode with a fixed length. However, fixed-length episodes can be wasteful. For example, continuing an episode where already the start shows undesirable behavior such as strong oscillations seems pointless. Therefore, we propose a BO method that stops an episode early if suboptimality becomes apparent before an episode is completed. Such early stopping results in partial observations of the controller’s performance, which cannot directly be included in standard BO. We propose three heuristics to facilitate partially observed episodes in BO. Through five numerical and one hardware experiment, we demonstrate that early stopping BO can substantially reduce the time needed for optimization.