From industrial exoskeletons to implantable medical devices, robots that interact closely with people are poised to improve every aspect of our lives. Yet designing these systems is very challenging; humans are incredibly complex and, in many cases, we respond to robotic devices in ways that cannot be modelled or predicted with sufficient accuracy. A new approach, human-in-the-loop optimization, can overcome these challenges by systematically and empirically identifying the device characteristics that result in the best objective performance for a specific user and application. This approach has enabled substantial improvements in human-robot performance in research settings and has the potential to speed development and enhance products. In this Perspective, we describe methods for applying human-in-the-loop optimization to new human-robot interaction problems, addressing each key decision in a variety of contexts. We also identify opportunities to develop new optimization techniques and answer underlying scientific questions. We anticipate that our readers will advance human-in-the-loop optimization and use it to design robotic devices that truly enhance the human experience.