Human–Machine Interfaces (HMIs) in passenger cars have become more complex over the years, with touch screens replacing physical buttons and with layered menu-structures. This can lead to distractions. The purpose of this study is to investigate how often vehicle controls are used while driving and which underlying factors contribute to usage. Thirty drivers were observed during driving a familiar route twice, in their own car and in an unfamiliar car. In a 2 × 1 within-subject design, the experimenter drove along with each participant and used a predefined checklist to record how often participants interacted with specific functions of their vehicle while driving. The results showed that, in the familiar car, direction indicators are the most frequently used controls, followed by adjusting radio volume, moving the sun visor, adjusting temperature and changing wiper speed. Factors that influenced task frequencies included car familiarity, gender, age and weather conditions. The type of car also appears to impact task frequency. Participants interacted less with the unfamiliar car, compared to their own car, which may indicate drivers are regulating their mental load. These results are relevant for vehicle HMI designers to understand which functions should be easily and swiftly available while driving to reduce distraction by the HMI design.