In urban areas, drivers frequently interact with vulnerable road users. On-road studies have shown that drivers are more likely to have safety-relevant interactions with pedestrians when they are inattentive and when pedestrians behave unexpectedly. Notwithstanding these behavioural effects, most microscopic traffic flow models do not accurately describe driver response to pedestrian crossing behaviour. This study investigates the factors influencing driver behaviour characteristics when pedestrians cross the road in front of the vehicle. The data were collected in the UDRIVE naturalistic driving study in France and the UK. The interactions with pedestrians in daylight were identified using the MobilEye® smart camera. The minimum time to zebra and the maximum deceleration during each interaction were investigated in regression models. The results showed that, controlling for the initial speed of the subject vehicle, the minimum time to zebra during interactions was significantly shorter when the pedestrian crossed while the driver had a green traffic light, the vehicle segment was medium, and other pedestrians had already crossed. Controlling for initial speed and acceleration, the maximum deceleration during interactions was lower when the pedestrian crossed while the driver had a green traffic light, no other pedestrians had already crossed, the pedestrian was not a child, teenager or elderly person, and the pedestrian did not glance toward the vehicle. These factors can be incorporated into traffic simulations to describe driver responses more realistically. Further research is needed to understand the influence of the driver’s state because most drivers looked toward pedestrians.