This research investigates the question how the choice of scenarios and metrics influences the calibration of pedestrian simulation models. By calibrating a pedestrian model using different combinations of scenarios and metrics and comparing the results it was concluded that it matters what scenarios and metrics are included during the calibration. The implication of this is that one should calibrate a pedestrian model based on its intended usage and that this should be leading in the choice of which scenarios and metrics to use.