Sound evaluation based on listening experiments is very costly and time-consuming. Alternatively, the sounds under evaluation can be compared using psychoacoustic metrics, under the hypothesis that sounds that differ by more than one just-noticeable difference (JND) are perceived as significantly different from each other. Wellestablished psychoacoustic metrics are obtained from models of loudness, sharpness, roughness, fluctuation strength, and tonality. In the present study, we use these psychoacoustic models to analyse a selection of aircraft flyovers and train pass-by sounds, as well as fluctuating sounds obtained from resonances of a rotating flexible tube. In addition to the description of the adopted model implementation of each psychoacoustic metric, our discussion focuses on technical and practical aspects including (1) the dependency of the results on the appropriate sound and model calibration, and (2) how to interpret the results based on reported JNDs from the literature. We reflect on how much we can rely on a specific model implementation considering the status of the metric—standardised (or not), accessibility to algorithms and to their validation results (or not)—and external factors that may influence the obtained estimates.