This study examines the effect of analogies on conceptual understanding of machine learning (ML) loss functions, and the motivation to learn in first-year bachelor computer science students. For a set of 10 ML loss functions, analogies were generated and evaluated by 15 experts. 3 of these analogies were subsequently tested with 22 students. The results show no conclusive evidence for improvement in understanding and motivation to learn. The study outlines a general strategy for evaluation of analogies on student understanding and motivation. The study further provides 10 expert-rated analogies, 3 of which have been tested with students.