Proposing Threshold Concepts in Machine Learning

Abstract

Machine learning (ML) courses are difficult to design and teach for many reasons: new approaches emerge rapidly, and students are already overwhelmed with the volume of material. This position paper addresses the question of what is essential to teach in ML courses through the framework of threshold concepts - -transformative ways of understanding that fundamentally shift how learners view a discipline. We propose five threshold concepts in ML: ''Learning is Optimization,'' ''There are Tradeoffs in Sources of Error,'' ''ML is an Empirical Science,'' ''ML Describes Geometric Processes,'' and ''ML Demands a Probabilistic Lens.'' We believe the first three concepts to be essential for students aiming to become adept builders of ML systems, and the remaining two to be necessary for graduate-level research in developing new ML methods. For each concept, we analyze whether and how it is transformative, troublesome, integrative, irreversible, and bounded. These concepts are intentionally chosen to be broad and model-agnostic, so that they can be applied to a wide range of settings and remain relevant as the field progresses. Our proposals reflect the reasoned analysis of experienced ML educators and offer a learner-centered framework for prioritizing limited instructional time while helping students develop a deeper understanding of ML. Furthermore, we contribute to the ongoing conversation about the role of mathematics in ML education by arguing that mathematics is necessary for the two theoretical concepts and for moving beyond mimicry to true understanding through the liminal space of threshold concepts.