Detecting human operator distraction is critical for safety in high-risk domains like aeronautics and the automotive industry. This paper explores distraction detection by framing the problem as both binary classification and anomaly detection and investigates the feasibility of training solely on not distracted data. Three approaches are evaluated: first, an ensemble Time Series Classifier for binary classification using both distracted and not distracted data; second, an LSTM-based auto-encoder trained only on not distracted data for binary classification; and third, a hybrid anomaly detector combining One-Class Support Vector Machine and Mahalanobis Distance, trained on not distracted data. These methods are tested on an existing dataset of pursuit tracking runs with a side task. The Time Series Classifier achieves 86.47% accuracy and an F1 score of 0.42 in the binary classification task, improving accuracy by over 30% and F1 by 0.17 compared to prior methods. However, it suffers from high false alarms and sensitivity to domain shifts due to its reliance on distracted and not distracted training data. The auto-encoder achieves an Area Under Curve of 0.76 and performs best in low false-positive scenarios. For the anomaly detection task, the combined anomaly detector outperforms individual models by reducing false alarms and improving recall, particularly in low false alarm regions, achieving precision above 0.85 and F1 scores exceeding 0.51, surpassing the ensemble time series classifier and the auto-encoder. These findings highlight the strengths of statistical methods and machine learning in distraction detection, providing a foundation for future advancements in human operator safety and distraction mitigation.