Technological devices are ubiquitous, think of for example smartphones and in-vehicle information systems. Both can contribute towards distracted driving where the visual field of the human controller is shifted away from the primary control task. In this paper a neural network model is trained using the InceptionTime architecture and used to detect distractions in pursuit and preview tracking tasks. For this purpose an experiment has been designed to collect data in which participants are distracted using a visual distraction called the Surrogate Reference Task (SuRT). It was found that distractions are easier to detect in tracking tasks with pursuit displays instead of preview displays. This is because in preview displays the future target trajectory is shown to the human controller, resulting in a lower tracking error compared to pursuit displays. Apart from the tracking error, the InceptionTime neural network was also trained using the time-series data of the control input and system output. Important characteristic of distracted data found were a reduced control input and higher tracking errors, which may have helped in detecting distractions. The classification models were able to predict data samples correctly with an accuracy of 80.78% and 61.66% in pursuit and preview tracking tasks with distractions, respectively. Lastly, individualised models showed better performance when compared to 'one-size-fits-all models'. Results show clear opportunities for applying neural network models in real-time to detect distractions for increasing safety in human operated machines.