Classifying Human Manual Control Behavior in Tracking Tasks with Various Display types Using the InceptionTime CNN

Abstract

Despite the increasing amount of automation in manual control tasks, such as driving a car or piloting an aircraft, the human ability to adapt to unexpected events still makes us an essential part of the control loop. Before we can remove the human completely, a better understanding of this unique characteristic is necessary so that we can apply it to automation itself. Currently, however, research opportunities are limited mainly due to a lack of techniques that can recognize such control adjustments within short periods of time. One well known form of adaptation is characterized by changes in control behavior in response to three types of displays used in manual control tasks: compensatory, pursuit and preview. This research aims to use time series classification to recognize such changes using samples of 1.5 s of tracking data from experiments with double integrator controlled element dynamics. InceptionTime, a convolutional neural network, is optimized for the task, resulting in an average attained accuracy of 95.4%. It was found that the best configuration is a basic three-class classifier that uses the e, u, x signals and their first-order derivatives as input features. Additionally, results show performance highly depends on what data sets are used for training and testing, with accuracies ranging from 60% to 99% for different train-test allocations. Most errors occur in separating the pursuit and preview strategies, where it was expected to be between compensatory and pursuit. In general, the results exceed expectations and could signify a breakthrough in recognizing and understanding human adaptation in future experiments.