Active Learning To Reduce Human Labeling For Automatic Psychological Text Classification

Abstract

In recent years there has been an increase in the number of patients for issues relating to mental illness. To this effect to help with this increase, schema mode assessment through a conversation agent is being used to conduct schema therapy, a form of psychological treatment. To train such an agent, training data labeled by humans is necessary but can be very expensive to conduct. The question being researched is through the use of Active Machine learning is it possible to reduce the amount of required labeled data to do such classification. Three experiments on the use of active learning with currently available classifiers were performed where the active learner attempted to train the classifiers to an accuracy within +/- 3% of the same classifier trained with traditional machine learning on the full data set. The experimental results found that in all cases the use of active learning drastically decreased the number of necessary labeled data the classifier needed to achieve a similar accuracy. Consistently reducing the number by 98% and above answering the initial question. Though possible limitations of the data set and classifiers for such texts may be positively influencing the magnitude of the reduction.