Annotation practices in affective computing
What are these algorithms actually trained on?
S.J.M. Backer (TU Delft - Electrical Engineering, Mathematics and Computer Science)
Cynthia CS Liem – Mentor (TU Delft - Multimedia Computing)
A.M. Demetriou – Mentor (TU Delft - Multimedia Computing)
F. Broz – Graduation committee member (TU Delft - Interactive Intelligence)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
In the machine learning research community, significant importance is given to the optimization of techniques which are employed once a benchmark dataset is given. However, less importance is assigned to the quality of these datasets and to how these datasets are obtained. In this work, we look into annotation practices in the research area of affective computing, analysing datasets of emotion classification tasks from text, video, audio, EEG data and more. We find annotation practices of varying quality and recommend that annotation practices be improved, especially with regard to multiple annotator overlap.