Exploring the Music Perception Skills of Crowd Workers

Exploring the Music Perception Skills of Crowd Workers

Samiotis, I.P. (TU Delft Web Information Systems)
Qiu, S. (TU Delft Web Information Systems)
Lofi, C. (TU Delft Web Information Systems)
Yang, J. (TU Delft Web Information Systems)
Gadiraju, Ujwal (TU Delft Web Information Systems)
Bozzon, A. (TU Delft Human-Centred Artificial Intelligence)

2021

Music content annotation campaigns are common on paid crowdsourcing platforms. Crowd workers are expected to annotate complicated music artefacts, which can demand certain skills and expertise. Traditional methods of participant selection are not designed to capture these kind of domain-specific skills and expertise, and often domain-specific questions fall under the general demographics category. Despite the popularity of such tasks, there is a general lack of deeper understanding of the distribution of musical properties - especially auditory perception skills - among workers. To address this knowledge gap, we conducted a user study (N=100) on Prolific. We asked workers to indicate their musical sophistication through a questionnaire and assessed their music perception skills through an audio-based skill test. The goal of this work is to better understand the extent to which crowd workers possess higher perceptions skills, beyond their own musical education level and self reported abilities. Our study shows that untrained crowd workers can possess high perception skills on the music elements of melody, tuning, accent and tempo; skills that can be useful in a plethora of annotation tasks in the music domain.

http://resolver.tudelft.nl/uuid:c5268eb6-3714-4bd0-94ca-19d668d27fb3

2021-12-01

Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, 9, 108-119

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

journal article

© 2021 I.P. Samiotis, S. Qiu, C. Lofi, J. Yang, Ujwal Gadiraju, A. Bozzon