Microtask crowdsourcing for music score Transcriptions: an experiment with error detection

Microtask crowdsourcing for music score Transcriptions: an experiment with error detection

Samiotis, I.P. (TU Delft Web Information Systems)
Qiu, S. (TU Delft Web Information Systems)
Mauri, A. (TU Delft Human-Centred Artificial Intelligence)
Liem, C.C.S. (TU Delft Multimedia Computing)
Lofi, C. (TU Delft Web Information Systems)
Bozzon, A. (TU Delft Human-Centred Artificial Intelligence)

2020

Human annotation is still an essential part of modern transcription workflows for digitizing music scores, either as a standalone approach where a single expert annotator transcribes a complete score, or for supporting an automated Optical Music Recognition (OMR) system. Research on human computation has shown the effectiveness of crowdsourcing for scaling out human work by defining a large number of microtasks which can easily be distributed and executed. However, microtask design for music transcription is a research area that remains unaddressed. This paper focuses on the design of a crowdsourcing task to detect errors in a score transcription which can be deployed in either automated or human-driven transcription workflows. We conduct an experiment where we study two design parameters: 1) the size of the score to be annotated and 2) the modality in which it is presented in the user interface. We analyze the performance and reliability of non-specialised crowdworkers on Amazon Mechanical Turk with respect to these design parameters, differentiated by worker experience and types of transcription errors. Results are encouraging, and pave the way for scalable and efficient crowdassisted music transcription systems.

Domain knowledge
Representations of music
Evaluation
datasets and reproducibility
Annotation protocols
Human-centered MIR
Human-computer interaction and interfaces, MIR fundamentals and methodology, Multimodality, MIR tasks, Music transcription and annotation, Optical Music Recognition (OMR)

http://resolver.tudelft.nl/uuid:56ea7e1a-55b9-4696-aaba-45f2749d2089

978-0-9813537-0-8

Proceedings of the 21st International Society for Music Information Retrieval Conference

21st International Society for Music Information Retrieval Conference, 2020-10-11 → 2020-10-15

Institutional Repository

conference paper

© 2020 I.P. Samiotis, S. Qiu, A. Mauri, C.C.S. Liem, C. Lofi, A. Bozzon