Just the right mood for HIT!: Analyzing the role of worker moods in conversational microtask crowdsourcing
Qiu, S. (TU Delft Web Information Systems)
Gadiraju, Ujwal (TU Delft Web Information Systems)
Bozzon, A. (TU Delft Web Information Systems; TU Delft Human-Centred Artificial Intelligence)
Bielikova, Maria (editor)
Mikkonen, Tommi (editor)
Pautasso, Cesare (editor)
Conversational agents are playing an increasingly important role in providing users with natural communication environments, improving outcomes in a variety of domains in human-computer interaction. Crowdsourcing marketplaces are simultaneously flourishing, and it has never been easier to acquire large-scale human input from online workers. Recent works have revealed the potential of conversational interfaces in improving worker engagement and satisfaction. At the same time, worker moods have been shown to have significant effects on quality related outcomes. Little is known about the role of worker moods in shaping work in conversational microtask crowdsourcing. In this paper, we conducted a crowdsourcing study addressing 600 unique online workers, to investigate the role that worker moods play in conversational microtask crowdsourcing. We also explore whether suitable conversational styles of the agent can affect the performance of workers in different moods. Our results show that workers in a pleasant mood tend to produce significantly higher quality results (over 20%), exhibit greater engagement (an increase by around 19%) and report a lower cognitive load (by over 12%), and a suitable conversational style can have a significant impact on workers in different moods. Our findings advance the current understanding of conversational microtask crowdsourcing and have important implications on designing future conversational crowdsourcing systems.
To reference this document use:
Springer Science+Business Media, Cham
Web Engineering - 20th International Conference, ICWE 2020, Proceedings
20th International Conference on Web Engineering, ICWE 2020, 2020-06-09 → 2020-06-12, Helsinki, Finland
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 12128
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
© 2020 S. Qiu, Ujwal Gadiraju, A. Bozzon