Task recommendation in human computation

Abstract

Crowdsourcing and Human computation have enabled industry, and scientists to create innovative solutions by harnessing organized collective human effort. In human computation platforms, it is observed that workers spend a considerable amount of time searching for appropriate tasks, thus losing revenues that they could have made and, ultimately, affecting their motivation, productivity and quality of their work. Task recommendation can help solving this problem, by suggesting workers with the task most suited to them. To enable effective task recommendation, access to worker profiles, as well as execution history is fundamental. Also, commercial human computation platform act as black-boxes to researcher, limiting access to such information. This work provides a threefold contribution to this field of research. First, we propose Bruteforce, a novel human computation platform that provides worker profiling and task recommendation capabilities. By interacting with existing (commercial) solutions, Bruteforce provides researchers and practitioners with an extensible and configurable tool to conduct studies over (and with) human computation platforms. Second, we make available a novel dataset specifically designed to support studies in the task recommendation. Finally, we evaluate the performance of several recommendation algorithms on the new dataset, addressing several common use cases for human computation platforms.