Print Email Facebook Twitter Evaluating Crowdworkers as a Proxy for Online Learners in Video-Based Learning Contexts Title Evaluating Crowdworkers as a Proxy for Online Learners in Video-Based Learning Contexts Author Davis, D.J. (TU Delft Web Information Systems) Hauff, C. (TU Delft Web Information Systems) Houben, G.J.P.M. (TU Delft Web Information Systems) Date 2018 Abstract Crowdsourcing has emerged as an effective method of scaling-up tasks previously reserved for a small set of experts. Accordingly, researchers in the large-scale online learning space have begun to employ crowdworkers to conduct research about large-scale, open online learning. We here report results from a crowdsourcing study (N=135) to evaluate the extent to which crowdworkers and MOOC learners behave comparably on lecture viewing and quiz tasks---the most utilized learning activities in MOOCs. This serves to (i) validate the assumption of previous research that crowdworkers are indeed reliable proxies of online learners and (ii) address the potential of employing crowdworkers as a means of online learning environment testing. Overall, we observe mixed results---in certain contexts (quiz performance and video watching behavior) crowdworkers appear to behave comparably to MOOC learners, and in other situations (interactions with in-video quizzes), their behaviors appear to be disparate. We conclude that future research should be cautious if employing crowdworkers to carry out learning tasks, as the two populations do not behave comparably on all learning-related activities. Subject Learning AnalyticsMOOCsReplicationCrowdwork To reference this document use: http://resolver.tudelft.nl/uuid:4d7b65bf-d139-40c7-a208-177454caf60c DOI https://doi.org/10.1145/3274311 Embargo date 2019-05-31 Source ACM Proceedings on Human-Computer Interaction, 2 (CSCW), 1-16 Bibliographical note Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. Part of collection Institutional Repository Document type journal article Rights © 2018 D.J. Davis, C. Hauff, G.J.P.M. Houben Files PDF cscw042_davis.pdf 2.29 MB Close viewer /islandora/object/uuid:4d7b65bf-d139-40c7-a208-177454caf60c/datastream/OBJ/view