The Real-Time Learning Tracker

Abstract

Massive Open Online Courses (MOOCs) have the potential to make quality education affordable and available to the masses and reduce the gap between the most privileged and the most disadvantaged learners worldwide. However, this potential is overshadowed by low completion rates, often below 15%. Due to the high level of autonomy that is required when learning with a MOOC, literature identifies limited self-regulated learning skills as one of the causes that lead to early dropouts in MOOCs. Moreover, most of the existing tools designed to aid learners in the online learning environment fail to provide the support needed for the development of such skills. The aim of the present work is to investigate how self-regulated learning skills can be enhanced by encouraging metacognition and reflection in MOOC learners by means of real-time personalised learner feedback through social comparison. To this end, we have developed three versions of an interactive widget, the Real- Time Learning Tracker , which allows learners to visualize in real-time changes in their learning behaviour and compare it to that of previous graduates of the same MOOC. The three versions of the Real-Time Learning Tracker differ in the degree of complexity of the presented feedback so as to investigate how learners interpret varying visualisations of their learning behaviour and which one of them leads to more changes on their behaviour during the course. The Real-Time Learning Tracker was evaluated in a live TU Delft MOOC running on the edX platform while engaging nearly 2000 MOOC learners over the course of 10 weeks. Our results show that learners that have access to the Real-Time Learning Tracker are more likely to graduate the MOOC. Moreover, we have observed that the widget has a positive impact on learners’ self-regulation whereas, we have little evidence that learners developed their engagement with the course content by the end of our experiment. Based on our results, we argue that the mere fact of receiving real-time feedback on a limited number of learning habits could trigger self-reflection in learners and lead to improved learner performance. Finally, our results reveal that the exposure of learners to the most detailed version of the widget engaged more learners and this type of feedback affected positively the learning performance and behaviour of highly educated learners. This work underlines the powerful effect of real-time feedback and self-reflection on one’s learning performance and behaviour. We recommend that future research should investigate learners’ feedback literacy and devise effective ways of presenting learners with real-time personalized feedback based on their goals and learning skill level.