Learning analytics dashboards (LADs) are designed as feedback tools for learners, but until recently, learners rarely have had a say in how LADs are designed and what information they receive through LADs. To overcome this shortcoming, we have developed a customisable LAD for Coursera MOOCs on which learners can set goals and choose indicators to monitor. Following a mixed-methods approach, we analyse 401 learners' indicator selection behaviour in order to understand the decisions they make on the LAD and whether learner goals and self-regulated learning skills influence these decisions. We found that learners overwhelmingly chose indicators about completed activities. Goals are not associated with indicator selection behaviour, while help-seeking skills predict learners' choice of monitoring their engagement in discussions and time management skills predict learners' interest in procrastination indicators. The findings have implications for our understanding of learners' use of LADs and their design.

@en

The purpose of this paper is to explore the expectations of academic staff to learning analytics services from an ideal as well as a realistic perspective. This mixed-method study focused on a cross-case analysis of staff from Higher Education Institutions from four European universities (Spain, Estonia, Netherlands, UK). While there are some differences between the countries as well as between ideal and predicted expectations, the overarching results indicate that academic staff sees learning analytics as a tool to understand the learning activities and possibility to provide feedback for the students and adapt the curriculum to meet learners' needs. However, one of the findings from the study across cases is the generally consistently low expectation and desire for academic staff to be obligated to act based on data that shows students being at risk of failing or under-performing.

@en

In March 2020 COVID-19 brought the world and with that aviation to a standstill. Also in March 2020, the third run of the DelftX MOOC Introduction to Aerospace Structures and Materials started on edX. This MOOC generally attracts a mixture of young aviation enthusiasts (often students) and aviation professionals. Given the large interest MOOCs have received as the pandemic hit, we investigate how the new global context affected the motivation and the way learners interact with our course material. For this project, we will use learning analytics approaches to analyse the log data available from the edX platform and the data from pre- and post-course evaluations of two runs of the same MOOC (2019 and 2020). With the insights gathered through this analysis, we wish to better understand our learners and adjust the learning design of the course to better suit their needs. Our paper will present the first insights of this analysis.@en

This paper presents results from our design and evaluation studies of the Learning Analytics Cockpit (LA Cockpit) for a quiz app, which aims to provide lecturers with important information about students’ knowledge levels. We define a LA Cockpit as a tool for instructors that enables them to steer students’ learning process by providing a LA Dashboard which visualizes students’ learning indicators and an intervention feature enabling instructors to give feedback based on students’ knowledge levels. To address the needs of lecturers we applied the Double Diamond (DD) design process model which consists of four stages: discover, define, develop & refine. Following the DD process, we first conducted a qualitative study by interviewing four lecturers and student teachers to discover their needs. Results from the interviews allowed us to define requirements of the lecturers. We used these results to develop the first version of the tool where we refined it through informal feedback by the interviewed teachers. In preparation for a larger effectiveness-study, we evaluated the LA Cockpit in terms of usefulness and usability in a preliminary study with 16 university lecturers. Results from this qualitative study indicate that the LA Cockpit can measure the students’ knowledge level and supports self-reflection for lecturers. Moreover, results show that the LA Cockpit enables lecturers to address knowledge gaps and provide interventions to students before the exams.

@en

Unequal stakeholder engagement is a common pitfall of adoption approaches of learning analytics in higher education leading to lower buy-in and flawed tools that fail to meet the needs of their target groups. With each design decision, we make assumptions on how learners will make sense of the visualisations, but we know very little about how students make sense of dashboard and which aspects influence their sense-making. We investigated how learner goals and self-regulated learning (SRL) skills influence dashboard sense-making following a mixed-methods research methodology: a qualitative pre-study followed-up with an extensive quantitative study with 247 university students. We uncovered three latent variables for sense-making: transparency of design, reference frames and support for action. SRL skills are predictors for how relevant students find these constructs. Learner goals have a significant effect only on the perceived relevance of reference frames. Knowing which factors influence students' sense-making will lead to more inclusive and flexible designs that will cater to the needs of both novice and expert learners.

@en

This paper aims to link student facing Learning Analytics Dashboards (LADs) to the corpus of research on Open Learner Models (OLMs), as both have similar goals. We conducted a systematic review of literature on OLMs and compared the results with a previously conducted review of LADs for learners in terms of (i) data use and modelling, (ii) key publication venues, (iii) authors and articles, (iv) key themes, and (v) system evaluation. We highlight the similarities and differences between the research on LADs and OLMs. Our key contribution is a bridge between these two areas as a foundation for building upon the strengths of each. We report the following key results from the review: in reports of new OLMs, almost 60% are based on a single type of data; 33% use behavioral metrics; 39% support input from the user; 37% have complex models; and just 6% involve multiple applications. Key associated themes include intelligent tutoring systems, learning analytics, and self-regulated learning. Notably, compared with LADs, OLM research is more likely to be interactive (81% of papers compared with 31% for LADs), report evaluations (76% versus 59%), use assessment data (100% versus 37%), provide a comparison standard for students (52% versus 38%), but less likely to use behavioral metrics, or resource use data (33% against 75% for LADs). In OLM work, there was a heightened focus on learner control and access to their own data.

@en

Learning analytics for learners has the ability to greatly improve learners' self-regulation. Current learner dashboards are mostly providing learners with an isolated view of their learning behavior, while we believe learners will gain more from a comparison of their own behavior with that of successful peer learners. In this work-in-progress demonstration we describe our design of a Learning Tracker widget that provides MOOC learners with timely and goal-oriented (i.e. towards passing the course) feedback in a manner that encourages reflection and self-regulation. We also present some preliminary ndings which show how exposure to feedback can signicantly increase student success and engagement.@en