Designing the Learning Analytics Cockpit - A Dashboard that Enables Interventions

Abstract

This paper presents results from our design and evaluation studies of the Learning Analytics Cockpit (LA Cockpit) for a quiz app, which aims to provide lecturers with important information about students’ knowledge levels. We define a LA Cockpit as a tool for instructors that enables them to steer students’ learning process by providing a LA Dashboard which visualizes students’ learning indicators and an intervention feature enabling instructors to give feedback based on students’ knowledge levels. To address the needs of lecturers we applied the Double Diamond (DD) design process model which consists of four stages: discover, define, develop & refine. Following the DD process, we first conducted a qualitative study by interviewing four lecturers and student teachers to discover their needs. Results from the interviews allowed us to define requirements of the lecturers. We used these results to develop the first version of the tool where we refined it through informal feedback by the interviewed teachers. In preparation for a larger effectiveness-study, we evaluated the LA Cockpit in terms of usefulness and usability in a preliminary study with 16 university lecturers. Results from this qualitative study indicate that the LA Cockpit can measure the students’ knowledge level and supports self-reflection for lecturers. Moreover, results show that the LA Cockpit enables lecturers to address knowledge gaps and provide interventions to students before the exams.