Learners in the process of developing complex skills need a rich mental model of what such skills entail. Textual analytics rubrics (TR) are a widely used instrument to support formative assessment of complex skills, supporting feedback, reflection, and thus mental model development of complex skills. However, the textual nature of a rubric limits the possibility to deliver contextual and dynamic (time-and behaviour-oriented) information. In the Viewbrics online tool, we developed a version supporting the delivery of contextual and dynamic information by adding video-modelling examples with embedded self-explanation prompts to textual analytics rubrics. We called this combination as video-enhanced rubrics (VERs). Our current study investigates whether the Viewbrics online tool supports complex skills development and whether either textual- or video-enhanced rubrics best support complex skills' mastery. The study was a three-group (VERS n = 49, TR n = 54, control n = 50) within-subjects design. Learners' performance of complex skills was measured through expert, peer, and self-assessment using the Viewbrics online tool. A multilevel regression analysis shows learners in the TR and VERS conditions consistently outperforming the control condition to varying degrees across skills. However, no differences have been found between the two experimental conditions. Positive results across different complex skills indicate the Viewbrics online tool can be used to support the development of a wide range of complex skills in secondary education.@en

Learners can experience difficulties to form a mental model of a complex skill and what its mastery implicates in terms of actual behavior. Text-based analytic rubrics can support training and formative assessment of complex skills. However, they lack (1) contextual information needed to convey real-world attributes (2) sensor-motoric information (such as gesturing in the complex skill of presenting) (3) procedural information needed to support the automation of (sub)skills. Text-based analytic rubrics deficiencies for skill-mastery can be remedied by adding video-modeling examples with embedded self-explanation prompts, turning text-based analytic rubrics (TR) into so-called ‘video-enhanced analytic rubrics’ (VERS). The current study contrasts two experimental conditions (TR, n = 54; VERS, n = 49) for fostering the growth of learners' mental model of complex skills. This was done using the formative assessment supporting Viewbrics online tool and one control condition with a secondary school's current approach for skills-development (without rubrics nor Viewbrics-formative assessment (n = 50)). We measured learners' mental models at three times during their project work by assessing the richness of their constructed concept maps. A multilevel regression analysis showed that learners in the VERS condition form richer mental models when compared to the control condition for most complex skills.@en

High-quality elaborative peer feedback is a blessing for both learners and teachers. However, learners can experience difficulties in giving high-quality feedback on complex skills using textual analytic rubrics. High-quality elaborative feedback can be strengthened by adding video-modeling examples with embedded self-explanation prompts, turning textual analytic rubrics (TR) into so-called 'video-enhanced analytic rubrics' (VER). This study contrasts two experimental conditions (TR, n = 54; VERs, n = 49) with their version of the anonymized online tool (used to collect the given feedback in 'Tips for improvement and Tops identifying strengths'). Peer feedback quality (concreteness and consistency) was evaluated using Natural Language Processing. As expected, the video-enhanced rubrics condition resulted in a higher quantity of words used and a lower amount of naive wording compared to the textual rubric condition. Contrary to our assumptions, it did not lower the amount of non-constructive wording nor improved the amount of behavioral and process-related feedback. Possibly, the transition from providing more feedback to delivering more accurate behavioral and process-related feedback has not yet been made in the time set for the study.@en