Marcus Specht
Please Note
39 records found
1
Large Language Models (LLMs) offer scalable, personalized support but risk hindering independent problem-solving through overreliance.
To explore this tension between support and tension, we investigated the interactions of 18 high school students in JELAI, a programming environment with an LLM chatbot, over a 12-week Python course.
We analysed interaction frequency, the impact of instrumental (adaptive: seeking understanding) versus executive (non-adaptive: seeking solutions) query types on exam performance, and the effectiveness of an intervention promoting instrumental strategies. We also explored the feasibility of automatic query classification using a BERT model, given its potential for scalable analysis of student-LLM interactions.
Higher LLM interaction frequency correlated negatively with exam scores, particularly for students with frequent executive queries.
As expected, executive help-seeking strongly predicted lower grades.
Surprisingly, however, frequent instrumental comprehension queries also correlated negatively with performance, likely indicating that high interaction frequency signals persistent difficulty.
A mid-course intervention successfully reduced the proportion of executive queries and fostered more effective interaction workflows, coinciding with a small average grade increase. Automatic classification successfully distinguished instrumental and executive query types (82.5\% accuracy).
This study highlights that LLM integration requires pedagogical strategies and system designs that actively guide students towards instrumental help-seeking and function as learning partners, rather than mere solution providers. ...
Large Language Models (LLMs) offer scalable, personalized support but risk hindering independent problem-solving through overreliance.
To explore this tension between support and tension, we investigated the interactions of 18 high school students in JELAI, a programming environment with an LLM chatbot, over a 12-week Python course.
We analysed interaction frequency, the impact of instrumental (adaptive: seeking understanding) versus executive (non-adaptive: seeking solutions) query types on exam performance, and the effectiveness of an intervention promoting instrumental strategies. We also explored the feasibility of automatic query classification using a BERT model, given its potential for scalable analysis of student-LLM interactions.
Higher LLM interaction frequency correlated negatively with exam scores, particularly for students with frequent executive queries.
As expected, executive help-seeking strongly predicted lower grades.
Surprisingly, however, frequent instrumental comprehension queries also correlated negatively with performance, likely indicating that high interaction frequency signals persistent difficulty.
A mid-course intervention successfully reduced the proportion of executive queries and fostered more effective interaction workflows, coinciding with a small average grade increase. Automatic classification successfully distinguished instrumental and executive query types (82.5\% accuracy).
This study highlights that LLM integration requires pedagogical strategies and system designs that actively guide students towards instrumental help-seeking and function as learning partners, rather than mere solution providers.
Gamification, Interdisciplinarity, and Learning Networks
A Holistic Approach to Professional Development
Computational Thinking Assessment in Higher Education
An Exploration of Computational Thinking Assessment and Integration in Higher Education in the Netherlands
Playlist: A personal sequence of learning activities
Full report on a holistic approach towards designing an educational tool
This report holds the full description of the integrated thesis. The first part including the chapters of ”Fostering a personal sequence of learning activities in Skill Circuits”.
The study uses a design-based research approach, which involves iterative development and research cycles and including students in the evaluation phases. By drawing on educational theories and addressing practical needs identified at TU Delft, the Playlist tool was designed to seamlessly integrate into the existing learning environment. The tool’s functionalities were prioritized based on user feedback provided by Computer Science students and gathered through surveys and focus group sessions.
Key findings indicate that playlists can improve student engagement, motivation, and overall learning experience by supporting students in finding learning activities aligned with their learning preferences and goals. The research highlights the importance of providing students with tools that improve navigating the learning environment and support them in self-directed learning. The Playlist feature not only aids students in organizing their learning activities but also promotes a holistic approach to education by incorporating supporting activities such as reflection and taking breaks.
The integrated thesis hopes to demonstrate the potential and importance of designing tools to personalize learning and support students in achieving their learning goals. It also explains why personalization tools in higher education should be implemented in an iterative manner. ...
This report holds the full description of the integrated thesis. The first part including the chapters of ”Fostering a personal sequence of learning activities in Skill Circuits”.
The study uses a design-based research approach, which involves iterative development and research cycles and including students in the evaluation phases. By drawing on educational theories and addressing practical needs identified at TU Delft, the Playlist tool was designed to seamlessly integrate into the existing learning environment. The tool’s functionalities were prioritized based on user feedback provided by Computer Science students and gathered through surveys and focus group sessions.
Key findings indicate that playlists can improve student engagement, motivation, and overall learning experience by supporting students in finding learning activities aligned with their learning preferences and goals. The research highlights the importance of providing students with tools that improve navigating the learning environment and support them in self-directed learning. The Playlist feature not only aids students in organizing their learning activities but also promotes a holistic approach to education by incorporating supporting activities such as reflection and taking breaks.
The integrated thesis hopes to demonstrate the potential and importance of designing tools to personalize learning and support students in achieving their learning goals. It also explains why personalization tools in higher education should be implemented in an iterative manner.
Fostering a personal sequence of learning activities in Skill Circuits
Part 1 of a holistic approach towards designing an educational tool
This report holds a subset of an integrated thesis involving the development and testing of an educational tool named ”Playlist,” aimed at improving the learning experience of Computer Science & Engineering bachelor students at the TU Delft. The Playlist tool was developed as a feature for the existing tool called Skill Circuits, supporting students in creating personalized sequences of learning activities for their study sessions. The integrated research contributes to the complexities of modern digital learning environments and the challenges students face in navigating vast amounts of educational content. The study uses a design-based research approach, which involves iterative development and research cycles and including students in the evaluation phases. This report focuses on the description and design of these phases. For each phase, it then reports on how the design of the Playlist tool utilized user feedback gathered through surveys and focus group sessions with Computer Science students. Key findings in the integrated thesis indicate that playlists can improve student engagement, motivation, and overall learning experience by supporting students in finding learning activities aligned with their learning preferences and goals. The research highlights the importance of providing students with tools that improve navigating the learning environment and support them in self-directed learning. The Playlist feature not only aids students in organizing their learning activities but also promotes a holistic approach to education by incorporating supporting activities such as reflection and taking breaks. The integrated thesis makes two contributions: it serves as an example of the design and implementation of personalization tools in higher education and provides insights into integrating such tools into an existing educational system. The thesis aims to demonstrate the potential of designing tools in an iterative manner to personalize learning and support students in achieving their learning goals. ...
This report holds a subset of an integrated thesis involving the development and testing of an educational tool named ”Playlist,” aimed at improving the learning experience of Computer Science & Engineering bachelor students at the TU Delft. The Playlist tool was developed as a feature for the existing tool called Skill Circuits, supporting students in creating personalized sequences of learning activities for their study sessions. The integrated research contributes to the complexities of modern digital learning environments and the challenges students face in navigating vast amounts of educational content. The study uses a design-based research approach, which involves iterative development and research cycles and including students in the evaluation phases. This report focuses on the description and design of these phases. For each phase, it then reports on how the design of the Playlist tool utilized user feedback gathered through surveys and focus group sessions with Computer Science students. Key findings in the integrated thesis indicate that playlists can improve student engagement, motivation, and overall learning experience by supporting students in finding learning activities aligned with their learning preferences and goals. The research highlights the importance of providing students with tools that improve navigating the learning environment and support them in self-directed learning. The Playlist feature not only aids students in organizing their learning activities but also promotes a holistic approach to education by incorporating supporting activities such as reflection and taking breaks. The integrated thesis makes two contributions: it serves as an example of the design and implementation of personalization tools in higher education and provides insights into integrating such tools into an existing educational system. The thesis aims to demonstrate the potential of designing tools in an iterative manner to personalize learning and support students in achieving their learning goals.
...
Using survey data for the learners' reasons for enrolment and clickstream data for their behavioural engagement, we applied k-means clustering to identify engagement patterns. Our analysis reveals that the three most important reasons for men and women are career-related, interest-related, and degree-related, in that order. Women are more likely to enrol for career-related reasons than men, while men are more driven by interest in the topic than women. Women also tend to show lower engagement levels compared to men, who are more likely to complete the courses. We found no significant association between reasons for enrolment and engagement for men and women.
These findings highlight the need for gender-sensitive course design strategies to enhance engagement and completion rates. Providing mentorship opportunities, fostering peer interaction platforms, and highlighting role models in the field could also help create a more inclusive learning environment. Future research should explore specific learner challenges and incorporate a more comprehensive engagement model. ...
Using survey data for the learners' reasons for enrolment and clickstream data for their behavioural engagement, we applied k-means clustering to identify engagement patterns. Our analysis reveals that the three most important reasons for men and women are career-related, interest-related, and degree-related, in that order. Women are more likely to enrol for career-related reasons than men, while men are more driven by interest in the topic than women. Women also tend to show lower engagement levels compared to men, who are more likely to complete the courses. We found no significant association between reasons for enrolment and engagement for men and women.
These findings highlight the need for gender-sensitive course design strategies to enhance engagement and completion rates. Providing mentorship opportunities, fostering peer interaction platforms, and highlighting role models in the field could also help create a more inclusive learning environment. Future research should explore specific learner challenges and incorporate a more comprehensive engagement model.
From Course to Online Learning Paths
Improving the Teacher's Experience of an Existing Online Node-link Course Tool
This thesis aimed to help new teachers understand the concept of these node-link structures --- called skill circuits --- and also expand the website with features to help them evaluate their skill circuits, both before and during the course.
A documentation help page was developed, which explained the concept and intended usage of the website and was accompanied by a proof-of-concept skill circuit explaining the same topics. Tools to help evaluate a skill circuit were also developed, which included tools that could be used before the course was held, and tools that used analytics generated by students. One noteworthy tool took inspiration from the Constructive Alignment principle, where a teacher can label parts of their skill circuit with learning objectives to see if there is alignment with their course material.
All of these features were evaluated with a small focus group consisting of teachers, which showed enthusiasm toward the proposed features. ...
This thesis aimed to help new teachers understand the concept of these node-link structures --- called skill circuits --- and also expand the website with features to help them evaluate their skill circuits, both before and during the course.
A documentation help page was developed, which explained the concept and intended usage of the website and was accompanied by a proof-of-concept skill circuit explaining the same topics. Tools to help evaluate a skill circuit were also developed, which included tools that could be used before the course was held, and tools that used analytics generated by students. One noteworthy tool took inspiration from the Constructive Alignment principle, where a teacher can label parts of their skill circuit with learning objectives to see if there is alignment with their course material.
All of these features were evaluated with a small focus group consisting of teachers, which showed enthusiasm toward the proposed features.
Effects of Virtual Reality characteristics on collaboration between learners
Do visualizations of activities have an effect on an individual’s situational awareness when collaborating with others inside Virtual Reality?
This research explores the role of visualization tools in enhancing collaborative learning in virtual reality environments.
Methodology
An experiment was conducted, involving two groups of participants, each tasked with collaboratively solving a complex maze in a virtual reality environment. Both groups experienced two conditions: the experimental condition, where participants had access to visualization tools such as visual cones and highlighting capabilities, and the
controlled condition, where these tools were not available. The participants’ situational awareness were assessed using both the Situation Awareness Rating Technique questionnaire and the Situational Awareness Linked Indicators Adapted to Novel Tasks framework.Results
The Situational Awareness Linked Indicators Adapted to Novel Tasks scores were higher for the experimental session for Group 2, while being lower for Group 1. However, the Situation Awareness Rating Technique scores were generally lowerin the experimental condition for both groups.
Conclusion
These results demonstrate the potential virtual reality has as a platform for collaborative learning and highlight the need for further research into the design of effective visualization tools and the measurement of situational awareness in virtual reality. Furthermore, this research opens up new possibilities into the design of virtual reality environments for collaborative learning and the potential of virtual reality as a tool for enhancing collaborative learning. ...
This research explores the role of visualization tools in enhancing collaborative learning in virtual reality environments.
Methodology
An experiment was conducted, involving two groups of participants, each tasked with collaboratively solving a complex maze in a virtual reality environment. Both groups experienced two conditions: the experimental condition, where participants had access to visualization tools such as visual cones and highlighting capabilities, and the
controlled condition, where these tools were not available. The participants’ situational awareness were assessed using both the Situation Awareness Rating Technique questionnaire and the Situational Awareness Linked Indicators Adapted to Novel Tasks framework.Results
The Situational Awareness Linked Indicators Adapted to Novel Tasks scores were higher for the experimental session for Group 2, while being lower for Group 1. However, the Situation Awareness Rating Technique scores were generally lowerin the experimental condition for both groups.
Conclusion
These results demonstrate the potential virtual reality has as a platform for collaborative learning and highlight the need for further research into the design of effective visualization tools and the measurement of situational awareness in virtual reality. Furthermore, this research opens up new possibilities into the design of virtual reality environments for collaborative learning and the potential of virtual reality as a tool for enhancing collaborative learning.
"A-Maze-ing Arguments in Virtual Reality"
An Analysis of the Connection between Shared Situational Awareness and Social Modes of Co-Construction in Virtual Reality
Virtual Reality is an increasingly hot topic these days. As a user, you can play more and more high-quality games and strides are being made to use this medium for education and new kinds of workspaces. Previous research on Virtual Reality shows that it could improve teamwork by letting users share information easier. Users in Virtual Reality can communicate in ways exclusive to the medium, like highlighting exact locations, so this paper analyzed how effective verbal communication is in this situation.
Methods
To analyze how Virtual Reality influences group interaction, an experiment was set up that ascertained in the presence and absence of visualizations of user actions whether a group being more aware influences how they communicate in Virtual Reality. To approach this, this paper used a maze in Virtual Reality where participants need to communicate hints of their color.
Results
The data encoding of the experiment showed that when a group's Shared Situational Awareness decreased between sessions, their level of Social Modes of Co-Construction did as well to a similar degree. When the awareness remained around the same level, only decreasing slightly, the level of social modes remained as well, going slightly up or down depending on the reading of the data.
Conclusions
From the research experiment, this paper observes there is a positive correlation between group Shared Situational Awareness and their Social Modes of Co-Construction.
Because of limited time and data sources, it is suggested that this experiment is reproduced over a longer period on a larger scale to support these findings. ...
Virtual Reality is an increasingly hot topic these days. As a user, you can play more and more high-quality games and strides are being made to use this medium for education and new kinds of workspaces. Previous research on Virtual Reality shows that it could improve teamwork by letting users share information easier. Users in Virtual Reality can communicate in ways exclusive to the medium, like highlighting exact locations, so this paper analyzed how effective verbal communication is in this situation.
Methods
To analyze how Virtual Reality influences group interaction, an experiment was set up that ascertained in the presence and absence of visualizations of user actions whether a group being more aware influences how they communicate in Virtual Reality. To approach this, this paper used a maze in Virtual Reality where participants need to communicate hints of their color.
Results
The data encoding of the experiment showed that when a group's Shared Situational Awareness decreased between sessions, their level of Social Modes of Co-Construction did as well to a similar degree. When the awareness remained around the same level, only decreasing slightly, the level of social modes remained as well, going slightly up or down depending on the reading of the data.
Conclusions
From the research experiment, this paper observes there is a positive correlation between group Shared Situational Awareness and their Social Modes of Co-Construction.
Because of limited time and data sources, it is suggested that this experiment is reproduced over a longer period on a larger scale to support these findings.
Methods: A maze, specifically designed for collaboration, has been chosen to use for this study and visualization tools such as laser pointing and vision cones were made available for certain sessions.
Results.: The findings from this maze experiment did not provide conclusive evidence regarding the impact of visualization tools on social modes of co-construction, mainly due to the limited comparison material available. An interesting finding however, is that sessions with visualization tools tend to have more distracted participants compared to the absence of them.
Conclusion: Further research is needed to examine the relationship between visualization tools and social modes of co-construction, as well as to explore whether the observed distractions are specific to the participants in this study or representative of a bigger spectrum. By addressing these aspects, we can gain a better understanding of the role of visualization tools in collaborative learning and uncover strategies to mitigate any potential distractions they may introduce. ...
Methods: A maze, specifically designed for collaboration, has been chosen to use for this study and visualization tools such as laser pointing and vision cones were made available for certain sessions.
Results.: The findings from this maze experiment did not provide conclusive evidence regarding the impact of visualization tools on social modes of co-construction, mainly due to the limited comparison material available. An interesting finding however, is that sessions with visualization tools tend to have more distracted participants compared to the absence of them.
Conclusion: Further research is needed to examine the relationship between visualization tools and social modes of co-construction, as well as to explore whether the observed distractions are specific to the participants in this study or representative of a bigger spectrum. By addressing these aspects, we can gain a better understanding of the role of visualization tools in collaborative learning and uncover strategies to mitigate any potential distractions they may introduce.
Learning is a core part of how we grow as humans. Over the last few years it has increasingly been shown that learning in groups tends to be more effective than learning individually. This research aims to show a link between an individual’s situational awareness and the degree of collaboration.
Method
An experiment was conducted where groups of participants navigated a maze. The maze was designed to require collaboration to reach the end. Data were gathered about both the situational awareness of the participants and their levels of Social Modes of Co-Construction.
Conclusion
The current data suggests no strong correlation between situational awareness and levels of Social Modes of Co-Construction. However, more data will need to be gathered to increase the reliability of this conclusion. ...
Learning is a core part of how we grow as humans. Over the last few years it has increasingly been shown that learning in groups tends to be more effective than learning individually. This research aims to show a link between an individual’s situational awareness and the degree of collaboration.
Method
An experiment was conducted where groups of participants navigated a maze. The maze was designed to require collaboration to reach the end. Data were gathered about both the situational awareness of the participants and their levels of Social Modes of Co-Construction.
Conclusion
The current data suggests no strong correlation between situational awareness and levels of Social Modes of Co-Construction. However, more data will need to be gathered to increase the reliability of this conclusion.
Instructional Designs of Machine Learning
A research into the instructional designs used in introductory courses of machine learning in computer science bachelors