Exposure to research is an important component of undergraduate university education, cultivating critical thinking, problem-solving, and preparation for advanced study. However, providing individual research experiences for large cohorts of undergraduate students poses significant logistical challenges. This paper demonstrates how an undergraduate research experience can be achieved at scale for a large computer science program. Our approach integrates individual research projects into the undergraduate computer science curriculum for up to almost 400 students within a single 10-week course. We describe three key features of our approach: (1) a matching algorithm that assigns students to research projects based on their preferences, (2) peer-group collaboration, and (3) a distributed supervision and assessment model to guide students through key research activities that include reformulating research questions, designing experiments/user studies, and presenting research. Results and feedback indicate that both students and supervisors are satisfied, demonstrating the feasibility and effectiveness of this scalable approach for integrating research experiences into large undergraduate computer science programs.

Engaging students with effective learning materials continues to be a significant challenge in programming education. Analogies are commonly used to simplify complex topics, enabling learners to relate unfamiliar concepts to familiar ones. Additionally, visual representations of these analogies can enhance engagement and improve the overall learning experience. This work presents a prototype of a novel AI tool that generates analogy-based explanations and corresponding video animations for programming education. The tool leverages Large Language Models (LLMs) for analogy generation and a structured animation workflow for visualization. This poster invites discussion on the effectiveness of AI-generated educational content and its implications for programming education.

Learner behaviours often provide critical clues about learners' cognitive processes. However, the capacity of human intelligence to comprehend and intervene in learners' cognitive processes is often constrained by the subjective nature of human evaluation and the challenges of maintaining consistency and scalability. The recent widespread AI technology has been applied to learning analytics (LA), aiming at a more accurate, consistent and scalable understanding of learning to compensate for challenges that human intelligence faces. However, machine intelligence has been criticized for lacking contextual understanding and difficulties dealing with complex human emotions and social cues. In this work, we aim to understand learners' internal cognitive processes based on the external behavioural cues of learners in a digital reading context, using a hybrid intelligence (HI) approach, bridging human and machine intelligence. Based on the behavioural frameworks and the insights from human experts, we scope specific behavioural cues that are known to be relevant to learners' attention regulation, which is highly relevant for learners' cognitive processes. We utilize the public WEDAR dataset with 30 subjects' video data, behaviour annotation and pre–post tests on multiple choice and summarization tasks. We apply the explainable AI (XAI) approach to train the machine learning model so that human evaluators can also understand which behavioural features were essential for predicting the usage of the cognitive processes (ie, higher-order thinking skills [HOTS] and lower-order thinking skills [LOTS]) of learners, providing insights for the next-round feature engineering and intervention design. The result indicates that the dominant use of attention regulation behaviours is a reliable indicator of low use of LOTS with 79.33% prediction accuracy, while reading speed is a valuable indicator for predicting the overall usage of HOTS and LOTS, ranging from 60.66% to 78.66% accuracy, highly surpassing random guess of 33.33%. Our study demonstrates how various combinations of behavioural features supported by HI can inform learners' cognitive processes accurately and interpretably, integrating human and machine intelligence. Practitioner notes What is already known about this topic Human attention is a cognitive process that allows us to choose and concentrate on relevant information, which leads to successful learning. In affective computing, certain behavioural cues (eg, attention regulation behaviours) are used to indicate learners' attentional states during learning. What this paper adds Attention regulation behaviours during digital reading can work as predictors of different levels of cognitive processes (ie, the utilization of higher-order thinking skills [HOTS] and lower-order thinking skills [LOTS]), leveraged by computer vision and machine learning. By developing an explainable AI model, we can predict learners' cognitive processes, which often cannot be achieved by human observations, while understanding behavioural components that lead to such machine decisions is critical. It can provide valuable machine-driven insights into the relationship between humans' external and internal states in learning. Based on the frameworks spanning cognitive AI, psychology and education, expert knowledge can contribute to initial feature selection and engineering for the hybrid intelligence (HI) model development and next-round intervention design. Implications for practice and/or policy Human and machine intelligence form an iterative cycle to build a HI to understand and intervene in learners' cognitive processes in digital reading, balancing each other's strengths and weaknesses in decision-making. It can eventually inform automated feedback loops in widespread e-learning, a new education norm since the COVID-19 pandemic. Our framework also has the potential to be extended to other scenarios with digital reading, providing concrete examples of where human intelligence and machine intelligence can contribute to building a HI. It represents more systematic supports that apply to real-life practices.

With the fast integration of Machine Learning (ML) across industries, effective pedagogical strategies are essential for teaching this complex and evolving field. Machine Learning is now widely integrated into various university programs and introduced at earlier educational stages, including high school and secondary school. However, ML pedagogy lacks standardized teaching methods compared to other science-related subjects, which have established norms for topic introduction, teaching tools, and assessment methods. Inspired by other fields, this research explores the use of interactive visualizations in teaching ML topics, more specifically in teaching Gradient Descent (GD) and Principal Component Analysis (PCA). The target population consists of Computer Science and Engineering Bachelor students who have not yet followed any Machine Learning courses but have foundational knowledge in calculus, linear algebra, and statistics. The evaluation measures knowledge gained and student motivation, compared to a static version of the materials. Results show a significant positive effect in knowledge related to PCA with interactive visualizations, but no differences in knowledge gain for GD or in learning motivation for either topic. With these results, we contribute to the body of evidence-based teaching methods in Machine Learning and identify further research needed to generalize the effect of interactive visualizations as a teaching method for teaching ML basic concepts.

Different individual features of the learner data often work as essential indicators of learning and intervention needs. This work exploits the personas in the design thinking process as the theoretical basis to analyze and cluster learners’ learning behavior patterns as groups. To adapt to the learning practice, we develop data-driven personas by clustering learners’ features based on factual learning outcomes (i.e., knowledge gain, perceived learning experience, perceived social presence) based on unsupervised learning, a more accessible and objective intervention design strategy for e-reading practices. Using the Chi-square test, we quantitatively evaluate different clusters driven by various unsupervised learning methods on the multimodal SKEP dataset. Furthermore, for a more practical real-life application, we achieved automatic persona prediction based on the attention regulation behaviors of learners. The subject-independent evaluation results indicate the best classification accuracy of 70% for the four-level classification task, differentiating three personas of learners with needs and another without feedback needs. It also shows that time-based sampling on both independent and cumulative learner behaviors works as robust predictors of learner personas, achieving a stable accuracy range of 65%-70% throughout the e-reading with the SVM classifier. Our work inspires the design of a real-time feedback loop for e-learning based on conversational agents.

This study is built upon a behavior-based framework for real-time attention evaluation of higher education learners in e-reading. Significant challenges in AI model developments for learning analytics have been 1) defining valid indicators and 2) connecting the analytics results to interventions, balancing the generalization and personalization needs. To address this, we utilized a public multimodal WEDAR dataset and trained a neural network model based on real-time features of learners, aiming at predicting learners’ moment-to-moment distractions. Real-time features for model training include 30 learners’ attention regulation behaviors annotated every second, reaction times to blur stimuli, and page numbers indicating various reading phases. Our preliminary model based on a neural network has achieved 66.26% accuracy in predicting self-reported distractions. Based on the model, we suggest a framework of a Behavior-based Feedback Loop for Attentive e-reading (BFLAe). It has text blur as feedback, a mechanism responsive to learners’ distractions that also works as data for next-round feedback. The general feedback implementation rules are established on a statistical analysis conducted on all learners. In addition, we propose a strategy for personalizing feedback using a quartile analysis of individual data, promoting learner-specific feedback. Our framework addresses the high demand for an automated e-learning assistant with non-intrusive data collection based on real-world settings and intuitive feedback provision. The feedback system aims to help learners with longer attention spans and less frequent distractions, leading to more engaging e-reading.

We present ReproducedPapers.org : an open online repository for teaching and structuring machine learning reproducibility. We evaluate doing a reproduction project among students and the added value of an online reproduction repository among AI researchers. We use anonymous self-assessment surveys and obtained 144 responses. Results suggest that students who do a reproduction project place more value on scientific reproductions and become more critical thinkers. Students and AI researchers agree that our online reproduction repository is valuable.

Many universities digitize exams or the process of grading the exams. This potentially allows for faster grading, is less labor intensive and less error-prone. But are the grades produced by online grading consistent with how we grade on paper? In this paper we present preliminary results of the comparison between scores given by grading online and grading on paper.