Title
Improving Adaptive Learning Models Using Prosodic Speech Features
Author
Wilschut, Thomas (University Medical Center Groningen)
Sense, Florian (LLC)
Scharenborg, O.E. (TU Delft Multimedia Computing) ![ORCID 0000-0003-0693-8852 ORCID 0000-0003-0693-8852](/sites/all/themes/tud_repo3/img/icons/orcid_16x16.png)
van Rijn, Hedderik (University Medical Center Groningen)
Contributor
Wang, Ning (editor)
Rebolledo-Mendez, Genaro (editor)
Matsuda, Noboru (editor)
Santos, Olga C. (editor)
Dimitrova, Vania (editor)
Date
2023
Abstract
Cognitive models of memory retrieval aim to describe human learning and forgetting over time. Such models have been successfully applied in digital systems that aid in memorizing information by adapting to the needs of individual learners. The memory models used in these systems typically measure the accuracy and latency of typed retrieval attempts. However, recent advances in speech technology have led to the development of learning systems that allow for spoken inputs. Here, we explore the possibility of improving a cognitive model of memory retrieval by using information present in speech signals during spoken retrieval attempts. We asked 44 participants to study vocabulary items by spoken rehearsal, and automatically extracted high-level prosodic speech features—patterns of stress and intonation—such as pitch dynamics, speaking speed and intensity from over 7,000 utterances. We demonstrate that some prosodic speech features are associated with accuracy and response latency for retrieval attempts, and that speech feature informed memory models make better predictions of future performance relative to models that only use accuracy and response latency. Our results have theoretical relevance, as they show how memory strength is reflected in a specific speech signature. They also have important practical implications as they contribute to the development of memory models for spoken retrieval that have numerous real-world applications.
Subject
Adaptive Learning
Automatic Speech Recognition
Cognitive Modeling
Intensity
Machine learning
Pitch
Speaking Speed
Speech prosody
To reference this document use:
http://resolver.tudelft.nl/uuid:977dd986-58c9-4df9-9336-7bcb31a1b58a
DOI
https://doi.org/10.1007/978-3-031-36272-9_21
Publisher
Springer
Embargo date
2024-01-01
ISBN
9783031362712
Source
Artificial Intelligence in Education - 24th International Conference, AIED 2023, Proceedings
Event
24th International Conference on Artificial Intelligence in Education, AIED 2023, 2023-07-03 → 2023-07-07, Tokyo, Japan
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 13916 LNAI
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2023 Thomas Wilschut, Florian Sense, O.E. Scharenborg, Hedderik van Rijn