P.G.T. Raingeard de la Blétière
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
5 records found
1
A Storytelling Robot for People with Dementia
Enhancing Collaborative Storytelling for People with Dementia through AI-Based Media Generation
Storytelling is a powerful non-pharmacological intervention in care for People with Dementia (PwD), offering the opportunity of self-expression, emotional connection and identity preservation. However, creation of multimedia material to accompany such stories usually requires trained experts and a considerable effort, which limits scalability and personalization. This paper presents a modular system using Artificial Intelligence (AI) which, together with a social robot, aims to transform collaboratively written stories into personalized images and songs. Using a local Large Language Model (LLM) for prompt generation and external diffusion models for media synthesis, the system also supports an interactive feedback system in which users can refine the output. Because of ethical constraints related to data confidentiality, fully autonomous music generation was not implemented, however, the system allows users to generate music manually with any external AI model, automatically detecting the song and playing it in the storytelling interaction flow. The system was tested using simulated stories on consumer-grade hardware, as such confirming the feasibility of media generation in real time, while also respecting confidentiality. The results show strong alignment between the story content and generated visuals, supporting the system's potential to enrich dementia storytelling experiences in an ethical and accessible way.
...
Storytelling is a powerful non-pharmacological intervention in care for People with Dementia (PwD), offering the opportunity of self-expression, emotional connection and identity preservation. However, creation of multimedia material to accompany such stories usually requires trained experts and a considerable effort, which limits scalability and personalization. This paper presents a modular system using Artificial Intelligence (AI) which, together with a social robot, aims to transform collaboratively written stories into personalized images and songs. Using a local Large Language Model (LLM) for prompt generation and external diffusion models for media synthesis, the system also supports an interactive feedback system in which users can refine the output. Because of ethical constraints related to data confidentiality, fully autonomous music generation was not implemented, however, the system allows users to generate music manually with any external AI model, automatically detecting the song and playing it in the storytelling interaction flow. The system was tested using simulated stories on consumer-grade hardware, as such confirming the feasibility of media generation in real time, while also respecting confidentiality. The results show strong alignment between the story content and generated visuals, supporting the system's potential to enrich dementia storytelling experiences in an ethical and accessible way.
A Storytelling Robot for People with Dementia
Keeping people with dementia and family members involved in the storytelling process
Storytelling has many benefits for people with dementia (PwD), such as improved well-being, confidence and communication. However, there is not much research on robots conducting such activities in dementia care. Thus, the goal of this project was to implement a system that can facilitate a storytelling session between PwD and their family members, keeping them involved through the entire process.
The system uses a Large Language Model (LLM), specifically Gemma 3, to generate responses to user's messages based on carefully written prompts. Other strategies used to facilitate this system are separation of the storytelling phases, turn-taking, story personalisation based on participant's wishes and empathetic responses to participant's responses. To test the system, personas based on PwD and their family members were used.
The results showed that the system successfully facilitates a storytelling session, following the different phases and keeping the participants involved in the process. However, some unexpected behaviour was noticed, for example not switching from one phase to the other on time. These findings demonstrate the potential of LLM-based storytelling robots in dementia care, while also underlining the need for further refinement and testing with real users. ...
The system uses a Large Language Model (LLM), specifically Gemma 3, to generate responses to user's messages based on carefully written prompts. Other strategies used to facilitate this system are separation of the storytelling phases, turn-taking, story personalisation based on participant's wishes and empathetic responses to participant's responses. To test the system, personas based on PwD and their family members were used.
The results showed that the system successfully facilitates a storytelling session, following the different phases and keeping the participants involved in the process. However, some unexpected behaviour was noticed, for example not switching from one phase to the other on time. These findings demonstrate the potential of LLM-based storytelling robots in dementia care, while also underlining the need for further refinement and testing with real users. ...
Storytelling has many benefits for people with dementia (PwD), such as improved well-being, confidence and communication. However, there is not much research on robots conducting such activities in dementia care. Thus, the goal of this project was to implement a system that can facilitate a storytelling session between PwD and their family members, keeping them involved through the entire process.
The system uses a Large Language Model (LLM), specifically Gemma 3, to generate responses to user's messages based on carefully written prompts. Other strategies used to facilitate this system are separation of the storytelling phases, turn-taking, story personalisation based on participant's wishes and empathetic responses to participant's responses. To test the system, personas based on PwD and their family members were used.
The results showed that the system successfully facilitates a storytelling session, following the different phases and keeping the participants involved in the process. However, some unexpected behaviour was noticed, for example not switching from one phase to the other on time. These findings demonstrate the potential of LLM-based storytelling robots in dementia care, while also underlining the need for further refinement and testing with real users.
The system uses a Large Language Model (LLM), specifically Gemma 3, to generate responses to user's messages based on carefully written prompts. Other strategies used to facilitate this system are separation of the storytelling phases, turn-taking, story personalisation based on participant's wishes and empathetic responses to participant's responses. To test the system, personas based on PwD and their family members were used.
The results showed that the system successfully facilitates a storytelling session, following the different phases and keeping the participants involved in the process. However, some unexpected behaviour was noticed, for example not switching from one phase to the other on time. These findings demonstrate the potential of LLM-based storytelling robots in dementia care, while also underlining the need for further refinement and testing with real users.
A Storytelling Robot for People with Dementia
Designing a simple interface, suitable for People with Dementia
This project investigates the design process of a straightforward, user-friendly interface that enables cooperative storytelling between a person with dementia and a family member, through the mediation of a social robot. Due to ethical constraints, evaluation was conducted only with a medical specialist and HCI students. Low cognitive load, sequential navigation, and multimodal visuals are given top priority in the final prototype. Working without direct user input and striking a balance between functionality and simplicity were two major obstacles. The project contributes to the development of supportive, dementia-friendly human-robot interfaces and provides a basis for future testing with individuals who have dementia.
...
This project investigates the design process of a straightforward, user-friendly interface that enables cooperative storytelling between a person with dementia and a family member, through the mediation of a social robot. Due to ethical constraints, evaluation was conducted only with a medical specialist and HCI students. Low cognitive load, sequential navigation, and multimodal visuals are given top priority in the final prototype. Working without direct user input and striking a balance between functionality and simplicity were two major obstacles. The project contributes to the development of supportive, dementia-friendly human-robot interfaces and provides a basis for future testing with individuals who have dementia.
A Storytelling Robot for People with Dementia
LLM-Based Persona Simulation to Support Testing of a Storytelling Robot for People with Dementia
Personas are a particularly useful testing tool for storytelling robots for people with dementia (PwD) because they offer an alternative to direct user involvement, which is often limited by recruitment, privacy, and consent-related challenges. The manual creation of realistic personas is often complex and time-intensive, whereas pretrained large language models (LLMs) offer a promising alternative due to their impressive zero-shot and in-context role-playing performance. This study investigates whether commercially available LLMs can accurately simulate personas of PwD for use in testing of a storytelling robot for PwD. To this end, we developed a custom probabilistic system based on a single prompt chain composed of multiple few-shot and zero-shot prompts, along with an independent storage system for custom memory manipulation. The simulated personas underwent repeated assessments using the Mini-Mental State Examination (MMSE), a standardized assessment for evaluating memory, comprehension, and executive function. Results demonstrated statistical similarity to real scores and indicated that LLM-based personas can closely mirror many of the cognitive profiles characteristic of these conditions: early-stage Alzheimer's personas exhibited marked impairments in recent memory, late-stage Alzheimer's personas showed significant global cognitive impairment, and vascular dementia personas displayed relatively preserved memory but reduced executive functioning. These findings indicate that pretrained LLMs possess the capability to simulate accurate personas of PwD to a significant extent.
...
Personas are a particularly useful testing tool for storytelling robots for people with dementia (PwD) because they offer an alternative to direct user involvement, which is often limited by recruitment, privacy, and consent-related challenges. The manual creation of realistic personas is often complex and time-intensive, whereas pretrained large language models (LLMs) offer a promising alternative due to their impressive zero-shot and in-context role-playing performance. This study investigates whether commercially available LLMs can accurately simulate personas of PwD for use in testing of a storytelling robot for PwD. To this end, we developed a custom probabilistic system based on a single prompt chain composed of multiple few-shot and zero-shot prompts, along with an independent storage system for custom memory manipulation. The simulated personas underwent repeated assessments using the Mini-Mental State Examination (MMSE), a standardized assessment for evaluating memory, comprehension, and executive function. Results demonstrated statistical similarity to real scores and indicated that LLM-based personas can closely mirror many of the cognitive profiles characteristic of these conditions: early-stage Alzheimer's personas exhibited marked impairments in recent memory, late-stage Alzheimer's personas showed significant global cognitive impairment, and vascular dementia personas displayed relatively preserved memory but reduced executive functioning. These findings indicate that pretrained LLMs possess the capability to simulate accurate personas of PwD to a significant extent.
A Storytelling Robot for People with Dementia
Evaluating Data Bias and User Enjoyment in the Full System
This paper presents a unified evaluation framework for assessing multimodal storytelling robots used in dementia care. Dementia increasingly affects the quality of life of older adults, and co-creative storytelling with social robots has shown promise in supporting social engagement and emotional well-being. However, existing evaluations often overlook whether generated content fairly reflects the contributions of people with dementia (PwD). To address this, a framework is proposed that jointly evaluates the accuracy of textual, visual, and audio outputs to the original input and their emotional coherence. The method incorporates alignment metrics (AlignScore and BERTScore) for text, image relevance (VQAScore), and audio emotion analysis (valence-arousal), as well as speaker attribution to ensure equitable representation. Results from experimental sessions show that data biases can be quantitatively identified and correlated with user enjoyment indicators. These findings offer a scalable approach to evaluating storytelling robots, ensuring both therapeutic benefit and respect for user identity in sensitive care contexts.
...
This paper presents a unified evaluation framework for assessing multimodal storytelling robots used in dementia care. Dementia increasingly affects the quality of life of older adults, and co-creative storytelling with social robots has shown promise in supporting social engagement and emotional well-being. However, existing evaluations often overlook whether generated content fairly reflects the contributions of people with dementia (PwD). To address this, a framework is proposed that jointly evaluates the accuracy of textual, visual, and audio outputs to the original input and their emotional coherence. The method incorporates alignment metrics (AlignScore and BERTScore) for text, image relevance (VQAScore), and audio emotion analysis (valence-arousal), as well as speaker attribution to ensure equitable representation. Results from experimental sessions show that data biases can be quantitatively identified and correlated with user enjoyment indicators. These findings offer a scalable approach to evaluating storytelling robots, ensuring both therapeutic benefit and respect for user identity in sensitive care contexts.