Enhancing the Audience Experience for VR and AR Theatre with AI-generated Subtitles
Irene Viola (Centrum Wiskunde & Informatica (CWI))
Moonisa Ahsan (Centrum Wiskunde & Informatica (CWI))
Olga Chatzifoti (National and Capodistrian University of Athens, Maggioli Group)
Atanas Yonkov (Centrum Wiskunde & Informatica (CWI))
Eleni Oikonomou (Athens-Epidaurus Festival)
Ioannis Radin (Athens-Epidaurus Festival)
Paweł Maka (Maastricht University)
Abderrahmane Issam (Maastricht University)
Pablo Cesar (TU Delft - Multimedia Computing, Centrum Wiskunde & Informatica (CWI))
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Recent technological developments on AI and immersive media are transforming the artistic landscape, providing novel mechanisms for artists and audiences. Following a human-centric approach, together with a theatre company in Greece, this paper investigates how subtitle placement affects user experience and cognitive load in a live theatre performance enhanced by AR glasses. To do so, we design and develop a system for displaying subtitles in VR and AR. We evaluated the system in two conditions (N = 19;N = 12), both in a controlled environment (VR) and an actual theatre (AR). In the latter, we integrate AI solutions to provide automatic captioning and translation in real time, and VFX to further augment the experience. Our quantitative and qualitative results showed no difference between subtitle placements in terms of cognitive load and user experience, with users equally liking the two proposed approaches. Results also highlighted the perceived usefulness of AR to enhance theatre performances, indicating new paths for wider accessibility and further immersion.
help_outline