Stepping Into Stories: Envisioning a Generative AI Pipeline to Create Story-based VR Reading Environments
A. Vitali (TU Delft - Knowledge and Intelligence Design)
C. Schneegass (TU Delft - Perceptual Intelligence)
Tilman Dingler (TU Delft - Knowledge and Intelligence Design)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Where you read matters—so what if you could read literally inside your book? Reading in Virtual Reality (VR) has been shown to support deep immersion and narrative engagement. We argue that Generative Artificial Intelligence (GenAI) can significantly expand what is currently possible in VR reading by dynamically producing story-driven environments from text. In this vision paper, we present a pipeline that combines recent advances in language, audio, and 3D scene generation to allow automatically augmenting fiction reading with environmental backdrops. As the reader progresses, these AI-generated environments transition in real time, acting as cognitive props for visual imagery. We also describe a user evaluation, discuss limitations, and address implications and open questions raised by the proposed approach.
help_outline