Prompting Realities
Exploring the Potentials of Prompting for Tangible Artifacts
Mahan Mehrvarz (TU Delft - Human Technology Relations)
Dave Murray-Rust (TU Delft - Human Technology Relations)
Himanshu Verma (TU Delft - Human-Centred Artificial Intelligence)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Designing meaningful tangible and embodied interactions remains challenging due to their situated nature, complex user needs, and the limited programming skills of many users as well as designers. We developed an interaction model where users and LLMs co-perform tangible actions through prompt engineering beyond deterministic logic of commercial smart systems. In this model AI systems interpret natural language descriptions of environmental context, internalize technical functionalities and spatial cues, and translate these into tangible actions. We encapsulated the interaction model within a LLM-enabled tangible artifact as a HCI provotype and conducted an initial exploratory study around it. Our preliminary findings point to opportunities in refinement and reappropriation of such systems over the use period as well as challenges in adapting deictic spatial references.
help_outline