This graduation thesis explored how large language models (LLMs) can enable new forms of tangible storytelling. Through hands-on prototyping and testing, I created a series of interactive prototypes of AI-enabled tangible storytelling systems and gathered users' experience when telling stories with it. I developed a conceptual framework to guide and evaluate the design of these storytelling systems. Synthesizing the findings from user tests and the conceptual framework, I developed a refined demo as the final outcome. The demo provided valuable insights on design opportunities and challenges, sparking multiple future directions. Together, these insights form a toolkit for designers who want to combine AI and tangible interactions in storytelling. LLM-driven artifacts have the power to enable creative, embodied, and social narratives, opening doors to new ways of defining and developing stories.

There exist many ways to use an object outside of its designed function. Chairs were made for sitting; however, people routinely use them to reach things from high places. The invention of the micro-plane (a cooking appliance that is used to grate food) was discovered because of the unintended use of a Wood File. Products everywhere exist to service one function but can serve other purposes they were not designed for. This is not the result of bad design, but of the underlying communication that occurs between a user and a product every time they are interacting. This communication is due to a lot of factors, one of which being sensory communication. A user's interpretation of the sensory aspects of a product influence how they believe said product can and should be used. This misalignment is due to a lack of consideration for sensory experiences and the communication a user is having with a product. This graduation project attempts to develop a method that can be used to generate product ideas that have a more coherent and aligned form, experience, and function because they will lead to innovative designs with stronger emotional connections between the user and the product.

The method that ends up being developed uses the areas of multi-sensory design and prototyping to create a process that allows designers to capitalize on unintended use within products to discover new product directions. The Multi-sensory Unintended Object Method (MsUOM) focuses on user interviews to collect multi-sensory information surrounding two designed objects. This multi-sensory information will be used to combine those designed objects into one prototype. With this prototype, designers will see how users interpret its functionality based on the form and their interaction with it. This leaves the designer with the newly designed object with a form, multiple perceived functions, and a multi-sensory understanding of the user experience. The method gives designers the tools to enter the traditional design process equipped with the knowledge to develop a product with an intuitive experience inspired by the prototype developed through the Multi-sensory Unintended Object Method (MsUOM).

Through a user focused evaluation, the Multi-sensory Unintended Object Method is in a state where it is viable to use and will lead designers to discovering product spaces where their designed object exists with innovative functionality possibilities and strong sensory focuses that inform a consistent and unique form and experience.

The method bodes well for the future of designed objects, as it will generate product ideas that have form, function, and experience aligned and embedded in their development; while also relaying to designers the importance of the user relationship with a product without sacrificing innovation within design.

As AI technology continues to advance, there's a growing need to integrate it into UX design. However, AI's unique characteristics do not seamlessly align with current design tools and mastering the technical aspects for designers is a significant challenge. The project goal is to develop a tool based on a developed semi-formal representation for Human-AI (HAI) interactions, which uses a set of communicative acts1 to specify the communicated information between users and AI models as exchanges of messages.

The project followed an iterative prototyping method across 4 phases. The Pre-Phase aimed at testing communicative acts with design students using a use case ("CV-Screening") and paper materials. At the same time, it also expected to get insights on the data structure and develop the specific design considerations based on those for the Model-Informed Prototyping (MIP),

Phase 1 explored effective workflows of the digital prototype to present communicative acts by following the design considerations from the Pre-Phase and using the low-fidelity digital prototype in Figma. The use case in this phase was the same as that in the Pre-Phase.

In Phase 2, based on insights from the last two phases, there was a high-fidelity prototype in Figma which was inspired by the user journey map. It was used to assess if the design output achieved the design goals and considerations, and it helped the final test materials work better.

The Final Phase utilizes the refined digital prototype for the final test which had the same goals as that in Phase 2, providing important insights for future development.

The final output of the project is a partial prototype of a digital tool designed to facilitate the early stages of human-AI interaction design. Grounded in the principles of communicative acts and human-centered design, this tool assists designers during the Ideation stage of Design process. It achieves this by visualizing the roles, data, and information involved in the process of information exchange during Human-AI Interactions. The goal is to enhance efficiency and ease in designing these interactions.