Research in the wild can reveal human behaviors, contexts, and needs around products that are difficult to observe in the lab. Telemetry data from the use of physical products can help facilitate in the wild research, in particular by suggesting hypotheses that can be explored through machine learning models. This paper explores ways for designers without strong data skills to engage with multimodal data to develop a contextual understanding of product use. This study is framed around a lightweight version of a data enhanced design research process where multimodal telemetry data was captured by a GoPro camera attached to a bicycle. This was combined with the video data and conversation with the rider to carry out an exploratory sensemaking process and generate design research questions that could potentially be addressed through data capture, annotation, and machine learning. We identify a range of ways that designers could make use of the data for ideation and developing context through annotating and exploring the data. Participants used data and annotation practices to connect the micro and macro, spot interesting moments, and frame questions around an unfamiliar problem. The work follows the designers’ questions, methods, and explorations, both immediate concerns and speculations about working at larger scales with machine learning models. This points to the possibility of tools that help designers to engage with machine learning, not just for optimization and refinement, but for creative ideation in the early stages of design processes.@en

In this work, we develop a vision for entangled ethnography, where constellations of people, artefacts, algorithms and data come together to collectively make sense of the relations between people and objects. This is grounded in New Materialism’s picture of a world understood through entanglement, through resonant constellations, through a multiplicity of unique individual viewpoints and their relationships. These perspectives are especially relevant for design ethnography, in particular for research around smart connected products, which collect data about their environment, the networks they are a part of, and the ways they are used. However, we are concerned about the current trend of many connected systems towards surveillance capitalism, as data is colonised, machinations are hidden, and a narrow definition of value is extracted. There is a key tension that while design, particularly of networked objects, attempts to go beyond human centeredness, the infrastructures that support it are moving towards a less than human perspective in their race to accumulate and dispossess. Our work tries to imagine the situations where participants in networked systems are richly engaged, rather than exploited. We hope for a future where human agency is central to a respectful and acceptable collaborative development of understanding.@en

Chatty Factories is a three-year investment by the Engineering and Physical Sciences Research Council (EPSRC) through its programme for New Industrial Systems. The project explores the transformative potential of placing IoT-enabled data driven systems at the core of design and manufacturing processes. The research focuses on the opportunity to collect data from IoT-enabled sensors embedded in products during real-time use by consumers, explores how that data might be immediately transferred into usable information to inform design, and considers what characteristics of the manufacturing environment might optimise the response to such data. The project also considers implications arising for skills development in the education sector as well as ethics in manufacturing. In this paper we provide a vision for future “Chatty Factories”.@en

Chatty Factories is a three-year investment by the Engineering and Physical Sciences Research Council (EPSRC) through its programme for New Industrial Systems. The project explores the transformative potential of placing IoT-enabled data driven systems at the core of design and manufacturing processes. The research focuses on the opportunity to collect data from IoT-enabled sensors embedded in products during real-time use by consumers, explores how that data might be immediately transferred into usable information to inform design, and considers what characteristics of the manufacturing environment might optimise the response to such data. The project also considers implications arising for skills development in the education sector as well as ethics in manufacturing. In this paper we provide a vision for future “Chatty Factories”.@en

Connected products present new opportunities for conducting in-the-wild design research, where live data is transmitted by devices about their use and function. However, industry data gathering practices have raised public concerns around privacy and security. Thus, we need to account for users’ perspectives on how data is gathered and used. A technology provocation was used to spark discussion on the acceptability of physical devices collecting information for design research. Attitudes ranged from extreme unease to lack of concern, with varying beliefs about the trustworthiness and capability of researchers and companies. A range of real and speculative contexts prompted participants to examine value trade-offs between themselves and corporations, privacy and ethical issues, agency, and informed consent. Based on this we set out implications for carrying out data-driven design in order to unlock potential value while respecting user privacy and time.@en

Connected devices present new opportunities to advance design through data collection in the wild, similar to the way digital services evolve through analytics. However, it is still unclear how live data transmitted by connected devices informs the design of these products, going beyond performance optimisation to support creative practices. Design can be enriched by data captured by connected devices, from usage logs to environmental sensors, and data about the devices and people around them. Through a series of workshops, this paper contributes industry and academia perspectives on the future of data-driven product design. We highlight HCI challenges, issues and implications, including sensemaking and the generation of design insight. We further challenge current notions of data-driven design and envision ways in which future HCI research can develop ways to work with data in the design process in a connected, rich, human manner.@en

A key challenge in carrying out product design research is obtaining rich contextual information about use in the wild. We present a method that algorithmically mediates between participants, researchers, and objects in order to enable real-time collaborative sensemaking. It facilitates contextual inquiry, revealing behaviours and motivations that frame product use in the wild. In particular, we are interested in developing a practice of use driven design, where products become research tools that generate design insights grounded in user experiences. The value of this method was explored through the deployment of a collection of Bluetooth speakers that capture and stream live data to remote but co-present researchers about their movementand operation. Researchers monitored a visualisation of the real-time data to build up a picture of how the speakers were being used, responding to moments of activity within the data, initiating text conversations and prompting participants to capture photos and video. Based on the findings of this explorative study, we discuss the value of this method, how it compares to contemporary research practices, and the potential of machine learning to scale it up for use within industrial contexts. As greater agency is given to both objects and algorithms, we explore ways to empower ethnographers and participants to actively collaborate within remote real-time research@en

IoT products are embedded with sensors that transmit live data about their use and environment. A key challenge for designers is to gather useful insights from this data in order to accelerate product research, which can be time consuming and labour intensive. Through the Chatty Products dashboard we aim to explore how virtual representations of IoT products and their sensor data, also known as digital twins, can support insight gathering. This demo will present a series of Bluetooth IoT speakers, which are connected to the Chatty Products dashboard, a data exploration and visualisation research tool containing supervisory digital twinsof the speakers. The project aims to visualise live data as it relates to the physical product in the wild, enabling contextual inquiry and supporting data exploration. The demo will promote a dialogue around how digital twins can be used to gather design insights based on live data.@en