Research in the wild can reveal human behaviors, contexts, and needs around products that are difficult to observe in the lab. Telemetry data from the use of physical products can help facilitate in the wild research, in particular by suggesting hypotheses that can be explored through machine learning models. This paper explores ways for designers without strong data skills to engage with multimodal data to develop a contextual understanding of product use. This study is framed around a lightweight version of a data enhanced design research process where multimodal telemetry data was captured by a GoPro camera attached to a bicycle. This was combined with the video data and conversation with the rider to carry out an exploratory sensemaking process and generate design research questions that could potentially be addressed through data capture, annotation, and machine learning. We identify a range of ways that designers could make use of the data for ideation and developing context through annotating and exploring the data. Participants used data and annotation practices to connect the micro and macro, spot interesting moments, and frame questions around an unfamiliar problem. The work follows the designers’ questions, methods, and explorations, both immediate concerns and speculations about working at larger scales with machine learning models. This points to the possibility of tools that help designers to engage with machine learning, not just for optimization and refinement, but for creative ideation in the early stages of design processes.

Connected devices present new opportunities to advance design through data collection in the wild, similar to the way digital services evolve through analytics. However, it is still unclear how live data transmitted by connected devices informs the design of these products, going beyond performance optimisation to support creative practices. Design can be enriched by data captured by connected devices, from usage logs to environmental sensors, and data about the devices and people around them. Through a series of workshops, this paper contributes industry and academia perspectives on the future of data-driven product design. We highlight HCI challenges, issues and implications, including sensemaking and the generation of design insight. We further challenge current notions of data-driven design and envision ways in which future HCI research can develop ways to work with data in the design process in a connected, rich, human manner.

Chatty Factories is a three-year investment by the Engineering and Physical Sciences Research Council (EPSRC) through its programme for New Industrial Systems. The project explores the transformative potential of placing IoT-enabled data driven systems at the core of design and manufacturing processes. The research focuses on the opportunity to collect data from IoT-enabled sensors embedded in products during real-time use by consumers, explores how that data might be immediately transferred into usable information to inform design, and considers what characteristics of the manufacturing environment might optimise the response to such data. The project also considers implications arising for skills development in the education sector as well as ethics in manufacturing. In this paper we provide a vision for future “Chatty Factories”.

In this work, we develop a vision for entangled ethnography, where constellations of people, artefacts, algorithms and data come together to collectively make sense of the relations between people and objects. This is grounded in New Materialism’s picture of a world understood through entanglement, through resonant constellations, through a multiplicity of unique individual viewpoints and their relationships. These perspectives are especially relevant for design ethnography, in particular for research around smart connected products, which collect data about their environment, the networks they are a part of, and the ways they are used. However, we are concerned about the current trend of many connected systems towards surveillance capitalism, as data is colonised, machinations are hidden, and a narrow definition of value is extracted. There is a key tension that while design, particularly of networked objects, attempts to go beyond human centeredness, the infrastructures that support it are moving towards a less than human perspective in their race to accumulate and dispossess. Our work tries to imagine the situations where participants in networked systems are richly engaged, rather than exploited. We hope for a future where human agency is central to a respectful and acceptable collaborative development of understanding.