The growing sophistication of Large Language Models allows conversational agents (CAs) to engage users in increasingly personalized and targeted conversations. While users may vary in their receptiveness to CA persuasion, stylistic elements and agent personalities can be adjusted on the fly. Combined with image generation models that create context-specific realistic visuals, CAs have the potential to influence user behavior and decision making. We investigate the effects of linguistic and visual elements used by CAs on user perception and decision making in a charitable donation context with an online experiment (n=344). We find that while CA attitude influenced trust, it did not affect donation behavior. Visual primes played no role in shaping trust, though their absence resulted in higher donations and situational empathy. Perceptions of competence and situational empathy were potential predictors of donation amounts. We discuss the complex interplay of user and CA characteristics and the fine line between benign behavior signaling and manipulation.

Personas are hypothetical representations of real-world people used as storytelling tools to help designers identify the goals, constraints, and scenarios of particular user groups. A well-constructed persona can provide enough detail to trigger recognition and empathy while leaving room for varying interpretations of users. While a traditional persona is a static representation of a potential user group, a chatbot representation of a persona is dynamic, in that it allows designers to “converse with” the representation. Such representations are further augmented by the use of large language models (LLMs), displaying more human-like characteristics such as emotions, priorities, and values. In this paper, we introduce the term “Synthetic User” to describe such representations of personas that are informed by traditional data and augmented by synthetic data. We study the effect of one example of such a Synthetic User – embodied as a chatbot – on the designers’ process, outcome, and their perception of the persona using a between-subjects study comparing it to a traditional persona summary. While designers showed comparable diversity in the ideas that emerged from both conditions, we find in the Synthetic User condition a greater variation in how designers perceive the persona’s attributes. We also find that the Synthetic User allows novel interactions such as seeking feedback and testing assumptions. We make suggestions for balancing consistency and variation in Synthetic User performance and propose guidelines for future development.

Data Humanism has gained prominence in personal visualization and Personal Informatics, advocating for a subjective and slow approach to engage with personal data. Collaborative sensemaking has great potential for aiding the understanding of personal data, yet little is known about addressing requirements of structure and coordination when integrating Data Humanism into collaborative visualization. In this paper, we propose design principles for creating both subjective and effective collaborative visualizations, while coordinating the slow sensemaking process and promoting data awareness and communication. We operationalize these principles into a personal visualization toolkit, which we evaluate with an observational study involving 16 university students (8 pairs) analyzing each other's screen-time data. Our findings reveal that implementing the proposed design principles: (1) facilitated data comparison from shared subjective perspectives, (2) helped coordinate sensemaking while allowing time for understanding personal data, and (3) helped the contextualization of data patterns, in turn aiding self-reflection.

Reflection plays a vital role in the development of designers, enabling them to evaluate their experiences, enhance their learning, and foster professional growth. This research analyzed reflections of 56 design students, as part of graded coursework, using content and dictionary-based approaches (LIWC). Building on an existing model of reflection with eight components (experience, belief, difficulty, perspective, feeling, learning, intention, and descriptive) we identify, using descriptive statistics, the linguistic features associated with each component and correlate these to grades achieved. We distinguish two types of reflections associated with higher grades: those emphasizing personal experiences that we term holistic narrators, and those that focus on critical self-evaluation that we term in-depth explorers. Our results provide insights for design educators, guiding interventions to enhance critical thinking and self-reflection among design students. They also inform the development of automated tools to assess and enhance reflective practice in educational and design settings.

With the most advanced natural language processing and artificial intelligence approaches, effective summarization of long and multi-topic documents - such as academic papers - for readers from different domains still remains a challenge. To address this, we introduce ConceptEVA, a mixed-initiative approach to generate, evaluate, and customize summaries for long and multi-topic documents. ConceptEVA incorporates a custom multi-task longformer encoder decoder to summarize longer documents. Interactive visualizations of document concepts as a network reflecting both semantic relatedness and co-occurrence help users focus on concepts of interest. The user can select these concepts and automatically update the summary to emphasize them. We present two iterations of ConceptEVA evaluated through an expert review and a within-subjects study. We find that participants' satisfaction with customized summaries through ConceptEVA is higher than their own manually-generated summary, while incorporating critique into the summaries proved challenging. Based on our findings, we make recommendations for designing summarization systems incorporating mixed-initiative interactions.

Design thinking concepts, such as storytelling, framing, and co-evolution, have been established from close readings of design activity. The increase in easy-to-use computational methodologies provides an opportunity to validate these concepts more widely. Among these concepts, storytelling is already operationalised through various computational approaches. In this chapter, we create one corpus of design activity data from the four shared-data Design Thinking Research Symposium (DTRS) workshops and use Linguistic Inquiry and Word Count (LIWC) in attempting to automatically detect components of stories. However, the conversational nature of the data indicates that further development in methodology is needed. The contribution of the chapter lies both in outlining how an automated method for identifying stories could work and showing how the DTRS corpus can be compared with other large datasets outside of the design discipline. This represents a further step on the way to understanding design thinking in conversational contexts.

Analysing transcripts of design activity typically involve either close reading or manual coding of data, which limits the amount of data that can be analysed. In contrast, we explore a machine-learning based linguistic analysis tool called Empath to identify patterns of reasoning in design talk. The data we use derives from the Design Thinking Research Symposium (DTRS) shared-data workshops which we analyse to look at two contrasting aspects of design talk: the expression of tentativeness, characterising designers' generative thinking; and the articulation of explanations, characterising their deductive or analytical thinking. We show, at the level of speech turns, how tentativeness and explanation relate to, and overlap, each other. Finally, we discuss the limitations of this ‘linguistic analysis at scale’ approach.

Artificial intelligence (AI) is increasingly being viewed as a creative partner rather than as a tool. How to design such collaborations is still a subject of speculation. In this pictorial, we propose a collaborative role for AI to prompt self-reflection. We explore this through a practice-based inquiry of whether and how AI could help a designer reflect on and relate to their own work. Three designers annotate a collection of images representing their fascinations, with subjective labels, indicating different dimensions of their visual concepts. These labels are used to teach an object detection model the designers’ perspectives. Then, they used this trained model on their own design work to evaluate the AI's potential to prompt self-reflection. By describing this process of AI-training we explore how an AI can help us become aware of our own implicit perspectives.

Creative conversation among designers and stakeholders in a design project enables new ideas to naturally originate and evolve. Language allows for the exchange of values, priorities, and past experience whilst keeping solution forms usefully ambiguous. Yet there is a danger that only the language of people directly involved in the design process gets to be heard, limiting how inclusively the problems are interpreted, which in turn can impede how complex design problems are addressed. Recent advances in artificial intelligence (AI) have shown the exclusionary spaces that are often inhabited by designers, engineers, and developers of new artefacts and technologies. On the other hand, text data used to train language models for machine learning applications have the potential to highlight societal biases in ways that designers can utilise. In this paper, we report the results of an exploratory study using AI text generation to synthesize and narrate opinions and experiences that may be unfamiliar to designers. Three pairs of designers were given a complex socio-technical problem to solve. Of these, two pairs interacted with an AI text generator during the task, while one pair acted as a baseline condition. Analysing the conversational exchanges between the designers and the designers & AI, we observe how the use of AI leads to prompting nuanced interpretations of problems and ideas, opening up the objective problem and design lenses and interpretations. Finally, we discuss how the designers (re)assign different roles to the AI to suit their creative purposes.

Multidimensional data is often visualized using coordinated multiple views in an interactive dashboard. However, unlike in infographics where text is often a central part of the presentation, there is currently little knowledge of how to best integrate text and annotations in a visualization dashboard. In this paper, we explore a technique called FacetNotes for presenting these textual annotations on top of any visualization within a dashboard irrespective of the scale of data shown or the design of visual representation itself. FacetNotes does so by grouping and ordering the textual annotations based on properties of (1) the individual data points associated with the annotations, and (2) the target visual representation on which they should be shown. We present this technique along with a set of user interface features and guidelines to apply it to visualization interfaces. We also demonstrate FacetNotes in a custom visual dashboard interface. Finally, results from a user study of FacetNotes show that the technique improves the scope and complexity of insights developed during visual exploration.

Visual data storytelling is gaining importance as a means of presenting data-driven information or analysis results, especially to the general public. This has resulted in design principles being proposed for data-driven storytelling, and new authoring tools being created to aid such storytelling. However, data analysts typically lack sufficient background in design and storytelling to make effective use of these principles and authoring tools. To assist this process, we present ChartStory for crafting data stories from a collection of user-created charts, using a style akin to comic panels to imply the underlying sequence and logic of data-driven narratives. Our approach is to operationalize established design principles into an advanced pipeline that characterizes charts by their properties and similarities to each other, and recommends ways to partition, layout, and caption story pieces to serve a narrative. ChartStory also augments this pipeline with intuitive user interactions for visual refinement of generated data comics. We extensively and holistically evaluate ChartStory via a trio of studies. We first assess how the tool supports data comic creation in comparison to a manual baseline tool. Data comics from this study are subsequently compared and evaluated to ChartStory's automated recommendations by a team of narrative visualization practitioners. This is followed by a pair of interview studies with data scientists using their own datasets and charts who provide an additional assessment of the system. We find that ChartStory provides cogent recommendations for narrative generation, resulting in data comics that compare favorably to manually-created ones.

Abstract. Analysing records of design activity such as transcripts or documents have typically involved close reading of transcripts and manual identification of concepts and behaviours. We explore the applicability of a machine-learning based computational tool—called Empath—in identifying high-level patterns in design talk. Specifically, we use it to examine the datasets from the Design Thinking Research Symposium (DTRS) workshops for two contrasting aspects of design talk—the expression of tentativeness that characterises designers’ exploration of the problem-solution space, and the expression of causal reasoning that characterises designers’ analytical thinking. We find that such a tool can be effectively used as a means of “distant reading”. However, the lack of design relevance in the tool’s training data results in ambiguities and mis-categorisations that still need resolution through close reading.

Design thinking concepts such as framing, storytelling, and co-evolution, have been widely identified as part of design activity though generally have been evidenced from manual coding of design conversations and close reading of transcripts. The increase in easy-to-use computational linguistic methodologies provides an opportunity not only to validate these concepts, but compare them to other kinds of activity in large datasets. However, the process of systematically identifying such concepts in design conversation is not straightforward. In this paper we explore methods of linguistic analysis for revealing problem frames within design process transcripts. We find that frames can be identified through n-grams with high mutual information scores, used at low frequencies, along with subsequent lexical entrainment. Furthermore, we show how frames are organised in primary and secondary structures. Our results represent a step forward in computationally determining frames in datasets featuring design, or design-like activity.

In today’s world, the advantages of data-enabled design are undeniable, increasing the performance of organisations drastically by informing and inspiring the design process. While organisations seem to be more experienced with quantitative data for evaluative purposes, they do struggle to use data as creative material to inspire the design process. Choosing the right type of data representation is critical for using data for creative purposes. Data visualization has proven to be highly effective in increasing understanding of data, as it is fast, accurate and flexible. Data physicalization, on the other hand, remains unexplored in comparison, especially its effect on creativity. This paper presents the results of two studies (one preliminary and one follow-up study), which explored the use of data physicalization in creative settings. The preliminary study enabled to collect initial requirements for the development of a physicalization toolkit, while the follow-up study investigated its impact on the design process, in comparison to data visualization. From the studies, we developed Concreate, a collaborative data physicalization toolkit designed to lead to creative insights from quantitative data. Our results show that Concreate can potentially stimulate creative thinking, by encouraging intense, tangible interaction with data leading to increased reflection-in-action and a deeper understanding of data. The two studies and toolkit development were carried out at a multinational automotive company, interested in innovating by incorporating data as creative material. Besides the immediate practical implications, we conclude this paper with a discussion on future recommendations for using data physicalization in the design process.

Emerging practices of using ‘off the shelf’ AI as a creative partner in design processes are receiving increasing attention in design research. This paper takes the well-known concept of ‘framing’ in design, along with the Schönian concept of ‘surprise’ to explore how a human-AI dialogue could work. The approach taken is practice-based, with the human designer documenting her process of inquiry and decision making. We show how artificial creativity is expressed through misfiring object detection algorithms, and further how these ‘mistakes’ can be perceived and interpreted by the human designer. The contribution of the research is in laying the foundations for a novel human-AI dialogic practice.

Design thinking concepts such as storytelling, framing, and co-evolution, have been established from close readings of design activity. The increase in easy-to-use computational methodologies provides an opportunity to validate these concepts more widely. Among these concepts, storytelling is already operationalised through various computational approaches. In this paper, we create one corpus of design activity data from the four shared-data DTRS workshops and use Linguistic Inquiry and Word Count (LIWC) in attempting to automatically detect components of stories. However, the conversational nature of the data indicates that further development in methodology is needed. The contribution of the paper lies both in outlining how an automated method for identifying stories could work and showing how the DTRS corpus can be compared with other large datasets outside of the design discipline. This represents a further step on the way to understanding design thinking in conversational contexts.

Dynamic networks-networks that change over time-can be categorized into two types: offline dynamic networks, where all states of the network are known, and online dynamic networks, where only the past states of the network are known. Research on staging animated transitions in dynamic networks has focused more on offline data, where rendering strategies can take into account past and future states of the network. Rendering online dynamic networks is a more challenging problem since it requires a balance between timeliness for monitoring tasks-so that the animations do not lag too far behind the events-and clarity for comprehension tasks-to minimize simultaneous changes that may be difficult to follow. To illustrate the challenges placed by these requirements, we explore three strategies to stage animations for online dynamic networks: time-based, event-based, and a new hybrid approach that we introduce by combining the advantages of the first two. We illustrate the advantages and disadvantages of each strategy in representing low- and high-throughput data and conduct a user study involving monitoring and comprehension of dynamic networks. We also conduct a follow-up, think-aloud study combining monitoring and comprehension with experts in dynamic network visualization. Our findings show that animation staging strategies that emphasize comprehension do better for participant response times and accuracy. However, the notion of 'comprehension' is not always clear when it comes to complex changes in highly dynamic networks, requiring some iteration in staging that the hybrid approach affords. Based on our results, we make recommendations for balancing event-based and time-based parameters for our hybrid approach.

Wearable technologies draw on a range of disciplines, including fashion, textiles, HCI, and engineering. Due to differences in methodology, wearables researchers can experience gaps or breakdowns in values, goals, and vocabulary when collaborating. This situation makes wearables development challenging, even more so when technologies are in the early stages of development and their technological and cultural potential is not fully understood. We propose a common ground to enhance the accessibility of wearables-related resources. The objective is to raise awareness and create a convergent space for researchers and developers to both access and share information across domains. We present CHIMERA, an online search interface that allows users to explore wearable technologies beyond their discipline. CHIMERA is powered by a Wearables Taxonomy and a database of research, tutorials, aesthetic approaches, concepts, and patents. To validate CHIMERA, we used a design task with multidisciplinary designers, an open-ended usability study with experts, and a usability survey with students of a wearables design class. Our findings suggest that CHIMERA assists users with different mindsets and skillsets to engage with information, expand and share knowledge when developing wearables. It forges common ground across divergent disciplines, encourages creativity, and affords the formation of inclusive, multidisciplinary perspectives in wearables development.

Creating Spatial Computing (SComp) artifacts (including Virtual Reality, Augmented Reality, Mixed Reality, and Ambient Intelligent artifacts) is a rapidly-emerging domain in need of new design methodologies. In this paper, we examine whether and how ethics are procedurally integrated into the creations of SComp artifacts. After an introduction to terminology—including a re-framed definition of Spatial Computing—findings of interviews with Spatial Computing practitioners are shared. The interviews indicated an awareness among professionals about the inordinate vulnerability of SComp artifacts, and about the need for—and the lack thereof—processes and tests to mitigate negative effects of SComp artifacts. Results from the domain expert interviews are integrated into a proposed framework: The Framework for Ethical Spatial Computing Design Engineering. Our framework serves to support researchers and practitioners in devising new methodologies unique to Spatial Computing by highlighting considerations central to the creation of ethical artifacts. The framework integrates the findings from the in-depth interview study and builds on existing models in Design Process, Methods, and Human-Computer Interaction (HCI) Research that highlight important barriers and opportunities between research and practice. It maps the three-phases journey consisted of (1) Enablers, (2) Synthesizers, and (3) SComp Artifacts. We trust that our work sheds light on considerations necessary to the creation of ethical Spatial Computing artifacts.

Analysis of large, high-dimensional, and heterogeneous datasets is challenging as no one technique is suitable for visualizing and clustering such data in order to make sense of the underlying information. For instance, heterogeneous logs detailing machine repair and maintenance in an organization often need to be analyzed to diagnose errors and identify abnormal patterns, formalize root-cause analyses, and plan preventive maintenance. Such real-world datasets are also beset by issues such as inconsistent and/or missing entries. To conduct an effective diagnosis, it is important to extract and understand patterns from the data with support from analytic algorithms (e.g., finding that certain kinds of machine complaints occur more in the summer) while involving the human-in-the-loop. To address these challenges, we adopt existing techniques for dimensionality reduction (DR) and clustering of numerical, categorical, and text data dimensions, and introduce a visual analytics approach that uses multiple coordinated views to connect DR + clustering results across each kind of the data dimension stated. To help analysts label the clusters, each clustering view is supplemented with techniques and visualizations that contrast a cluster of interest with the rest of the dataset. Our approach assists analysts to make sense of machine maintenance logs and their errors. Then the gained insights help them carry out preventive maintenance. We illustrate and evaluate our approach through use cases and expert studies respectively, and discuss generalization of the approach to other heterogeneous data.