In this third installment of our GenAI workshop series at DIS, we focus on ‘stopsigns’—the blockages that impede progress in design research with GenAI. These stopsigns manifest as both semantic barriers (political, social, or mental frameworks) and pragmatic hurdles (technical limitations or implementation challenges) that persist despite the rapid advancements since the GenAI boom. Such stopsigns present a productive tension—they often contain partial truths worthy of consideration while simultaneously being shortsighted in ways that prevent progression. From blanket rejection to uncritical acceptance, these barriers affect how meaningfully we engage with GenAI’s potential. Our workshop welcomes both returning and first-time participants to share their experiences with these persistent challenges and work together to develop practical solutions. Through analysis of real cases and hands-on activities, we’ll build strategies for moving beyond these obstacles while acknowledging their legitimate concerns. Our goal is to foster more thoughtful integration of GenAI in design research and practice.

INTRODUCTION: With Alzheimer's disease and related dementias (ADRD) representing an enormous public health challenge, there is a need to support individuals in learning about and addressing their modifiable risk factors (e.g., diet, sleep, and physical activity) to prevent or delay dementia onset. However, there is limited availability for evidence-informed tools that deliver both quality education and support for positive behavior change such as by increasing self-efficacy and personalizing goal setting. Tools that address the needs of Latino/a, at higher risk for ADRD, are even more scarce. METHODS: We established a multidisciplinary team to develop the Healthy Actions and Lifestyles to Avoid Dementia or Hispanos y el ALTo a la Demencia (HALT-AD) program, a bilingual online personalized platform to educate and motivate participants to modify their risk factors for dementia. Grounded in social cognitive theory and following a cultural adaptation framework with guidance from a community advisory board, we developed HALT-AD iteratively through several cycles of rapid prototype development, user-centered evaluation through pilot testing and community feedback, and refinement. RESULTS: Using this iterative approach allowed for more than 100 improvements in the content, features, and design of HALT-AD to improve the program's usability and alignment with the interests and educational/behavior change support needs of its target audience. Illustrative examples of how pilot data and community feedback informed improvements are provided. DISCUSSION: Developing HALT-AD iteratively required learning through trial and error and flexibility in workflows, contrary to traditional program development methods that rely on rigid, pre-set requirements. In addition to efficacy trials, studies are needed to identify mechanisms for effective behavior change, which might be culturally specific. Flexible and personalized educational offerings are likely to be important in modifying risk trajectories in ADRD.

The brain is an incredibly complex organ capable of perceiving and interpreting a wide range of stimuli. Depending on individual brain chemistry and wiring, different people decipher the same stimuli differently, conditioned by their life experiences and environment. This study’s objective is to decode how the CNN models capture and learn these differences and similarities in brain waves using three publicly available EEG datasets. While being exposed to a variety of media stimuli, each brain produces unique brain waves with some similarity to other neural signals to the same stimuli. However, to figure out whether our neural models are able to interpret and distinguish the common and unique signals correctly, we employed three widely used CNN architectures to interpret brain signals. We extracted the pre-processed versions of the EEG data and identified the dependency of time windows on feature learning for song and movie classification tasks, along with analyzing the performance of models on each dataset. While the minimum length snippet of 5 s was enough for the personalized model, the maximum length snippet of 30 s proved to be the most efficient in the case of the generalized model. The usage of a deeper architecture, i.e., DeepConvNet was found to be the best for extracting personalized and generalized features with the NMED-T and SEED datasets. However, EEGNet gave a better performance on the NMED-H dataset. Maximum accuracy of 69%, 100%, and 56% was achieved in the case of the personalized model on NMED-T, NMED-H, and SEED datasets, respectively. However, the maximum accuracies dropped to 18%, 37%, and 14% on NMED-T, NMED-H, and SEED datasets, respectively, in the generalized model. We achieved a 5% improvement over the state of the art while examining shared experiences on NMED-T. This marked the outof-distribution generalization problem and signified the role of individual differences in media perception, thus emphasizing the development of personalized models along with generalized models with shared features at a certain level.

Learning engineering adds tools and processes to learning platforms to support improvement research. One kind of tool is A/B testing-common in large software companies and also represented academically at conferences like the Annual Conference on Digital Experimentation (CODE). A number of A/B testing systems focused on educational apps have arisen recently, including UpGrade and E-TRIALS. A/B testing can help improve educational platforms, yet challenging issues in education go beyond the generic paradigm. In response, a number of of digital learning platforms is opening their systems to learning-improvement research by instructors and/or third-party researchers, with specific supports necessary for education-specific research designs. This workshop will explore how A/B testing in educational contexts is different, how learning platforms are opening up new possibilities, and how these empirical approaches can be used to drive powerful gains in student learning. It will also discuss forthcoming opportunities for funding to conduct platform-enabled learning research.

Background: Readability metrics provide us with an objective and efficient way to assess the quality of educational texts. We can use the readability measures for finding assessment items that are difficult to read for a given grade level. Hard-to-read math word problems can put some students at a disadvantage if they are behind in their literacy learning. Despite their math abilities, these students can perform poorly on difficult-to-read word problems because of their poor reading skills. Less readable math tests can create equity issues for students who are relatively new to the language of assessment. Less readable test items can also affect the assessment's construct validity by partially measuring reading comprehension. Objectives: This study shows how large language models help us improve the readability of math assessment items. Methods: We analysed 250 test items from grades 3 to 5 of EngageNY, an open-source curriculum. We used the GPT-3 AI system to simplify the text of these math word problems. We used text prompts and the few-shot learning method for the simplification task. Results and Conclusions: On average, GPT-3 AI produced output passages that showed improvements in readability metrics, but the outputs had a large amount of noise and were often unrelated to the input. We used thresholds over text similarity metrics and changes in readability measures to filter out the noise. We found meaningful simplifications that can be given to item authors as suggestions for improvement. Takeaways: GPT-3 AI is capable of simplifying hard-to-read math word problems. The model generates noisy simplifications using text prompts or few-shot learning methods. The noise can be filtered using text similarity and readability measures. The meaningful simplifications AI produces are sound but not ready to be used as a direct replacement for the original items. To improve test quality, simplifications can be suggested to item authors at the time of digital question authoring.

This one day workshop will explore the use of Generative Artificial Intelligence (GenAI) in design research and practice. Generative technologies are developing rapidly and many designers are using them. Yet, there remains little published work on the use of GenAI in design. Our goal is to not only showcase the potential of GenAI for design, but to engage in discussions of its shortcomings and opportunities as they have been already articulated by scholars. By synthesizing both published and unpublished works, we will develop best practices, ethical considerations, and future research directions for the use of GenAI in design. We will explore a range of topics and themes, including leveraging the characteristics of GenAI for design, mapping the diverse applications of GenAI in design, envisioning a framework for design, and guiding future work on GenAI in design research. Ultimately, we hope to provide a roadmap for the integration of GenAI into the design research process and to encourage designers and researchers to explore the potential of GenAI in a thoughtful and deliberate way.

Music recommendation systems struggle with predicting the aesthetic responses of listeners based solely on acoustic characteristics, which are dependent on the listener's perception. This research correlates acoustic music features with brain responses to report the neural aesthetic hypothesis that the intensity of an aesthetic experience can be decoded based on the degree of correlation to brain responses. We employ hybrid encoding-decoding model (Canonical Correlation Analysis) to identify music features that maximally covary with brain responses. EEG signals of 20 participants are analyzed while they listen to 12 songs and mark their enjoyment on a scale of 1 to 5. Firstly, 18 acoustic features are extracted from music signals and transformed into the first principal component (PC1). In addition, two other features used for analysis are root mean square (RMS) and Spectral Flux (Flux). The first principal canonical component (CC1) with PC1 determines significant (p<0.05) evidence of correlating with brain responses that increasing correlation reflects increased enjoyment. We consider each participant's average CC1 values and enjoyment rating over all 12 songs, followed by plotting a correlation graph to decode the relationship. We observe a significant (p<0.05) positive linear correlation with increasing CC1 scores of PC1 features against increased enjoyment rating. PC1 shows the maximum Pearson correlation (r = 0.48, p = 0.03). In addition, we segregate the brain responses based on low (1,2) and high (3,4) enjoyment ratings and find that higher CC1 values correspond to brain responses of high enjoyment and low values to low enjoyment in all three features. Our experiments reveal that Canonical correlation reflects music-induced pleasure and can be employed in EEG-enabled headphones to decode the user experience, leading to better recommendations.

Advances in neurotechnology have enhanced and simplified our ability to research brain activity with low-cost and effective equipment. One such scalable and noninvasive technique is Electroencephalography (EEG), which detects and records electrical brain activity. Brain activity recognition is one of the emerging problems as EEG wearables become more readily available. Our research has modeled EEG signals to classify three states (i) music listening, (ii) movie watching, and (iii) meditating. The datasets incorporating the brain signals induced while performing these activities are NMED-T for music listening, SEED for movie watching, and VIP_Y_HYT for meditating. EEG activity is transformed into deep representation using a convolutional neural network comprising three different types of 2D convolutions: Temporal, Spatial, and Separable, to capture dependencies and extract high-level features from the data. The Depthwise Convolution function is responsible for learning spatial filters within each temporal convolution, and combining these spatial filters across all temporal bands optimally is learned by the Separable Convolutions. EEGNet and EEGNet-SSVEP are specially designed for EEG Signal Processing and Classification, and the DeepConvNet has incorporated more convolution layers. Our finding demonstrates that increasing the number of layers in the Network provided a higher accuracy of 99.94% using DeepConvNet. In contrast, the accuracy of EEGNet and EEGNet-SSVEP resulted in 85.63% and 75.76%, respectively.

Entrainment is a phenomenon of phase or temporal matching of one system with that of another system. Human neural activity has been shown to resonate with external auditory stimuli. When we enjoy a piece of music, there is a resonance of brain responses with auditory signals. The crux of music cognition is based on this resonance of musical frequencies with intrinsic neural frequencies. It has also been demonstrated that the neural activities are synchronized across participants while listening to music, shown by high inter-subject correlation. In this work, we use this fact to predict the drumbeat a participant listens to based on their EEG response to the drumbeat. We also tested whether we could train on a smaller dataset and test with the rest of the dataset. We generated a frequency∗channel plot and fed it to a CNN model to predict drumbeat with a classification accuracy of 97% for 60-20-20 (train-dev-test) data split protocol and 94% accuracy for 20-20-60 data split. We also got 100% classification accuracy for predicting participants for both the data split protocols.

Introduction: Designing artificial intelligence (AI) to support health and wellbeing is an important and broad challenge for technologists, designers, and policymakers. Drawing upon theories of AI and cybernetics, this article offers a design framework for designing intelligent systems to optimize human wellbeing. We focus on the production of wellbeing information feedback loops in complex community settings, and discuss the case study of My Wellness Check, an intelligent system designed to support the mental health and wellbeing needs of university students and staff during the COVID-19 pandemic. Methods: The basis for our discussion is the community-led design of My Wellness Check, an intelligent system that supported the mental health and wellbeing needs of university students and staff during the COVID-19 pandemic. Our system was designed to create an intelligent feedback loop to assess community wellbeing needs and to inform community action. This article provides an overview of our longitudinal assessment of students and staff wellbeing (n = 20,311) across two years of the COVID-19 pandemic. Results: We further share the results of a controlled experiment (n = 1,719) demonstrating the enhanced sensitivity and user experience of our context-sensitive wellbeing assessment. Discussion: Our approach to designing “AI for community wellbeing,” may generalize to the systematic improvement of human wellbeing in other human-computer systems for large-scale governance (e.g., schools, businesses, NGOs, platforms). The two main contributions are: 1) showcasing a simple way to draw from AI theory to produce more intelligent human systems, and 2) introducing a human-centered, community-led approach that may be beneficial to the field of AI.

Learning engineering adds tools and processes to learning platforms to support improvement research. One kind of tool is A/B testing, which is common in large software companies and also represented academically at conferences like the Annual Conference on Digital Experimentation (CODE). A number of A/B testing systems focused on educational applications have arisen recently, including UpGrade and E-TRIALS. A/B testing can be part of the puzzle of how to improve educational platforms, and yet challenging issues in education go beyond the generic paradigm. For example, the importance of teachers and instructors to learning means that students are not only connecting with software as individuals, but also as part of a shared classroom experience. Further, learning in topics like mathematics can be highly dependent on prior learning, and thus A or B may not be better overall, but only in interaction with prior knowledge. In response, a set of learning platforms is opening their systems to improvement research by instructors and/or third-party researchers, with specific supports necessary for education-specific research designs. This workshop will explore how A/B testing in educational contexts is different, how learning platforms are opening up new possibilities, and how these empirical approaches can be used to drive powerful gains in student learning. It will also discuss forthcoming opportunities for funding to conduct platform-enabled learning research.

Naturalistic music typically contains repetitive musical patterns that are present throughout the song. These patterns form a signature, enabling effortless song recognition. We investigate whether neural responses corresponding to these repetitive patterns also serve as a signature, enabling recognition of later song segments on learning initial segments. We examine EEG encoding of naturalistic musical patterns employing the NMED-T and MUSIN-G datasets. Experiments reveal that (a) training machine learning classifiers on the initial 20s song segment enables accurate prediction of the song from the remaining segments; (b) β and γ band power spectra achieve optimal song classification, and (c) listener-specific EEG responses are observed for the same stimulus, characterizing individual differences in music perception.

Background: As smartphone technology has become nearly ubiquitous, there is a growing body of literature suggesting that ecological momentary cognitive testing (EMCT) offers advantages over traditional pen-and-paper psychological assessment. We introduce a newly developed platform for the self-administration of cognitive tests in ecologically valid ways. Objective: The aim of this study is to develop a Health Insurance Portability and Accountability Act-compliant EMCT smartphone-based platform for the frequent and repeated testing of cognitive abilities in everyday life. This study examines the psychometric properties of 7 mobile cognitive tests covering domains of processing speed, visual working memory, recognition memory, and response inhibition within our platform among persons with and without bipolar disorder (BD). Ultimately, if shown to have adequate psychometric properties, EMCTs may be useful in research on BD and other neurological and psychiatric illnesses. Methods: A total of 45 persons with BD and 21 demographically comparable healthy volunteer participants (aged 18-65 years) completed smartphone-based EMCTs 3 times daily for 14 days. Each EMCT session lasted approximately 1.5 minutes. Only 2 to 3 tests were administered in any given session, no test was administered more than once per day, and alternate test versions were administered in each session. Results: The mean adherence to the EMCT protocol was 69.7% (SD 20.5%), resulting in 3965 valid and complete tests across the full sample. Participants were significantly more likely to miss tests on later versus earlier study days. Adherence did not differ by diagnostic status, suggesting that BD does not interfere with EMCT participation. In most tests, age and education were related to EMCT performance in expected directions. The average performances on most EMCTs were moderately to strongly correlated with the National Institutes of Health Toolbox Cognition Battery. Practice effects were observed in 5 tests, with significant differences in practice effects by BD status in 3 tests. Conclusions: Although additional reliability and validity data are needed, this study provides initial psychometric support for EMCTs in the assessment of cognitive performance in real-world contexts in BD.

In the modern world, it is easy to get lost in thought, partly because of the vast knowledge available at our fingertips via smartphones that divide our cognitive resources and partly because of our intrinsic thoughts. In this work, we aim to find the differences in the neural signatures of mind-wandering and meditation that are common across different meditative styles. We use EEG recording done during meditation sessions by experts of different meditative styles, namely shamatha, zazen, dzogchen, and visualization. We evaluate the models using the leave-one-out validation technique to train on three meditative styles and test the fourth left-out style. With this method, we achieve an average classification accuracy of above 70%, suggesting that EEG signals of meditation techniques have a unique neural signature across meditative styles and can be differentiated from mind-wandering states. In addition, we generate lower-dimensional embeddings from higher-dimensional ones using t-SNE, PCA, and LLE algorithms and observe visual differences in embeddings between meditation and mind-wandering. We also discuss the general flow of the proposed design and contributions to the field of neuro-feedback-enabled mind-wandering detection and correction devices.