Aggregating multiple annotations into a single ground truth label may hide valuable insights into annotator disagreement, particularly in tasks where subjectivity plays a crucial role. In this work, we explore methods for identifying subjectivity in recognizing the human values that motivate arguments. We evaluate two main approaches: inferring subjectivity through value prediction vs. directly identifying subjectivity. Our experiments show that direct subjectivity identification significantly improves the model performance of flagging subjective arguments. Furthermore, combining contrastive loss with binary cross-entropy loss does not improve performance but reduces the dependency on per-label subjectivity. Our proposed methods can help identify arguments that individuals may interpret differently, fostering a more nuanced annotation process. ... Read more

Human values capture what people and societies perceive as desirable, transcend specific situations and serve as guiding principles for action. People’s value systems motivate their positions on issues concerning the economy, society and politics among others, influencing the arguments they make. Identifying the values behind arguments can therefore help us find common ground in discourse and uncover the core reasons behind disagreements. Transformer-based large language models (LLMs) have exhibited remarkable performance across language generation and analysis. However, leveraging LLMs in sociotechnical systems that assist with discourse and argumentation necessitates systematically evaluating their ability to analyse and identify the values behind arguments, an under-explored research direction. Using a multi-level human value taxonomy inspired by the Schwartz Theory of Basic Human Values, we present a systematic and critical evaluation of GPT-3.5-turbo in human value identification from a dataset of multi-cultural arguments, across the zero-shot, few-shot and chain-of-thought prompting strategies, carrying forward from prior research on this task which leveraged a fine-tuned BERT model. We observe that prompting strategies exhibit performance levels close to, but still behind fine-tuning for value classification. We also detail some challenges associated with value classification with LLMs, offering potential directions for future research. ... Read more

There is a lack of an intelligent platform that supports continuous deliberation and captures diverse views and stakeholders’ values during the architectural design process in the early stages. Using hybrid intelligence, this study proposes a method that integrates value, and design pattern theories, to support deliberation during the design process. Three steps comprise the method: eliciting value, extracting design patterns, and designing through deliberation with AI agents using natural language processing through hybrid intelligence. The final set of design patterns reflects the participants’ values and ideas, facilitating informed consensus and collaboration between stakeholders supported by AI agents. By integrating diverse perspectives into the loop through continuous deliberation, the proposed method incorporates stakeholders’ value for extracting design patterns that address primary design goals and challenges such as energy transition in the built environment. ... Read more