Within the field of negotiations, a recent publication is a paper called the NegotiAct[9], which analyzed existing coding schemes of negotiations and introduced an improvement on them that promises a viable way to analyze negotiations in depth. In this research, my goal is to develop a workflow for gathering information and analyzing it with the NegotiAct. To this end the Colored Trails Game [7] is used to design an experiment that simulates real-life negotiations. The Colored Trails Game is a game where each player, through negotiating tries to maximize their own score under limited resources. The game at its core offers the opportunity for both cooperation and competitiveness.
In the experiment a total of 15 participants took part and it was run a total of 20 times, resulting in 3 hours and 10 minutes of recordings. Encoding them with the NegotiAct resulted in the discerning of a total of 87 offers made, 24 offers accepted, 26 offers rejected, and 16 requests for offer modifications. Based on this coherent mapping it can be concluded that the Colored Trails Game is a suitable choice for the workflow of gathering data to be put through the analysis of the NegotiAct.

Meetings are a vital part of discussions and negotiations. Unfortunately, individuals often leave with a vague understanding of the topics covered during the meeting and tend to forget even more of what transpired as time goes on. Driven by previous research that attempts to solve the issue by using architectural shapes as a way of removing ambiguity along with recent advancements in Automatic Speech Recognition (ASR) and Natural Language Processing (NLP) this research attempts to improve user understanding of key topics discussed in meetings by combining ASR models with NLP tools to create a visual summary that would improve user understanding of key topics covered during meetings. To achieve this the research utilizes the speech-to-text transcription and speaker identification capabilities of the WhisperX model with noun phrase extraction features provided by Spacy and key topic recognition functionality of Microsoft's DeBERTa model. Finally, the data is presented as a node-based graph utilizing the D3.js library. The results show that the system is able to identify between 33% - 58% of meeting key topics. This shows the potential of combining ASR models with NLP tools for creating concise meeting summaries but also raises new questions such as why some topics were missed, how the system performance can be improved, and how to design an optimal user interface for such a task.

Meetings represent a key component of collabora- tion in the workplace, serving purposes like brain- storming, discussion, and negotiation. Despite their importance, reaching a consensus among partici- pants can frequently be difficult because different people can leave the debate with different perspec- tives. In order to promote efficient communication and decision-making in organisational contexts, the use of the Shape Language is proposed. The Shape Language consists of shapes that people can use in meetings in order to represent abstract ideas, that would be difficult to represent by only words. In order to track how people interact with these ob- jects, computer vision tools can be used. This study aims to explore the current existing computer vi- sion tools for segmenting and classifying objects in meetings, aiming to find limitations in how well these models are able to recognize objects in the context of meetings and negotiations. Results of this study show that after fine-tuning four models on the custom dataset, they can recognize the three shapes provided as classes in most of the cases, but still make mistakes when assigning classes, or miss objects that they should classify all together, which show limitations of these modern tools.

Large language models (LLMs) are widely used tools that assist us by answering various questions. Humans implicitly use contrast as a natural way to think about and seek explanations (i.e., "Why A and not B?"). Explainability is a challenging aspect of LLMs, as we do not truly understand how good the LLM answers are. The challenge is understanding to what extent LLMs can generate effective contrastive self-explanations for users. We introduce the Contrastive Self-Explanation Method (CoSEM) to narrow the gap between LLMs and explainability. It generates contrastive self- explanations and evaluates them through automation and a user study on generality, usefulness, readability, and relevance. Our results indicate that LLMs are capable of generating effective contrastive self-explanations. Lexical analysis of contrastive explanation indicates that explanations are not less general than the text those explain, and semantic analysis shows that more complex models generalize self-explanations more consistently. Although it is challenging to evaluate contrast in self-explanations semantically, user study shows that some models (Llama3-8B) help understand the contrast. Moreover, task selection affects how readable users find the explanations, where more self-explanations on general topics (movie reviews) are more readable than more specific topics (medical diagnoses). Lastly, some models, such as Llama3-8B, excel at generating contrastive self-explanations that contain relevant information regarding input text.

Reverberation is a key aspect when designing the interior of buildings, and must be carefully considered in the context of the function of the room. Defined by the reverberation time (RT), it is known to have a big influence on the intelligibility and quality of audio in closed spaces.
In this work, we investigate the relationship between the RT and explore the feasibility of using multilayer perceptron (MLP) networks to create an estimator for the RT by using the values of objective measures as input features. We investigate five measures in particular: the Perceptual Evaluation of Speech Quality (PESQ), Virtual Speech Quality Objective Listener (ViSQOL) and its extension focused on audio (ViSQOLAudio), and the Short-time Objective Intelligibility Measure (STOI) and its extension ESTOI.
We create a 3-layer MLP network that estimates the RT with a mean absolute error of 0.144 on our simulated RIR test sets and 0.196 on our real RIR test set.

Artificial Intelligence (AI) has rapidly advanced, significantly impacting software engineering through AI-driven tools like ChatGPT and Copilot. These tools, which have garnered substantial commercial interest, rely heavily on the performance of their underlying models, assessed via benchmarks. However, the current focus on performance scores has often overshadowed the quality and rigor of these benchmarks, as emphasized by the absence of studies on this topic. This thesis addresses this gap by reviewing and improving benchmarking practices in the field of AI for software engineering (AI4SE).

First, a categorized overview and analysis of nearly a hundred prominent AI4SE benchmarks from the past decade are provided. Based on this analysis, several challenges and future directions are identified and discussed, including quality control, programming and natural language diversity, task diversity, purpose alignment, and evaluation metrics. Lastly, a significant contribution of this work is the introduction of HumanEvalPro, an enhanced version of the original HumanEval benchmark. HumanEvalPro incorporates more rigorous test cases and edge cases, providing a more accurate and challenging assessment of model performance. The findings demonstrate substantial drops in pass@1 scores for various large language models, highlighting the necessity for well-maintained and comprehensive benchmarks.

This thesis aims to set a new standard for AI4SE benchmarks, providing a foundation for future research and development in this rapidly evolving field.

A Room Impulse Response (RIR) is a mathematical model for sound propagation in a room. Estimating RIR parameters such as the reverberation time (T60) allows Automatic Speech Recognition (ASR) systems to adapt to reverberation in input signals by changing their behavior based on these estimates. Currently, machine learning techniques provide the most accurate T60 estimations. We propose a novel methodology by using intrusive Speech Intelligibility Measures (SIMs) beyond their traditional application. In this study we utilize SIIB, SIIB^Gauss, STOI and ESTOI as SIMs. For each SIM we find a best fit curve with respect to the reverberation time (T60) using a statistical approach. The statistical analysis is applied on simulated RIRs obtained by using the Image Source Method. The estimator for SIIB^Gauss achieves the lowest Mean Squared Error of 0.353 on simulated data. Although this does not outperform state-of-the-art models, we offer recommendations for possible improvements. Preliminary experiments suggest that enhancing noise robustness is crucial and that the estimators could be generalized to real-world scenarios. However, further research is necessary to confirm this.

The accurate simulation of Room Impulse Responses (RIRs) is important in a variety of applications in acoustics such as automatic speech recognition, speech enhancement, and architectural acoustic design. While objective metrics for evaluating RIRs have been researched extensively, the subjective perceptual accuracy of the simulations is largely overlooked. This paper seeks to address this gap, designing a subjective testing methodology for evaluating the perceptual accuracy of simulated RIRs. A framework is proposed that combines the ABX testing methodology with a modified Multiple Stimuli with Hidden Reference and Anchor (MUSHRA) approach, measuring attributes such as clarity, warmth, environment, and reverbera- tion. The study involved 50 participants evaluating audio samples convolved with both real and simulated RIRs. Results seem to indicate that participants could reliably distinguish between real and simulated RIRs, with perceptual differences observed in the “clarity” and “reverberation” attributes. The findings suggest that current simulation methods for RIRs do not fully capture the perceptual aspects of acoustic environments.

Estimating reverberation time (RT60) accurately is crucial for enhancing the acoustic quality of various environments as it decides how you feel the sound fades away subjectively. Traditional methods, such as Sabine's equation, require extensive prior knowledge and assume ideal conditions, limiting their practicality. To address these limitations, this paper explores the application of convolutional neural networks (CNNs) enhanced with an encoder architecture based on transformer mechanisms for blind RT60 estimation. The proposed model leverages simulated and real-world datasets, incorporating environmental noise to improve robustness. Results indicate that the CNN-Encoder model achieves superior performance, with a mean squared error (MSE) as low as 0.0006 seconds for pure room impulse responses (RIRs) and 0.0011 seconds under +30dB signal-to-noise ratio (SNR) conditions. It also demonstrates potential in practical usage achieving an MSE of 0.0282 seconds under audio recordings. This approach offers a significant reduction in estimation error compared to the CNN-only architecture, demonstrating the potential for improved acoustic parameter estimation in varied environments. Future work will focus on further optimizing the model for real-world applications and reducing computational complexity while maintaining high accuracy.

Recommender systems play a large role on contemporary music platforms, but they tend to work less well for non-mainstream listeners such as children. Additionally, there is no one strategy to perfectly capture a listener's music preference. As children develop understanding of music in different stages, using features they respond to might make recommendations more accurate. Therefore, this study seeks to investigate the effectiveness of a recommender which utilises matrix factorisation augmented with the musical features of tempo, mode, dynamics and time signature in recommending songs a child user would like. We evaluate the quality of this recommender based on the Factorisation Machine algorithm by comparing it to a non-augmented variant of the same algorithm and similar ones using fewer of the same features. Results show that while adding features improves the quality of recommendations, adding too many or the wrong features diminishes said improvement, although more research is needed in this direction.

The promotion of desirable behaviours, such as socially appropriate or health-promoting actions, can be bolstered through a deeper understanding and awareness of the values that underpin the associated behavioural choices. Various implementations for promoting behaviour change based on goals already exist in Human-Robot Interaction, but, since values are the building blocks for our behaviour an agent can use values for behaviour change. By reflecting on value-related choices in a conversation, a conversational agent can assist in identifying the values at stake. This is particularly significant as these agents serve as accessible and non-judgemental platforms for discussing various concerns in a private setting. Such reflective conversations are time-consuming and may span multiple sessions, necessitating some form of memory (e.g., to reflect on earlier statements and compare the choices made for different situations).

While various robotic agents have been developed to provide behavioural support (e.g., for human health), the absence of a comprehensive memory structure and dialogue strategies capable of fostering personalised, reflective conversations based on the appreciation of certain values and actions in various scenarios through contextualised values remains a challenge. To address this, this study introduces the Contextualised Value Model – a dynamic memory model designed to facilitate value-based reflection and support personalised interactions between humans and robotic agents.

To realise this robotic memory, a conversational agent was designed that could elicit values from participants by discussing various scenarios that happen in daily life and reflecting on said values using perspective-taking and other dialogue strategies.

The evaluation of the Contextualised Value Model focused on three primary aspects: the model's accuracy, the influence on likeability and intelligence, and the effect on participants' value awareness. The model was evaluated during a between-subjects experiment (N=54), consisting of two conditions, one where the robot was able to update and use the Contextualised Value Model, and another one where the Contextualised Value Model was random throughout the conversation.

The outcome measures indicated that the integration of the memory model in conversations led to a personalised and relevant conversation, highlighting the potential of the Contextualised Value Model in enhancing conversation personalisation. Although participants' value awareness and perception of the robot's likeability and intelligence did not significantly differ based on the memory model, the study emphasised the need for extended observation to thoroughly evaluate long-term impacts.

Overall, the Contextualised Value Model presents a promising framework for enhancing personalised interactions in various real-world applications, emphasising the need for further research in this area. The ePartner4all project could be further developed to complement the efforts of primary school teachers and parents in supporting children's self-learning of socially, mentally, and physically desirable behaviours.

Many domain experts encounter significant challenges in leveraging machine learning due to the technical complexity of model selection and development. This thesis presents LLM-PQA, a system that enables natural language interaction with machine learning functionalities through large language models. The system combines LLM-based query interpretation with vector search techniques for resource matching and automated machine learning tools for model development. We provide a benchmark dataset for evaluating natural language-driven machine learning systems and conduct experiments examining feature extraction accuracy and embedding technique effectiveness, along with a comprehensive analysis of AutoML tool integration capabilities. Using our benchmark dataset, the evaluation demonstrates that one-shot prompting with LLMs achieves reliable feature extraction, while normalized vector embeddings enable precise resource matching. Analysis of various AutoML tools reveals integration strategies that maintain both technical capability and user accessibility. These findings advance the development of natural language interfaces for machine learning systems while providing insights into effective integration patterns for automated machine learning tools.

In 2022, the Dutch Open Government Act (Wet open overheid, Woo) has required that government institutions share requested documents with citizens, thereby enhancing government transparency and public access to information.
However, current document retrieval processes often struggle to meet the legal requirements of the Woo, as they frequently fail to respond to requests within the legally mandated time frame due to the lengthy retrieval process.

This study addresses the technical challenges of optimizing information retrieval systems in the context of the Woo, by focusing primarily on document precision and recall.
By critically analyzing existing workflows, we identify key inefficiencies and propose enhancements.
Our research includes a comparative evaluation of dense and sparse retrieval methods to assess their effectiveness in this domain.
Additionally, we explore different preprocessing techniques, investigating their impact on retrieval performance on both sparse and dense retrieval systems, to determine the optimal approach for handling noisy, unstructured government data.

Our results show that these changes in retrieval methods can significantly improve retrieval accuracy and reduce response times.
BM25 in particular, shows strong performance, effectively handling the noisy data often present in government documents, highlighting its suitability for this context.
These insights provide insights for government institutions to improve and streamline their information retrieval workflows, and reduce delays of the Woo requests.

Contract review is a critical yet time-consuming process in legal practice, with significant financial implications when errors occur. While Large Language Models (LLMs) have shown promise in legal document processing, they still face challenges with lengthy contracts and complex legal relationships. This research presents an advanced approach to automated contract review by integrating knowledge graphs into Retrieval-Augmented Generation (RAG) frameworks, addressing the limitations of current methodologies.

Through a comprehensive literature review of contract review automation and RAG systems, we conducted systematic experiments comparing RAG approaches with LLMs' in-context learning capabilities. Our empirical analysis validated that RAG-based methods significantly enhance long-context text analysis and information extraction in legal documents, particularly in terms of accuracy and consistency.

Building on these findings, we extensively investigated optimization techniques for the RAG retrieval phase, recognizing its critical role in contract review accuracy. Our experimental evaluation encompassed various chunking strategies, query expansion methods, and re-ranking approaches, establishing best practices for legal document processing.

Our primary contribution is a novel KG-RAG system that enhances contextual understanding in legal document analysis. We evaluate our approach using the Contract Understanding Atticus Dataset (CUAD) and ContractNLI dataset, demonstrating improved performance over traditional RAG implementations and long-context models. The research also explores optimal chunking strategies and investigates the efficiency-effectiveness trade-offs between different model architectures.

Results indicate that our KG-enhanced RAG framework achieves superior performance in identifying and analyzing complex legal relationships while maintaining computational efficiency. The integration of knowledge graphs particularly excels in capturing hierarchical and cross-referential relationships within legal documents, a crucial aspect often overlooked by conventional approaches.

This work advances the field of legal AI by providing a more robust and context-aware approach to contract review, while offering practical insights for implementing AI systems in legal practice. Our findings suggest promising directions for future research in legal document processing, particularly in areas requiring deep contextual understanding and relationship modeling.