The multi-armed bandit problem is a sequential learning scenario in which a learning algorithm seeks to obtain rewards by selecting an arm, or action, in each round, given limited initial knowledge. Contextual bandits present an additional context every round that informs the bandit algorithm and guides decision-making. While successfully applied in practice, research continues to explore efficient bandit algorithms for high-dimensional bandits with nonparametric, sparsely varying reward functions. One such algorithm is the two-phase SI-BO algorithm, which incorporates an initial subspace learning phase to identify the effective context subspace on which the function varies, and a subsequent Bayesian optimization phase that applies the Gaussian Process-based GP-UCB algorithm to the learned subspace. While the SI-BO offers a theoretical regret performance with weak sub-exponential dependence on the ambient dimension, it is hindered by a high computational cost stemming from the Gaussian Process regression. Building on the algorithm framework introduced in SI-BO, this paper aims to investigate the empirical regret performance of Gaussian Process-based learning algorithms that incorporate subspace learning. To that end, we introduce a novel algorithm, SI-BKB, which combines the subspace learning in SI-BO with the BKB sketching algorithm, reducing computational complexity while maintaining theoretical guarantees. Through synthetic data generation, this paper presents a systematic empirical study on linear and nonlinear bandit environments with varying levels of sparsity. The results demonstrate that the SI-BKB algorithm has comparable regret performance to the SI-BO. Additionally, the regret performance indicates that misalignment of the learned subspace results in suboptimal regret performance during the optimization phase. Moreover, we demonstrate that high sparsity, through subspace misalignment, can improve the regret performance. Repository is available at \url{https://github.com/Cheese-1/SparseSequentialLearning}.

Stochastic contextual linear bandits are widely used for sequential decision‐making across many domains. However, in high‐dimensional sparse settings, most candidate features are irrelevant to predicting outcomes, and collecting such data is costly. This study examines various SCLB algorithms combined with feature‐selection methods using a fixed reduced dimensionality. With a synthetic dataset exhibiting 90% feature sparsity, we simulated these algorithms and compared their performance to both their base counterparts and to one another. Our experiments demonstrate that integrating feature selection into posterior‐ and confidence‐based bandits can achieve nearly identical regret while requiring only a fraction of the data‐collection cost.

Deep reinforcement learning has shown strong performance in continuous control tasks, but its reliance on deep neural networks (DNNs) hinders interpretability, limiting deployment in safety-critical domains. While recent approaches using differentiable decision trees improve transparency, they often rely on fixed structures that limit flexibility and lead to unnecessarily complex policies.

We propose SPLIT-PO (Sparse Piecewise-Linear Interpretable Tree Policy Optimization), a novel framework that learns sparse, interpretable decision trees with linear leaf controllers and dynamically adaptive structure. SPLIT-PO introduces learnable gating and regularization to prune uninformative branches during training, enabling compact tree policies to emerge automatically. It maintains end-to-end differentiability and integrates crispification within the training loop, building on prior interpretable methods like ICCT.

Experiments on standard continuous control benchmarks show that SPLIT-PO matches neural network performance (e.g., 285 vs. 287 average reward on Lunar Lander) while producing trees with 100–1000× fewer parameters and as few as 1–3 leaf nodes. Additionally, we prove SPLIT-PO is a universal function approximator, offering neural-level expressivity in an interpretable form. Although it requires more samples to converge, SPLIT-PO provides a promising foundation for transparent and verifiable reinforcement learning.

Complex reinforcement learning (RL) models that receive high rewards in their environments are often hard to understand. To this end, more interpretable models can be used, such as decision trees. To be able to deploy these models in safety-critical environments, they need to be high-performing and verifiable. Optimal decision trees can fulfill both these goal. While some methods already exist to find optimal decision trees, none have applied them to RL environments with continuous state
and action spaces [4; 13]. Broccoli is a state-of-the-art approach to synthesizing decision tree policies for blackbox environments [3]. Given a discretisation of the continuous state space, it can find the optimal tree in discrete action environments.
This research will focus on discretising action spaces of continuous environments. In doing so, discrete action spaces are created for which Broccoli can find optimal decision trees. Additionally, a goal is to show that it is possible to discretise continuous action spaces and compute corresponding optimal decision trees in feasible time. Furthermore, it proposes informed discretisation techniques that result in better-performing decision tree policies.

This research addresses the challenge of interpretability in Reinforcement Learning (RL) for environments with continuous action spaces by extending the Decision Tree Policy Optimization (DTPO) algorithm, which was originally developed for discrete action spaces.
Unlike deep RL methods such as Proximal Policy Optimization (PPO), which are effective but difficult to interpret, DTPO offers transparent rule-based policies. We propose a continuous-action variant of the DTPO algorithm, DTPO-c, which allows decision trees to output Gaussian distribution parameters while maintaining interpretability. Our experiments on the Pendulum-v1 environment show that DTPO-c can achieve performance comparable to Robust Policy Optimization (RPO), although it requires more computational effort. Additionally, we investigate the impact of discretizing continuous actions and find that increasing action resolution does not always lead to improved performance, likely due to limited model capacity. These results confirm the feasibility of interpretable RL in continuous environments, making it suitable for applications where understanding and trusting the behavior of the model is important.

The use of research assistants has increased significantly, providing support and automation for researchers. However, there is limited research on researchers using research assistants and what assistance researchers require for each research stage.
We interview researchers to gather insights into their opinions and usage, and afterwards develop a prototype. Researchers highlight that the most difficult part of research is the experiment design, which is reflected in the lack of literature.
Participants use research assistants for reading literature and writing, but request more support. The research assistant must be factual, transparent, and correct, and including a conversation allows for feedback and discussion.
We evaluate the prototype for the experiment design phase, highlighting the effectiveness of the component architecture by generating correct experiments.

This study explores the application of risk-sensitive Reinforcement Learning (RL) in portfolio optimization, aiming to integrate asset pricing and portfolio construction into a unified, end-to-end RL framework. While RL has shown promise in various domains, its traditional risk-neutral approach is unsuitable for financial contexts where risk sensitivity is crucial. This research focuses on risk-sensitive RL methods that incorporate different risk measures to manage uncertainty and volatility in financial markets better. The project extends existing RL techniques by adapting the cross-sectional approach to risk-sensitive settings and introducing new variants like PPO-CVaR and PPO-Expectile for portfolio management. A comparative study of these methods is conducted to assess their performance with real market data and simulated environments. The research addresses key questions related to how different risk measures impact learned portfolio strategies, the influence of risk appetite on decision-making, and the performance gap between simulated and real market data. The findings aim to provide insights for practitioners looking to implement risk-sensitive RL in financial asset management.

This paper explores the challenges of converting architectural floor plans from raster to vector images. Unlike previous studies, our research focuses on domain adaptation to address stylistic and technical variations across different floor plan datasets. We develop and test our vectorization method on the CubiCasa5K benchmark, which includes 3 different floor plan styles. Our analysis reveals differences in input features across the CubiCasa5K styles, indicating the potential for domain adaptation research, mostly in room segmentation. However, we also find multiple indications that labelling in the CubiCasa5K dataset is ambiguous and inconsistent. Furthermore, styles with more training data do not always perform better, highlighting the complexity differences between floor plan styles. Our baseline shows a 0.7% gap for rooms yet a 0.6% improvement for objects, likely caused by the smaller feature gaps and inconsistent labelling. To address the adaptation gap, we add a Multi-Kernel Maximum Mean Discrepancy (MK-MMD) loss to the CubiCasa5K model to minimize feature distribution differences between domains. While our MK-MMD implementation shows potential for reducing the adaptation gap, persistence issues and mixed results across classes make it difficult to draw clear conclusions. Our findings also show the role of balancing spatial context in the MK-MMD calculation. These insights lay a foundation for future domain adaptation research in floor plan vectorization.

Image data augmentation has been regarded as a reliable and effective way to increase the data available for training. With the advent and rise of Generative AI, generative data augmentation has been shown to realize even better gains in performance for downstream tasks. However, these performance gains are often the cause of "extra information" being seeped into the generated examples via pre-trained model weights, heuristic inclusions etc. In this paper, we showcase the impact of text-in-image augmentation on the performance of an underlying downstream task (classification or recognition). This study specifically looks at the difference in performance when training a classifier under three settings- no augmentation, transform-based augmentation, and generative augmentation- and investigate whether and where this augmentation can be successfully employed to experience gains in performance, without letting any "extra information" seep in. We try to observe this difference in performance under varying amounts of training samples, and for samples with varying similarities to that of the original training data. We also present a new GAN structure- conditional Classification Deep Convolutional GAN (or the CcGAN)- as an improved baseline over the conditional Deep Convolutional GAN (cDCGAN) for our experiments which gave a 4\% performance gain over unaugmented data with no 'extra information'. We find that in certain settings and examples, there exists a performance advantage to train vision models in text-in-image settings using real and generated data. We also confirm that the amount of original training samples available affect the test accuracy achieved by generative augmentation, where a huge fall-off can be seen in extremely low- and high- data regimes; however, it seems to maximize performance at a ”sweet spot” where the robustness and variability added by the generated samples help to realize performance gains. It was also observed that the 1x and 5x augmentations performed better than other configurations. Lastly, we find that the similarity of generations does not affect model performance and does not vary consistently with model performance for most settings.

Trust is a fundamental component in human-AI relationships, serving as a critical element of user acceptance and satisfaction, particularly within the realm of Decision Support Systems (DSS). The technological advances in conversational user interfaces (CUIs) such as ChatGPT and digital assistants (e.g., Alexa) allow laypeople to interact with DSS without knowing the mechanisms behind them.
Extensive research has explored the benefits of CUIs and strives to improve their usability and adoption rate. However, while interacting with such CUIs, how to facilitate proper user trust for decision support is still under-explored. To address the research gap, we aim to test the impact of emotional expressiveness in CUIs on building user trust.

To analyze the impact of emotional expressiveness in CUIs to build user trust and whether voice-based CUI is more efficient in building user trust compared to text-based CUI. We implemented a conversational interface with varying emotional expressiveness that can serve six conditions: two text-based and four voice-based. Text-based CUIs are differentiated by lexical expressiveness. Voice-based CUIs are varying in both lexical expressiveness and prosodic expressiveness. Regardless of the modality and emotional expressiveness, each CUI serves as an interactive medium for users with the DSS, which supports them to find a suitable house given a scenario.

Through an empirical study (N = 151), the experimental results are insufficient to conclude the impact of prosodic expressiveness and lexical expressiveness on user trust and usability in CUIs. In addition, we did not find any statistically significant difference between text-based and voice-based CUIs in trust or perceived usability.

Our findings can potentially be explained by the uncanniness effect [46]: initially, increased emotional expressiveness in a chatbot could positively influence user trust, but over time this could turn into a negative impact. These results offer a potential way to explain the complex dynamics of trust in conversational DSS and some implications in chatbot design within the context of DSS. Our findings
can benefit the future design and development of conversational agents-based DSS by considering emotional expressiveness.

The ever-evolving power grid is becoming smarter and smarter. Modern houses come with smart meters and energy conscious consumers will buy additional smart meters to place in their home to help monitor their energy consumption. This new smart technology also opens the door to more accurate power consumption forecasting. In this study we look at utilizing a partial hierarchy, in which one of the appliances in a household is modelled separately from the rest of the house, to help improve household energy consumption forecasting accuracy. This is done in conjunction with Auto Regressive Moving Average (ARMA) based models. Three variants of ARMA based models will be looked at: Auto Regressive Integrated Moving Average (ARIMA), Seasonal Auto Regressive Integrated Moving Average (SARIMA), and Auto Regressive Integrated Moving Average with Exogenous variables (ARIMAX). These methods will then be compared to more baseline approaches such as a persistence method and a seasonal moving average. Our analysis has led us to conclude that the partial hierarchy model offers little to no benefit when applied in the field of household energy consumption forecasting when built upon ARMA based models. ARMA based models in general appeared to be poor performers when it came to household energy consumption forecasting.

In today's society, the rapid progression of digitization has led to the automation of various facets of human existence. This transformation has been facilitated by the utilization of algorithms, which are instrumental in driving efficient and effective automated processes. These algorithms have also found widespread adoption in the public sector, where they are employed to streamline and optimize various tasks and operations. The integration of algorithms in the public sector has brought about significant advancements in areas such as predictive policing, social welfare allocation, and healthcare.

However, the use and development of these automated processes were subjected to concerns from the public about privacy, bias, accountability, and transparency. Since these concerns are mainly coming from citizens, their involvement in the process of developing algorithmic systems can potentially be of help.

We explore the potential of participatory AI in marginalized communities as a means of obtaining valuable input from citizens regarding the development of these algorithmic systems employed by the public sector. One Piece of our approach involves hosting discussions in local community centers in marginalized neighborhoods. Our focus is on dilemmas that are relevant to algorithm design and evaluation decisions, and we frame these dilemmas in various ways, including forms that may not directly relate to societal impact, but are understandable for laypeople. Our key findings suggest that involving marginalized citizens can bring valuable perspectives and insights that are otherwise ignored. By incorporating public perspectives into algorithm development, we can promote inclusive decision-making processes and ensure that algorithms align with community values.

Event-based cameras represent a new alternative to traditional frame based sensors, with advantages in lower output bandwidth, lower latency and higher dynamic range, thanks to their independent, asynchronous pixels. These advantages prompted the development of computer vision methods on event data in the last decade, however event-based datasets are still in early stages in terms of size and complexity compared to normal datasets (e.g. ImageNet). This paper explores event data augmentation by superimposing two existing event datasets (N-MNIST and N-Caltech101) and by adding uniform noise. It shows that training an instance segmentation model on noisy datasets does not improve its performance, but the amount and type of noise added in the background decreases the performance of such model.

Event-based cameras do not capture frames like an RGB camera, only data from pixels that detect a change in light intensity, making it a better alternative for processing videos. The sparse data acquired from event-based video only captures movement in an asynchronous way. In this paper an evaluation is made on the efficiency and accuracy of object detection, specifically localization, between sparse and dense representations of data. Convolutional Neural Networks are used to train and test on images and event-based data. The results show a positive trade-off in terms of accuracy and efficiency for using sparse event-based data instead of dense data like images. These results provide a basis for an argument to use event-based cameras instead of RGB cameras when dealing with object detection.

The event-based camera represents a revolutionary concept, having an asynchronous output. The pixels of dynamic vision sensors react to the brightness change, resulting in streams of events at very small intervals of time. This paper provides a model to track objects in neuromorphic datasets, using clustering. In addition, a non-linear filter is applied to correct the estimation of the object position. Both single and multi-object tracking algorithms are provided and their performance is analyzed using different metrics, including the clustering evaluation scores and the tracking accuracy. The accuracy is over 0.6 for multi-target tracking and more than 0.7 for single object tracking. Besides the proposed model, a comparison between different possible approaches for event-based data tracking is provided.

Instance segmentation on data from Dynamic Vision Sensors (DVS) is an important computer vision task that needs to be tackled in order to push the research forward on these types of inputs. This paper aims to show that deep learning based techniques can be used to solve the task of instance segmentation on DVS data. A high performing model was used to solve this task, using event-based data that was transformed into RGB-D images. The chosen model for this work was Mask R-CNN, with an alteration for depth images, because of its high performance on frame based data. The N-MNIST dataset provides the event-based input, and the transformation of such an input is presented in this study. Furthermore, the masks are generated with the help of the MNIST dataset and heuristics are used for placing them at the correct positions. The results are promising and comparable to other results from literature on the task of semantic segmentation.

Powerful predictive AI systems have demonstrated great potential in augmenting human decision-making. Recent empirical work has argued that the vision for optimal human-AI collaboration requires ‘appropriate reliance’ of humans on AI systems. However, accurately estimating the trustworthiness of AI advice at the instance level is quite challenging, especially in the absence of performance feedback pertaining to the AI system. In practice, the performance disparity of machine learning models on out-of-distribution data makes the dataset-specific performance feedback unreliable in human-AI collaboration. Inspired by existing literature on critical thinking and explanation-based human debugging, we propose the use of debugging an AI system as an intervention to foster appropriate reliance. In this paper, we explore whether a critical evaluation of AI performance within a debugging setting can better calibrate users’ assessment of an AI system and lead to more appropriate reliance. Through a quantitative empirical study (N = 234), we found that our proposed debugging intervention does not work as expected in facilitating appropriate reliance. Instead, we observe a decrease in reliance on the AI system after the intervention — potentially resulting from early exposure to the AI system’s weakness. We explored the dynamics of user confidence to help explain how inappropriate reliance patterns occur and found that human confidence is not independent of AI advice, which is potentially dangerous when trying to achieve appropriate reliance. Our findings have important implications for designing effective interventions to facilitate appropriate reliance and better human-AI collaboration

Metrics are widely used in the software engineering industry and can serve as Key Performance Indicators (KPIs), which are used by management to make informed decisions and understand the performance of the organisation. Many companies measure themselves against industry-standard metrics, in addition to their own set of metrics. This thesis aims to investigate the relationship between these industry-standard metrics and the metrics that are additionally collected. Instead of focusing on the performance of a single organisation, the DORA report focuses on the comparison of organisations. It measures the Software Delivery and Operational (SDO) performance of organisations by four key and industry-standard metrics representing two aspects, stability and throughput, of a software product. The use of one metric as a proxy for another metric or a thematic group of metrics is a common phenomenon. However, there rarely is evidence supporting the assumption that the proxy reflects the intended metric or represents the full thematic group. This thesis performs a single case study within ING, a large and highly digital bank in the Netherlands. It investigates the KPIs that are collected by the bank and analyses the relationships between those KPIs and the four metrics from the DORA report. It establishes a list of 27 KPIs that are in use by ING and shows that there are no correlations between the DORA metrics as collected within the bank and that these metrics show very little correlation with the metrics that ING collects additionally. Furthermore, it is established that nearly all metrics contain a bias at the organisational level and that these biases have a significant impact on the correlation between metrics.

3D modeling techniques can be used to automate processes such as damage assessment in aircraft engines. Aircraft engines often have shiny and non-textured surfaces, where these modeling techniques often have poor performance. This paper gives more insight into the performance of interest detection/matching algorithms on shiny and non-textured surfaces as found in aircraft engine borescope inspection videos. These algorithms are often used in 3D modeling techniques. Three interest point detection/matching algorithms are executed on different test videos, and various metrics are calculated for each algorithm. This paper is the first paper that compares both recent and traditional computer vision interest point detection/matching algorithms in these specific settings, and contributes to a better understanding of the usability of feature-based 3D reconstruction techniques. The results show that more recent neural network-based approaches outperform traditional approaches.

Proper maintenance and inspection of aircraft and their engines is important for society. These engine inspections are performed using borescopes of which the footage is manually analysed. Having the opportunity to reconstruct a 3D model of the rotors would ease the inspection and introduce the possibility to automate the process. Monocular SLAM systems are capable of reconstructing such models in real-time using a video of the rotors. However, SLAM is not tested in environments similar to the aircraft turbine. This study, therefore, assesses the performance of different SLAM approaches in this specific setting. The results show that 3D reconstruction of aircraft engines using direct SLAM has potential for damage assessment. Further research into damage assessment using SLAM is therefore viable.