Offline reinforcement learning (RL) trains policies from pre-collected data, valuable in scenarios where real-world interaction is costly or risky. This paper systematically investigates the interpretability-performance trade-off of decision tree policies in a framework that reframes offline RL as supervised regression. Through extensive empirical evaluation of single and decomposed decision trees against an XGBoost ensemble on diverse D4RL environments, we show that compact trees, though inherently interpretable, suffer significant performance loss. Conversely, achieving competitive returns with larger trees sacrifices practical human auditability. Critically, return-to-go (RTG) conditioning introduces significant behavioral fragility; policies, despite structural transparency, exhibit unpredictable responses to RTG shifts, complicating their practical interpretability in dynamic environments. Our findings demonstrate that structural simplicity alone is insufficient for practical transparency in goal-conditioned RL, underscoring the need for further research in robustly interpretable sequential decision-making systems.

Deep reinforcement learning has shown strong performance in continuous control tasks, but its reliance on deep neural networks (DNNs) hinders interpretability, limiting deployment in safety-critical domains. While recent approaches using differentiable decision trees improve transparency, they often rely on fixed structures that limit flexibility and lead to unnecessarily complex policies.

We propose SPLIT-PO (Sparse Piecewise-Linear Interpretable Tree Policy Optimization), a novel framework that learns sparse, interpretable decision trees with linear leaf controllers and dynamically adaptive structure. SPLIT-PO introduces learnable gating and regularization to prune uninformative branches during training, enabling compact tree policies to emerge automatically. It maintains end-to-end differentiability and integrates crispification within the training loop, building on prior interpretable methods like ICCT.

Experiments on standard continuous control benchmarks show that SPLIT-PO matches neural network performance (e.g., 285 vs. 287 average reward on Lunar Lander) while producing trees with 100–1000× fewer parameters and as few as 1–3 leaf nodes. Additionally, we prove SPLIT-PO is a universal function approximator, offering neural-level expressivity in an interpretable form. Although it requires more samples to converge, SPLIT-PO provides a promising foundation for transparent and verifiable reinforcement learning.

This research addresses the challenge of interpretability in Reinforcement Learning (RL) for environments with continuous action spaces by extending the Decision Tree Policy Optimization (DTPO) algorithm, which was originally developed for discrete action spaces.
Unlike deep RL methods such as Proximal Policy Optimization (PPO), which are effective but difficult to interpret, DTPO offers transparent rule-based policies. We propose a continuous-action variant of the DTPO algorithm, DTPO-c, which allows decision trees to output Gaussian distribution parameters while maintaining interpretability. Our experiments on the Pendulum-v1 environment show that DTPO-c can achieve performance comparable to Robust Policy Optimization (RPO), although it requires more computational effort. Additionally, we investigate the impact of discretizing continuous actions and find that increasing action resolution does not always lead to improved performance, likely due to limited model capacity. These results confirm the feasibility of interpretable RL in continuous environments, making it suitable for applications where understanding and trusting the behavior of the model is important.

Anomaly detection is a cornerstone of data analysis, aimed at identifying patterns that deviate from expected behaviour. However, conventional anomaly detection methods often fail to differentiate between actionable anomalies and those that, while statistically anomalous, are irrelevant to the user’s goals. Such uninteresting anomalies, originating from distinct, unrelated distributions, contribute to false alarms and resource inefficiencies, particularly in critical domains like cybersecurity and healthcare. This thesis proposes a novel, adaptive framework that retrains anomaly detection models to exclude uninteresting anomalies from being flagged, thereby improving the relevance of detected anomalies.

Central to this framework is the use of the Synthetic Minority Over-sampling Technique for Nominal and Continuous (SMOTE-NC) to artificially augment datasets with labelled uninteresting anomalies, transforming them into regular data. The framework also incorporates user feedback to iteratively refine model performance during deployment. A key finding of this research is that the effectiveness of the framework is highly dependent on the degree of distinguishability between interesting and uninteresting anomalies. Specifically, when the two types of anomalies are clearly distinct in terms of their statistical and categorical features, the framework achieves a significant reduction in false positives without adversely affecting the detection rate of actionable anomalies or the accuracy of regular data.

The framework was evaluated using four state-of-the-art anomaly detection models: Isolation Forest, One-Class Support Vector Machines, Autoencoders, and Variational Autoencoders. It was then tested using two datasets: one in cybersecurity, involving various attack types, and another in healthcare, where anomalies represent different diagnostic categories. The results demonstrate that the framework can effectively identify and suppress uninteresting anomalies, achieving over 90\% accuracy in classifying these cases as regular data under favourable conditions. Notably, when the distinction between interesting and uninteresting anomalies was substantial, the models retained their ability to detect actionable anomalies, with minimal degradation in overall accuracy. Furthermore, it was seen that a significant number of samples are needed to be able to successfully represent the uninteresting anomalies class, a minimum of around 50. When not using the framework, it takes many more samples for the algorithm to successfully no longer detect these uninteresting anomalies as anomalies. This class then needs to make up a significant amount of the training data, so depending on the size of the training data set, this can mean thousands of samples. Gathering around 50 samples poses no problem for the framework, as it is meant to be especially relevant when we have too many uninteresting anomalies, where it is constantly giving false alerts, so there is usually a sufficient number of samples available.

This research project investigates the creative potential of AI-generated images, specifically those produced by the Dream-OOD diffusion model, through the lens of Margaret Boden’s creativity framework. We conducted a user study employing an Elo-based rating system to assess the novelty, surprisingness, and value of Dream-OOD generated images, comparing them to real-world IMAGENET images. The results show that Dream-OOD excels in generating images perceived as novel and surprising while aligning with Boden’s concept of exploratory creativity. However, AIgenerated images often fall short in terms of value compared to real images. This study serves as a proof of concept for using Boden’s framework together with an Elo based ranking system, to evaluate AI creativity, highlighting the potential of Dream-OOD in generating novel and surprising content while acknowledging its limitations in replicating the aesthetic and emotional depth of human-created art.

The need for fair automated decision making is increasing as algorithms continue to have a growing impact on humans. Runtime fairness monitors are algorithms that detect fairness violations of fairness constraints as an algorithm is being run on real-world data, through computing statistics over observed data instances. These monitors however often assume that the input data comes from a static distribution. Our research aims to build a runtime fairness monitor for credit scoring algorithms that tracks distribution shifts in the underlying economic state to detect fairness violations more quickly. To accomplish this, we have designed a synthetic credit scoring dataset that simulates economic fluctuations. Our main contribution is the development of an adaptive fairness monitoring algorithm that dynamically adjusts to economic fluctuations, through a sliding window that disregards older, less representative samples when fluctuations in the underlying distribution our detected. We have tested our adaptive monitor against a baseline monitor that simply computes fairness metrics over all observed data instances. Our preliminary results show that our adaptive approach improves the speed of detecting fairness violations compared to the more traditional monitoring method. In conclusion, there is potential for incorporating economic state detection into credit score fairness monitoring. Future work should validate these findings with real-world data and explore additional fairness metrics.

Fairness in machine learning is an increasingly important yet complex issue, especially as these algorithms are integrated into critical decision-making processes across various sec- tors. This research focuses on the impact of features under fairness properties across multiple sectors. The primary research question ad- dressed is: “Which data features are the most sensitive when monitoring fairness properties on criminal data, and how do these features perform when monitoring fairness properties on data from different sectors?” The study ex- amines features such as age, race, gender, and educational level across datasets from criminal justice, healthcare, finance, and education sectors. Utilizing logistic regression models and a proposed dynamic monitoring algorithm, sensitivity of features to fairness violations is assessed for Demographic Parity and Equal Opportunity properties. The findings indicate that age is the most sensitive feature in almost all sectors, highlighting inherent biases and the necessity for sector-specific fair- ness considerations. However, statistical analysis revealed that these differences in sensitivity values across sectors were not statistically significant, suggesting that the observed pat- terns are not strong enough to be deemed conclusive.

Recent scandals like the dutch Toeslagenaffaire have shown the importance of fairness monitoring of machine learning models. When not careful, automated decision making models can unfairly favor groups of people and discriminate other groups. The results can be devastating for the people involved. It has been recognised that this problem requires proper research. However, most of the already conducted research looks at the problem in a static context, while almost all the real life applications are a dynamic process. Datasets are constantly increasing, and an automated decision process can have an effect on the newer entries on this dataset. A new problem then arises, when looking at these prediction tasks in a dynamic context, is the older data just as relevant as the new entries? This question can be answered by the use of fading algorithms. Fading algorithms use different methods to prioritise new data and forget old data. This paper investigates the effect of these fading algorithms on the fairness of a model. The different methods researched are an abrupt fading algorithm, a gradual fading of weight algorithm and a gradual fading of amount of data algorithm. This research resulted in the showing of importance of looking at the data in a dynamic context, observing a significant improvement on the equality of opportunity, at the cost of the efficiency of the model.

As machine learning algorithms become more and more prevalent, so do the inherent risks of unfair classification of disadvantaged or underrepresented groups. Additionally, in a dynamic context, the underlying distributions can shift over time, so corrective measures that can work in a static context may end up being detrimental in the long run. In this paper, we propose a new algorithm with the aim to improve fairness in datasets, by modifying the commonly used SMOTE algorithm in a way to work better in a dynamic context, with an added focus on fairness criteria. The results in this paper indicate that this modification, labelled as the SMICT algorithm, can be a promising approach to improving fairness, albeit with limitations and challenges that need to be considered whenever the algorithm is used.

The ability of generative AI models to accurately depict emotional expressions is crucial for their use in virtual communication and entertainment. This study eval- uates Stable Diffusion’s capability to generate context-appropriate emotional expres- sions, focusing on fear and anger. We address three key questions: (1) How accurately can Stable Diffusion generate these emotions? (2) How does the specificity of textual prompts influence accuracy? (3) Are there observable biases in the generated expres- sions? Using scenarios from the EmotionBench dataset, we designed prompts to evoke specific emotional responses and analyzed the images with GPT-4V, an advanced emo- tion recognition model. Our findings indicate that while Stable Diffusion can generate fear with reasonable accuracy, it struggles with anger, suggesting a potential bias influ- enced by training data. Additionally, the model faces challenges in depicting emotions in complex contexts, such as scenes involving a person inside a car or in dark settings. We found no consistent impact of prompt specificity on accuracy, indicating other factors may influence performance. These insights highlight the need for diverse and high-quality training data and improved evaluation frameworks. Future research should incorporate a broader range of emotions, involve human evaluators, and standardize prompt specificity to enhance the reliability and comprehensiveness of AI-generated emotional expressions. Our study underscores the importance of these improvements to develop emotionally aligned and context-aware AI systems, ultimately enhancing human-computer interactions.

As AI-generated content becomes more prevalent, the risk of generative models consuming and regenerating their own outputs in a self-consuming loop increases. This study explores the phenomenon of self-poisoning in generative models, an iterative process where AI-generated outputs are repeatedly used as input for further generations. Using a dataset of artworks by renowned artists, the process involves captioning the artworks, generating images from the captions, and repeating this cycle. The research focuses on the impact of self-poisoning on creative novelty, evaluated through content and visual novelty metrics. The findings reveal that while self-poisoning introduces novel elements in both content and visuals, it simultaneously degrades the quality of the generated artifacts over time. Generative models struggle to maintain the complexity and creativity of the original artworks, leading to outputs that converge on certain themes and realistic styles. This study contributes to a broader understanding of AI's role in art and highlights potential limitations posed by iterative generative processes.

The rise of AI-generated images presents significant challenges in distinguishing between real and fake visuals. Such fake content can disseminate false information about someone or create false identities for fraud. This study evaluates the effectiveness of the Xception model in detecting AI-generated faces, a crucial task to mitigate the misuse of facial manipulation technology. By analyzing various datasets, including iFakeFaceDB, Diverse Fake Face Dataset (DFFD), CelebA, and CASIAWebFace, we assess the model's performance in real-world scenarios. Our findings highlight the strengths of the Xception model at recognising real and synthetic images. It achieves an accuracy upwards of 97.11% on DFFD, which includes both real and synthetic images, and 96.87%-98.07% on CASIA and CelebA. Furthermore, it achieves an accuracy of 73.15% on a different synthetic facial dataset - iFakeFaceDB. This work examines the Xception model's capabilities and underscores the need for comprehensive detection methods to safeguard against the potential harms of synthetic media.

The advancement of generative models in simulating human creativity has greatly impacted the art world. In this context, artists are concerned about the devaluation of their work, especially considering the questions that appear surrounding authenticity and ownership rights. This study uses DE-FAKE, a state-of-the-art research-based detection tool, to address the crucial problem of separating AI-generated art from human-created works. The performance of DE-FAKE across a wide range of artistic styles will be assessed by carefully examining images chosen from the reputed WikiArt and AI-ArtBench datasets. This serves to highlight the advantages and disadvantages of the system. While DE-FAKE performs well in recognizing AI-generated art in the modern and abstract domains, the results show that it faces significant difficulties in recognizing more realistic styles, highlighting the need for additional development. This study lays a solid foundation for future research and practical solutions for safeguarding artists’ intellectual property in the era of AI-generated art.

The autonomous vehicle industry has the potential to revolutionize the future of driving, making the understanding of vehicle-driver interactions crucial as we progress towards fully autonomous systems. Advanced Driver Assistance Systems (ADAS) are integral in this evolution, bridging the gap between automated and manual driving. Research indicates that ADAS influences human driving behavior, impacting safety. While short-term behavioral effects are documented, long-term adaptations remain underexplored. Addressing this knowledge gap is vital for enhancing safety and optimizing ADAS performance. This thesis proposes a method to automatically detect long-term behavioral adaptations using a neural network and a novelty detection technique. The neural network, designed for driver identification, detects individual behavioral changes, while the novelty detection component classifies these changes as normal or abnormal. Key considerations include the need for a representative dataset, informative input features, a robust neural network architecture, and effective training. The system uses multiple Long Short-Term Memory (LSTM) layers and a novelty monitor to identify unfamiliar inputs. Evaluation using the CARLA simulator and the Intelligent Driver Model shows the approach's effectiveness in detecting behavioral adaptations, though improvements are needed for capturing subtle changes. The method's primary limitation is its lack of realism, suggesting future testing in dynamic, real-world conditions. Future work should also address defining 'normal' behavior in diverse scenarios and developing automatic parameter-setting techniques for real-time application.

When trains are not actively traveling on the main rail network, they to be parked and prepared for their next journey. This is a complex problem, involving several interconnected subproblems. Additionally, there is uncertainty in this environment which can render initial plans infeasible during their execution. To ensure trains are able to depart in time, having finished all their required service tasks, a schedule is created in advance.

The focus of this thesis is to address the challenges associated with generating robust initial shunting plans in an uncertain environment. This thesis focuses on a sequential problem formulation, modeled as a Markov Decision Process (MDP) and uses a policy optimized for this environment. The goal is to design a method capable of deriving robust initial shunting plans from the policy that are likely to remain feasible for a large number of possible plan executions.

The limitation addressed in this thesis, is that conventional policy-rollout techniques generate action sequences that overlook most alternative outcomes, thereby making the overall plan not feasible for a large number of plan realizations. To address this issue, the thesis proposes two distinct solution methods, aimed to consider every possible state that might be encountered, either directly or indirectly.

Through experimentation on realistically generated problem instances, the research concludes that both proposed methods significantly outperform the baseline approach, demonstrating the possibility of extracting robust initial shunting plans from a given policy that was not explicitly designed for this purpose.

Intrusion detection systems (IDSs) are essential for protecting computer systems and networks from malicious attacks. However, IDSs face challenges in dealing with dynamic and imbalanced data, as well as limited label availability. In this thesis, we propose a novel elastic gradient boosting decision tree algorithm, namely Elastic CatBoost Uncertainty (eCBU), that adapts to concept drifts and copes with label scarcity by using a novel sequential uncertainty estimation method. We compare our method with state-of-the-art techniques on synthetic and real-world datasets and show that it achieves comparable accuracy and higher robustness to limited label availability in intrusion detection tasks.

With the ever-increasing digitalisation of society and the explosion of internet-enabled devices with the Internet of Things (IoT), keeping services and devices secure is becoming more important. Logs play a critical role in sustaining system reliability. Manual analysis of logs has become increasingly difficult, accelerating the development of automated methods for log-anomaly detection. Despite significant progress in automating log analysis, current state-of-the-art methods face challenges dealing with unstable log data, which means that the content of log messages evolves over time.

We show that LogBERT, a state-of-the-art technique based on Bidirectional Encoder Representations from Transformers (BERT), cannot deal with unstable log data. On the three most prevalent publicly available log datasets, Mathew's Correlation Coefficient (MCC) score (which measures the correlation between a model's output and the correct labels) of LogBERT dropped by 90% after increasing log data instability from 1% to over 80% normal sequences containing logkeys in the test set. Log data instability was increased by only reassigning samples between the train and test set. Furthermore, we show that the high performance of LogBERT reported in the original paper was achieved because the model relied on a simple heuristic that only worked under specific conditions.
To address this issue, we propose a novel sequence anomaly detection technique based on BERT: Vocabulary-Free BERT (VoBERT). VoBERT uses a novel pre-training task we designed specifically for anomaly detection: Vocabulary-Free Masked Language Modeling (VF-MLM). We adapted traditional MLM and removed the fixed vocabulary constraint, which allows VF-MLM to classify out-of-vocabulary logkeys correctly.
We highlight that VoBERT is more stable than LogBERT and outperforms the latter in certain situations where log data is very unstable. For the public datasets, the MCC score of the specific train-test split used in the LogBERT paper dropped by 90% after reassigning the train-test split, increasing log data instability. In addition to sequence-level anomaly predictions, we evaluated all approaches on element level, providing a more granular performance assessment.
To assess the generalisation of the experimental results to real-world scenarios, we conducted a case study evaluating the anomaly detection models on real-world security event data collected at a large bank (50,000+ employees). We found that the simple heuristic did not work for this real-world data, having a negative correlation with the correct results. VoBERT showed performance on par with LogBERT on this real-world security event dataset.
We urge future researchers to evaluate their methods on real-world data, as we showed that the commonly used public datasets do not represent real-world scenarios. Furthermore, it is important to assess how difficult it is to detect anomalies in datasets used for evaluation. When a simple heuristic can perform well, such datasets might not be well suited to evaluate a complex anomaly detection model.
This thesis is a proof of concept for the novel pre-training task VF-MLM and paves the way for future work to refine this technique further, as well as to develop additional robust and adaptable solutions for log and security event anomaly detection.

Sequential decision-making problems are problems where the goal is to find a sequence of actions that complete a task in an environment. A particularly difficult type of sequential decision-making problem to solve is one in which the environment has sparse rewards, a large state space, and where the goal is to complete a complex task. In this research we create a controller that can be used to solve these types of environments in cases where the task needs to be optimized for multiple objectives. We create MOPRL, an approach that combines techniques from planning, formal methods, and reinforcement learning to synthesize such a controller. W

Imitation learning algorithms, such as AggreVaTe, have proven successful in solving many challenging tasks accurately and efficiently. In practice, however, they have not been applied quite as much. Black box policies produced by imitation learning algorithms can not ensure the safety needed for real-world applications. This paper extends this field by outputting a decision tree surrogate model from AggreVaTe and comparing it to other imitation learning algorithms (Behavioral cloning, GAIL, DAgger, Viper) in terms of interpretability as well as performance. A modification to AggreVaTe is proposed to train decision tree policies that can be used to explain individual decision-making of the model. Three simple environments of open AI Gym have been used to compare the multiple different imitation learning algorithms. The experiments reveal that on performance, AggreVaTe overall performs better than the baseline behavioral cloning but slightly worse than GAIL, DAgger and Viper. AggreVaTe performs slightly better in terms of interpretability on these simple environments. Both of these conclusions could be explained by the fewer data points used by AggreVaTe. Further study can be done into the subjective interpretability of AggreVaTe as well as more difficult environments where the extra exploring of AggreVaTe should help with finding the best solution.

Machine learning models are increasingly being used in fields that have a direct impact on the lives of humans. Often these machine learning models are black-box models and they lack transparency and trust which is holding back the implementation. To increase transparency and trust this research investigates whether imitation learning, specifically Generative Adversarial ImitationLearning (GAIL), can be used to give insights into the black-box models by extracting decision trees. To achieve this, an extension of GAIL was made allowing it to extract decision trees. The decision trees were then measured in terms of performance, fidelity, behavior, and interpretability in three different environments. We find that GAIL is able to extract decision trees with high fidelity and can give insightful information into the expert models. Moreover, further research can be done on more complex environments and black-box models, other surrogate models, and possibilities for more specific local insights.