Machine learning (ML) models are used increasingly in high-stakes areas such as health and finance because of their strong performance. However, having good performance in metrics such as accuracy or the f1 score alone is not all that is important as trust is also essential in these areas. TalkToModel is a system that addresses this challenge, using a large language model (LLM), by letting the users interact with their models through natural language. However, this system only allows a user to ask black-box questions. Another way to achieve trust is through models that are constructed with interpretability in mind that a human can understand. Genetic programming (GP) models are such models that have the potential to be interpreted. This thesis investigates if GP models can be made even more interpretable
using TalkToModel. To do this an enhanced version of TalkToModel called TalkToGP is created with three main contributions: 1) Integration of GP Models into TalkToModel, 2) the ability to ask GP modelspecific questions and 3) the ability to do a comparative analysis between multiple GP models. This system is built using the feedback from GP users who gave insights on their experience with GP as well as their wishes for this system. In the end, the system is evaluated by GP users in an experiment. The experiments showed that the enhanced version of TalkToModel shows a strong indication that it increases the interpretability of GP models. This means the system could be a useful tool for anyone working with GP models.

Deformable Image Registration (DIR) is a medical imaging process involving the spatial alignment of two or more images using a transformation model that can account for non-rigid deformations. B-spline-based transformation models have emerged as a common approach to express such spatial alignments. However, without additional measures, their flexibility can lead to physically implausible deformations. This flexibility has motivated the inclusion of penalty terms to improve the smoothness and regularity of the transformation. Determining an appropriate weight for this penalty term is difficult, as each registration problem requires a different trade-off between this penalty term and the quality of the transformed image.

Gradient-based methods are commonly used as optimization methods in medical image registration toolboxes due to their computational efficiency and fast convergence rates. However, due to their gradient-based approach, they may converge prematurely in local minima. In this thesis, we investigate the efficacy of a gradient-less alternative: the Real-Valued Gene-pool Optimal Mixing Evolutionary Algorithm (RV-GOMEA), a population-based method that can exploit the problem structure of optimization problems through explicit mappings of dependencies between problem variables. To improve the computational efficiency of RV-GOMEA when applied to DIR, we show how to apply partial evaluations for common image similarity metrics and penalty terms when using B-spline-based transformation models.

We test RV-GOMEA on a synthetic registration problem to better understand its performance in the context of DIR. Based on our findings, we propose several methods that hybridize RV-GOMEA with a gradient-based method and impose specific constraints on the B-spline-based transformation model. We validate the performance of these methods on clinical registration problems and find that RV-GOMEA with a gradient-based local search operator can provide significant benefits over purely gradient-based methods for DIR problems. Additionally, placing specific constraints on the transformation model can increase the regularity of transformations without requiring a penalty term.

The Real-Valued Gene-pool Optimal Mixing Evolutionary Algorithm (RV-GOMEA) is a state-of-the-art algorithm for single-objective, real-valued optimization. As many practical applications are inherently constrained, evolutionary algorithms are equipped with constraint handling techniques to allow optimizing constrained problems. The approach currently in use with RV-GOMEA prioritizes solution feasibility over the objective value in all cases, pressuring the algorithm to find feasible solutions. However, this can be inefficient if the constrained optimum is located at the constraint boundary, as search is discouraged from exploring the search space close to infeasible solutions.

In this thesis, several well-known constraint handling techniques from literature are adapted for use with RV-GOMEA and evaluated on different benchmark problems, identifying the strengths and limitations of the various techniques. Furthermore, the inefficiency of the current technique is investigated in detail. Based on the insights gained, modifications to the existing techniques are proposed, leading to promising preliminary results.

Even if a Multi-modal Multi-Objective Evolutionary Algorithm (MMOEA) is designed to find all locally optimal approximation sets of a Multi-modal Multi-objective Optimization Problem (MMOP), there is a risk that the found approximation sets are not smoothly navigable because the solutions belong to various niches, which reduces the insight for decision makers. Moreover, when the multi-modality of MMOPs increases, this risk grows and the trackability of finding all locally optimal approximation sets decreases. One example where this issue occurs is that of High-Dose-Rate (HDR) brachytherapy for prostate cancer. In HDR brachytherapy a treatment plan is to be optimized that irradiates a tumour with a prescribed dose, whilst sparing all of the healthy organs surrounding the tumour. The radiation is administered through a radioactive source that is stopped at certain dwell positions within a set of hollow catheters that have been implanted into the patient. In a treatment plan, each of the dwell positions is given a specific dwell time for which the source is kept at that location in order to irradiate the surrounding tissue.
To tackle the navigability issues, two new MMOEAs are proposed: Multi-Modal Bézier Evolutionary Algorithm (MM-BezEA) and Set Bézier Evolutionary Algorithm (Set-BezEA). Both MMOEAs produce approximation sets that cover individual niches and exhibit inherent decision-space smoothness as they are parameterized by Bézier curves. MM-BezEA combines the concepts behind the recently introduced BezEA and MO-HillVallEA to find all locally optimal approximation sets. Set-BezEA employs a novel multi-objective fitness function formulation to find limited numbers of diverse, locally optimal, approximation sets for MMOPs of high multi-modality.
Both algorithms, but especially MM-BezEA, are found to outperform the MMOEAs MO_Ring_PSO_SCD and MO-HillVallEA on MMOPs of moderate multi-modality with linear Pareto sets. Moreover, for MMOPs of high multi-modality, Set-BezEA is found to indeed be able to produce high-quality approximation sets, each pertaining to a single niche. Set-BezEA is also shown to be comparable to the current BRIGHT approach used in the Amsterdam UMC for the optimization of treatment plans for prostate cancer HDR brachytherapy, which opens the way for it to be introduced in the clinical practice in the future.

Neural networks (NNs) have, in recent years, become a major part of modern pattern recognition, and both theoretical and applied research evolve at an astounding pace. NNs are usually trained via gradient descent (GD), but research has shown that GD is not always capable of training very small networks. As a result, networks trained via GD are often significantly larger than necessary, demanding more computing power and energy to evaluate, and thereby hampering adoption on lower- power devices. This thesis investigates whether evolutionary algorithms (EAs) can successfully train NNs and if these networks can be smaller than those required by GD. Four algorithms, namely the GD-based Adam, Adam with cold restarts, and the EAs GOMEA and BIPOP-CMA-ES, were used to train various configurations of multilayer perceptrons (MLPs) with one hidden layer for the Exclusive-OR (XOR) problem. Their relative performance was gauged by comparing the rates at which each algorithm attained an acceptable loss for a given number of hidden nodes. The main findings are that EAs could find smaller ReLU-activated networks than GD could, while, oppositely, the GD could generally find smaller networks for sigmoid- activated networks. However, GOMEA in particular was able to successfully train XOR networks using the highly discrete Heaviside activation function, whereas GD could not due gradient erasure. Problem-specific knowledge in the form of a soft symmetry breaking constraint was found to be effective to increase success rates in a limited number of cases. These findings indicate that using EAs is a viable strategy for training NNs, yielding smaller, and therefore more efficient networks than those trained via GD, provided that the network topology is suitable for the chosen EA. This opens up future research into, including but not limited to, the many ways EAs can be scaled up to training larger NNs, hybridization with GD, and niche network topologies.

Computer vision tasks, like supervised image classification, are effectively tackled by convolutional neural networks, provided that the architecture, which defines the structure of the network, is set correctly. Neural Architecture Search (NAS) is a relatively young and increasingly popular field that is concerned with automatically optimizing the architecture of neural networks. Previously known work shows that even though a recent trend has been to develop increasingly complex search strategies for NAS, several search strategies do not significantly outperform simple approaches like randomly sampling from the search space on single-objective NAS tasks. Additionally, proper ablation studies are often missing. Therefore, it is currently uncertain at best which mechanisms are key for an algorithm to have to achieve excellent NAS performance. In the first part of this thesis, Local Search (LS) and a differently biased form of random search, are proposed for multi-objective (MO) NAS. The multi-objective version of NAS is studied less and understanding the trade-off between between multiple objectives for architectures is arguably more interesting. We find that very simple algorithms can achieve search performance close to that of state-of-the-art evolutionary algorithms (EAs), while outperforming plain random search. Additionally, we find that the quality of the set of architectures found by LS is similar to the those found by the EAs, if compared with respect to test accuracy. Nevertheless, from the compared search strategies the Multi-Objective Gene-pool Optimal Mixing Evolutionary Algorithm (MO-GOMEA), a state-of-the-art model-based EA, achieves the best performance. In the second part of this thesis, it is explored which mechanisms are essential for MO-GOMEA to achieve an excellent search performance for NAS spaces. We find that the automatic population-sizing scheme of MO-GOMEA offers a welcome anytime-performance, but objective space clustering has only a small beneficial impact. The number of clusters can be set arbitrarily. Special (extreme) clusters that optimize for one objective only can be enabled to the practitioner’s preference, resulting in different search behaviors. The improvement in performance gained by automatically detecting and exploiting dependencies within architectures is limited: this model-based aspect of MO-GOMEA seems only helpful for finding highly accurate networks.

Model-based evolutionary algorithms (MBEAs) are praised for their broad applicability to black-box optimization problems. In practical applications however, they are mostly used to repeatedly optimize different instances of a single problem class, a setting in which specialized algorithms generally perform better. In this paper, we introduce the concept of a new type of MBEA that can automatically specialize its behavior to a given problem class using tabula rasa self-learning. For this, reinforcement learning (RL) is a naturally fitting paradigm. A proof-of-principle framework, called SL-ENDA, based on estimation of normal distribution algorithms in combination with reinforcement learning is defined. SL-ENDA uses an RL-agent to decide upon the next population mean while approaching the rest of the algorithm as the environment. A comparison of SL-ENDA to AMaLGaM and CMA-ES on unimodal noiseless functions shows mostly comparable performance and scalability to the broadly used and carefully manually crafted algorithms. This result, in combination with the inherent potential of self-learning model-based evolutionary algorithms with regard to specialization, opens the door to a new research direction with great potential impact on the field of model-based evolutionary algorithms.