Deformable Image Registration (DIR) is a medical imaging process involving the spatial alignment of two or more images using a transformation model that can account for non-rigid deformations. B-spline-based transformation models have emerged as a common approach to express such spatial alignments. However, without additional measures, their flexibility can lead to physically implausible deformations. This flexibility has motivated the inclusion of penalty terms to improve the smoothness and regularity of the transformation. Determining an appropriate weight for this penalty term is difficult, as each registration problem requires a different trade-off between this penalty term and the quality of the transformed image. Gradient-based methods are commonly used as optimization methods in medical image registration toolboxes due to their computational efficiency and fast convergence rates. However, due to their gradient-based approach, they may converge prematurely in local minima. In this thesis, we investigate the efficacy of a gradient-less alternative: the Real-Valued Gene-pool Optimal Mixing Evolutionary Algorithm (RV-GOMEA), a population-based method that can exploit the problem structure of optimization problems through explicit mappings of dependencies between problem variables. To improve the computational efficiency of RV-GOMEA when applied to DIR, we show how to apply partial evaluations for common image similarity metrics and penalty terms when using B-spline-based transformation models. We test RV-GOMEA on a synthetic registration problem to better understand its performance in the context of DIR. Based on our findings, we propose several methods that hybridize RV-GOMEA with a gradient-based method and impose specific constraints on the B-spline-based transformation model. We validate the performance of these methods on clinical registration problems and find that RV-GOMEA with a gradient-based local search operator can provide significant benefits over purely gradient-based methods for DIR problems. Additionally, placing specific constraints on the transformation model can increase the regularity of transformations without requiring a penalty term.

Deformable Image Registration (DIR) is a process in which the point-to-point correspondence between two or more medical images is estimated. This could allow spatial data to be transferred between these images, easing the work of practitioners in the field of radiation oncology. Many DIR approaches already exist. These are not yet applicable in all clinical settings, however, as DIR is a complex problem, partly because of the difficulty of evaluating image registration results. This is why often multiple objectives are considered in this evaluation. In this thesis, we combine two separate multi-objective DIR approaches into hybrids while evaluating their performances, with the aim to improve upon the individual approaches, thus reducing the gap toward clinical practice. In these hybrids, we also use an approach called Automated Landmark Detection which automatically detects corresponding landmarks between the source and target images. The first DIR approach, the Digital Phantom, creates a biomechanical model of the patient and uses an Evolutionary Algorithm (EA) to optimize parameters of a Finite Element Method (FEM)-based simulation. The second DIR approach, MOREA, creates a dual-dynamic biomechanical transformation model of the patient, using an EA to optimize its parameters. In the first hybrid approach, called the Landmark Guided Digital Phantom (LGDP), the Automated Landmark Detection approach is used to automatically find corresponding landmarks, which are then used as a guidance objective in the Digital Phantom. The second hybrid is called DP-MOREA, where the output of the Digital Phantom is used to initialize deformations in the MOREA approach. The final hybrid is called LGDP-MOREA. This hybrid combines all three components, using the generated landmarks to guide the Digital Phantom and initializing MOREA with the result. We analyze the effect of these hybrids on the quality of the final registration, both quantitatively and qualitatively. Additionally, we compare these hybrids with the components they are composed of. We test the performance of the approaches and hybrids on pelvic CT scans of three cervical cancer patients with large deformations. We observe that the individual Digital Phantom component outperforms the LGDP hybrid both quantitatively and qualitatively on these three patients. In addition, we observe that initializing MOREA with the output of the Digital Phantom and LGDP improves on how fast the optimization converges, that it improved the end results quantitatively and that it also has an effect on the resulting Deformation Vector Fields (DVFs). We conclude that in future research, these effects on the DVFs and their clinical relevance should be studied in more detail.