This thesis addresses the limitations of the classical condition number-based Conjugate Gradient (CG) iteration bound in solving high-contrast heterogeneous scalar elliptic problems, particularly when employing two-level Schwarz preconditioners. The classical bound, which relies solely on the condition number of the system matrix, fails to accurately predict the convergence behavior in scenarios where the eigenspectrum of the preconditioned system exhibits pronounced clustering and spectral gaps. Motivated by this observation, the thesis develops and analyzes sharpened CG iteration bounds that incorporate detailed spectral information, offering a more nuanced and descriptive understanding of convergence.

Building on foundational work in spectral analysis and iterative solvers, the thesis introduces novel multi-cluster and tail-cluster bounds for the CG method. These bounds are derived through a combination of theoretical analysis and practical algorithms for partitioning eigenspectra, and are validated both analytically and numerically. The new bounds utilize key spectral characteristics, such as cluster condition numbers and spectral width, to more accurately estimate the number of iterations required for convergence. Numerical experiments demonstrate that the sharpened bounds can be up to 1000 times tighter than the classical bound and are effective in distinguishing the robustness of different Schwarz preconditioners.

Despite their improved accuracy, the practical application of these bounds for a priori iteration estimation is challenged by the need for detailed spectral information, which is often unavailable in the early stages of iterative solvers. The thesis discusses heuristic approaches for leveraging partial spectral data and highlights the dependency of bound accuracy on the choice of coefficient functions and preconditioners.

In conclusion, the sharpened CG iteration bounds developed in this work provide a significant advancement in predictive performance analysis for high-contrast elliptic problems. Future research directions include refining cluster partitioning algorithms, improving a priori spectral estimation, and extending the applicability of these bounds to more complex problems and preconditioners.

Physics-informed neural networks (PINNs) provide a powerful framework for solving differential equations but often encounter difficulties when addressing high-frequency solutions. Finite basis physics-informed neural networks (FBPINNs) improve PINN performance through uniform overlapping domain decomposition, yet they may still struggle with problems involving non-uniform frequency solutions. In this work, we introduce a novel framework called adaptive domain decomposition-based FBPINNs (Adaptive DD FBPINNs), which incorporates partition of unity networks (POUnets) to learn domain partitions that adaptively decompose the domain in a data-driven manner. This dynamic decomposition significantly enhances the accuracy and efficiency of PDE solvers, particularly for problems with high-frequency components and complex geometries. Furthermore, the framework integrates a residual-based adaptive distribution (RAD) resampling strategy that concentrates training on regions with high residuals, further boosting performance. Experimental results demonstrate that the Adaptive DD FBPINN outperforms standard FBPINN in terms of accuracy, providing a flexible and robust solution for both regular and complex-shaped domains, while efficiently enforcing Dirichlet boundary conditions as hard constraints. Overall, this work provides an exploratory contribution, presenting a promising approach for adaptively learning partitions by combining data-driven POUnets and FBPINNs, which can be further generalized to complex-shaped domains.

Solving large-scale linear systems derived from partial differential equations (PDEs) is an important problem in the field of scientific computing. Classical stationary iterative methods are effective at eliminating high-frequency components of the error, but struggle with low-frequency components. Deep learning-based solvers like the Deep Operator Network (DeepONet) are excellent at learning low-frequency functions but suffer from the issue of spectral bias. The Hybrid Iterative Numerical Transferable Solver (HINTS) framework was recently proposed to combine these complementary strengths. However, the original HINTS framework has a significant convergence slowdown in later iterations. This thesis reveals that this problem is primarily caused by two limitations: (1) the accumulation of mid-frequency components in the error due to the different spectral preferences between classical stationary methods and the DeepONet, and (2) a distribution shift between the low-frequency-dominated training data and the mid-frequency-dominated residuals encountered during the the iterative process in the HINTS framework.

To address these limitations, this thesis introduces two enhancement strategies. First, we propose Gradient-Enhanced HINTS (GE-HINTS), a method that incorporates first-order derivative information into the DeepONet's loss function. Motivated by the anti-frequency principle, this approach mitigates the model's spectral bias, and thus improve the performance of HINTS. Second, we develop "HINTS-in-the-loop" training strategies, which makes the DeepONet model aware of the true residual distributions it will encounter during inference. This is achieved through both an offline data augmentation strategy and an online, end-to-end differentiable training loop that optimizes the solver's multi-step performance.

Numerical experiments on benchmark problems demonstrated the effectiveness of our proposed methods. Both GE-HINTS and the HINTS-in-the-loop strategies significantly accelerate the convergence of the single-level HINTS solver. Overall, this thesis provides
both mechanistic understanding and practical strategies for accelerating the HINTS framework. We hope these insights will aid researchers seeking effective hybrid iterative solvers and will contribute to further progress in this area.

High-dimensional data imputation is a critical challenge in semiconductor metrology, where secondary measurements are often purposely omitted to optimize throughput. This thesis examines the Missing By Design (MBD) framework—an industrially motivated scenario in which data are systematically uncollected to reduce measurement overhead—and investigates a range of imputation solutions tailored to the particular complexities of wafer reflectivity and overlay. After establishing the physical, rank-deficient nature of wafer metrology data through singular-value decompositions and principal component analyses, we explore several classes of methods: linear regressions and matrix-completion techniques for baseline comparisons; deep neural-network regression (MLP) to capture nonlinearities; a contrastive-learning adaptation of CLIP for pairwise matching of primary–secondary measurements; and novel Bridge Models that refine coarse CLIP estimates with localized residual translations. Additionally, we integrate overlay-based domain constraints into CLIP via domain-guided neural network regularization (DG), ensuring physically coherent tool-to-tool (T2T) predictions. Comprehensive experiments on proprietary wafer datasets confirm that linear approaches including regressions and matrix completion methods, despite capturing the low rank structure of the data, underperform in downstream overlay and T2T prediction due to subtle nonlinear relationships. Deep neural networks offer strong reconstruction accuracy, yet demand extensive hyperparameter tuning and deeper network structures than contrastive alternatives such as CLIP-like approaches, which yield architecturally efficient, instance-based retrievals, but can lack the precision needed for rigorous overlay alignment. DG regularization, as an extension of the CLIP framework, considerably enhances T2T consistency and reduces raw reconstruction error. Meanwhile, the Bridge Model combines a CLIP-derived coarse imputation with a smaller learnable residual map between encoder domains, bridging global pairwise alignment and localized corrections for improved reconstruction and downstream tasks. Overall, this thesis presents a flexible suite of tools that advance high-dimensional MBD imputation in wafer metrology, offering valuable insights and a robust methodological foundation for future industrial applications.

The curse of dimensionality poses a fundamental challenge in autonomous negotiations: as the number of issues and their interdependencies increase, exhaustive evaluation of the outcome space quickly becomes infeasible. This thesis addresses this problem by introducing a surrogate-based method that approximates uncertain hypercubic constraint-based utility functions with quadratic polynomials. An autonomous negotiation agent can then search for high-utility outcomes in this surrogate model. The research objective was to investigate how efficiently an autonomous negotiation agent can identify high-utility bids with this approach, and how this approach compares to linear approximations and established benchmark agents.

The main contributions of this thesis are threefold. First, it introduces a probabilistic complexity measure for these hypercubic functions, capturing how parameters such as dimensionality, constraint width, the number of constraints, and the number of issues interact to shape the function's complexity. Second, it develops a novel agent that leverages a regression model with quadratic basis functions to construct a surrogate model of a hypercubic constraint-based utility function. Third, it evaluates the agent through extensive experiments, demonstrating how performance scales with complexity. Following the steps outlined in this thesis, the performance of surrogate models can be directly compared.

The results demonstrate that the surrogate-based method is a promising approach, as the agent constructed in this thesis outperforms the agents from the 2014 Automated Negotiating Agent Competition which used similar scenarios as those considered in this thesis. These agents all have in common that they directly search the utility function as opposed to a surrogate model of it. Furthermore, the results indicate that simple basis functions, such as quadratic ones, enable the agent to reach the global maximum of its utility function in low-complexity hypercubic cases, with performance scaling reasonably well up to medium complexity. Beyond this point, however, performance deteriorates rapidly, clearly signaling the need for more expressive surrogate models.

Self-Adaptive Physics-Informed Neural Networks
(SA-PINNs) are a variation of traditional Physics-Informed Neural Networks (PINNs) designed to
solve the challenges of solving ”stiff” partial differential equations (PDEs). By using adaptive weighting, SA-PINNs are able to focus their attention on areas of the domain with higher errors, therefore improving accuracy. This work investigates the
roles of individual loss components, namely residuals, boundary conditions, and initial conditions, in the performance of SA-PINNs.

Today, machine learning has an accelerated impact in quantitative finance. Current models require large amounts of data, which can be expensive. A notable area of research, physics-informed neural networks (PINNs), has proven to be effective in approximating problems that are described by partial differential equations (PDEs). During training, the PDE is embedded in the loss function and evaluated at the residual points. This allows these types of neural networks to solve problems where data is scarce or noisy. Recent studies have shown that the method for sampling residual points has a great influence on training efficiency. Residual-based adaptive distribution (RAD) sampling is the adaptive sampling method used throughout this paper. This research applies PINNs with RAD sampling to solve the Black-Scholes PDE. Here, the Black-Scholes model is used to determine the price of options in a financial market. The fundamental goal of this paper is to study the difference in training performance between non-adaptive and RAD sampling. The types of options that are being considered in this study, are the European call options and the American put options. The results shown suggest that both types of options benefit from using RAD sampling compared to non-adaptive sampling. With a loss decrease of 39.33\%, American put options improve more using RAD sampling than European call options. Although European call options still show a decrease in loss of 7.57\%.

Physics-Informed Neural Networks(PINNs) have emerged as a potent, versatile solution to solving both forward and inverse problems regarding partial differential equations(PDEs), accomplished through integrating laws of physics into the learning process. The applications of this new approach are endless, as these types of equations appear across numerous fields: fluids mechanics, quantum mechanics, electrochemistry and many others. Ever since their conception, researchers have continuously improved the flexibility and performance of PINNs through advancements in the architecture of neural networks, optimization algorithms, creative sampling methods and many more. As computational power and the interest of researchers grow, the revolutionary potential of PINNs is closer to fulfillment than ever. This paper aims to examine a small part of this evolutionary process, specifically the performance and flexibility of different activation functions used in the training of the PINN, as well as potential problems this approach could solve.

Physics-Informed Neural Networks (PINNs) are intended to solve complex problems that obey physical rules or laws but have noisy or little data. These problems are encountered in a wide range of fields including for instance bioengineering, fluid mechanics, meta-material design and high-dimensional partial differential equations (PDEs). Whilst PINNs show promising results, they often fail to converge in the presence of higher frequency components; a problem known as the spectral bias. Multiple studies have explored ways to overcome or minimize spectral bias specifically for PINNs. This paper builds on previous studies by investigating the impact of different gradient descent methods on the spectral bias.

This thesis concerns the implementation of parallel Schwarz domain decomposition using node-level parallelism, focusing on the parallel Schwarz method in comparison with the Jacobi iterative method. The study goes into the complexities of domain decomposition methods for solving partial differential equations, which are essential in fields such as fluid dynamics, solid mechanics, quantum mechanics, and financial mathematics. The research examines the convergence and performance of these methods within a parallel computing framework. A large portion of the work involves the comparison of varying configurations of block sizes and overlaps within the use of the block Jacobi iterative method, used for the implementation of the parallel Schwarz method. Numerical experiments are conducted for a stationary heat problem on a 2-dimensional grid with 256 points in each direction. The results show optimal performance for small block sizes, attributed to the use of a dense solver for the subdomains. Larger blocks and larger overlaps show superior convergence properties, up to the limit of an overlap of half the block size. The efficiency of the parallel Schwarz method remains high for an increasing number of threads unlike the standard Jacobi iteration, showing it is better suitable to a parallel environment.

Burn injuries present a significant global health challenge. Among the most severe long-term consequences are contractures, which can lead to functional impairments and disfigurement. Understanding and predicting the evolution of post-burn wounds is crucial for developing effective treatment strategies. Traditional mathematical models, while accurate, are often computationally expensive and time-consuming, limiting their practical application. Recent advancements in machine learning, particularly in deep learning, offer promising alternatives for accelerating these predictions. This study investigates the use of a deep operator network (DeepONet), a type of neural operator, as a surrogate model for finite element simulations for predicting post-burn wound evolution. We trained DeepONets on various wound shapes, enhancing the architecture by incorporating initial wound shape information and applying sine augmentation to enforce boundary conditions. The most sophisticated model achieved an R² score of 0.9960, indicating strong predictive accuracy. Additionally, the model generalised well to convex combinations of basic shapes, with an R² score of 0.9944, and provided reliable predictions over an extended period of up to one year. These findings suggest that DeepONets can effectively serve as a surrogate for traditional finite element methods in simulating post-burn wound evolution, with potential applications in medical treatment planning.

Understanding multiphase flows is critical in nuclear engineering, particularly for processes such as coolant dynamics in nuclear reactors and safety scenario analyses involving different fluid phases. Numerical simulations are a valuable tool for studying these phenomena, especially when experimental approaches are impractical due to cost or safety concerns. While direct numerical simulations (DNS) offer detailed insights, their computational expense makes them impractical for turbulent flows, necessitating the use of turbulence models for efficiency.

This thesis introduces a novel machine learning framework designed to improve Reynolds-averaged Navier-Stokes models in turbulent stratified gas-liquid flows while employing the Boussinesq approximation. The framework encompasses two methods for turbulent viscosity field inversion and introduces correction terms in the turbulence model equations to ensure an accurate prediction of the turbulent viscosity field. Through sparse symbolic regression, the framework consistently discovers models that improve the accuracy of the baseline RANS model, even in untrained flow scenarios, though further testing is needed for varied flow regimes.

Key findings include the superior performance of sparse symbolic regression models over neural network (NN) models in improving the baseline RANS model accuracy. Notably, LASSO and elastic net techniques yielded the most successful models, significantly reducing baseline errors. However, these models did not surpass the Egorov damping approach in terms of accuracy, indicating the need for further refinement.

The developed models were numerically stable and robust, which is important for practical use. However, a main limitation is that the models' accuracy during training did not always correlate with the results when coupled with the RANS equations. Moreover, data from more varied flow conditions is needed to properly assess the generalizability of the models.

Overall, this research highlights the potential of data-driven turbulence modelling to enhance two-phase flow simulations, marking a significant step forward while also identifying areas for future improvement and exploration.

We consider three topics motivated by the Network Exploration Toolkit (NEExT) for building unsupervised graph embeddings. NEExT vectorizes the graphs in a graph collection using the Wasserstein (optimal transport) distance between the distributions of node features of each graph. We inspect the effect of sampling only a proportion of the nodes of each graph, and show that even if the sampling fraction tends to zero (sufficiently slowly), so does the Wasserstein distance. NEExT relies on the assumption that local node features are informative to global characteristics of graphs. With this in mind, we consider triangle count: triangles are a local feature that correlate with the strength of a graph's community structure. We give asymptotic results for the number of triangles in a 2-community stochastic block model setting and the ABCD random graph model in both a scale-free and finite variance degree regime. Lastly, we show by experiment that augmenting the node features of graphs with triangle count improves a Graph Neural Network (GNN) in its ability to pick up on community structure and local clustering. For this, we use a random graph dataset with varying community structure, a random graph dataset with varying clustering coefficient, and a real-life dataset. We position random graph models as an effective tool for benchmarking the expressiveness of GNNs.

This thesis addresses the challenge of segmenting ultra-high-resolution images. Limitations of current approaches to segment these are that either detailed spatial contextual information is lost or many redundant computations are necessary. To overcome these issues, we propose a novel approach combining the U-Net architecture with domain decomposition strategies to balance incorporating spatial context and maintaining computational efficiency. Our proposed method partitions input images into non-overlapping patches, each processed independently on a separate device. A communication network facilitates the exchange of information between patches, enhancing the model's understanding of the spatial context.

Through theoretical analysis and practical experimentation on device memory usage during training, we demonstrate that our approach incurs minimal additional memory overhead for inter-device communication. Evaluation on synthetic and realistic datasets, including the Inria Aerial Image and DeepGlobe Satellite Segmentation datasets, demonstrates the effectiveness of our approach. Our model achieves competitive performance compared to the baseline U-Net model, with consistent class predictions around boundaries. Visualization of feature maps highlights the role of the communication network in transferring contextual information. Furthermore, it is shown that our approach remains scalable even when trained on limited subdomains.

In conclusion, our proposed model offers an intuitive solution for segmenting ultra-high-resolution images by effectively incorporating spatial context. Future research could explore variations of our model, such as overlapping subdomains or communication on different levels of the U-Net, to further enhance boundary consistency and information transfer.

In this thesis, a deep learning-based surrogate model for predicting sea ice dynamics is developed that is capable of predicting linear kinematic features in a high-resolution setting. Predicting sea ice dynamics at high resolutions is critical for understanding climate patterns and enabling safe navigation in Arctic regions. Traditional continuum models based on the viscous-plastic rheology are computationally intensive when employed at high resolutions to capture linear kinematic features (LKFs), which are narrow zones of deformation in sea ice.

A supervised learning approach was adopted, focusing on convolutional neural network architectures, specifically the U-Net and a classic bottleneck model. Various loss functions were explored, including traditional metrics like the MSE and novel domain-specific functions such as the strain rate error (SRE), which incorporates physical knowledge of sea ice behaviour. The models were trained on a dataset generated from numerical simulations of sea ice dynamics on a 2 km grid.

The key findings indicate that the U-Net architecture combined with the MSE+SRE loss function outperforms other models. This architecture's depth and use of skip connections allow it to capture complex, multi-scale patterns inherent in sea ice dynamics. The integration of the SRE into the loss function significantly enhances the model's ability to predict LKFs, demonstrating the benefit of incorporating domain knowledge into machine learning models.

While the surrogate model effectively predicts LKFs for short-term forecasts up to approximately 5.5 hours (10 time steps), errors accumulate over longer periods, leading to significant errors after about 11 hours (20 time steps). However, the efficiency gain of this surrogate model is striking: the computation of 10 time steps can be done within a second, compared to the 30 minutes that traditional numerical methods need. The study also underscores the critical importance of training data selection and preparation in influencing model performance and generalisation capabilities.

In conclusion, this work demonstrates that a deep learning-based surrogate model, particularly utilising the U-Net architecture and the MSE+SRE loss function, can effectively predict sea ice dynamics at high resolutions with significant computational efficiency. The model generates forecasts within seconds, offering a viable alternative to traditional numerical simulations that require hours of computation. Future work should focus on mitigating error accumulation to extend the forecasting horizon and exploring advanced learning methods to further integrate physical insights into the modelling process.

To reduce computational efforts, surrogate models have been developed for dune erosion prediction. Current surrogate models can describe the relationship between the XBeach input and output (Athanasiou, 2022) and provides a prediction of a morphological indicator based on a parameterized input (profile shape parameters and hydrodynamics). In this research, first steps are taken to set-up a surrogate model that is able to deal with spatial input and the prediction of actual profile shapes along the Holland Coast. Using a convolutional neural network structure (U-Net), several model set-ups are explored and scaled-up to a realistic storm scenario along the Holland Coast.

This paper examines whether complex high-dimensional data that describes the dynamics of a cantilever beam can be transformed into a less complex system. In particular, the transformation that is examined is the reduction of the dimension. An essential aspect of this study involves the implementation of a linear autoencoder, which is a type of machine learning model that possesses the capability to effectively reduce the dimensionality of input data while adeptly reconstructing the original dataset. The model performs well and is successful in reconstructing complex data via the less complex system. However, the model struggles if the dynamics are made more complex by adding an external force. Although the dynamics seem to be present in the results, the amplitudes differ.

Commercial finite element software like COMSOL is build to be user-friendly. For example, the user does not have to find weak formulations, or discretise the partial differential equations by hand. One of the more difficult parts of using finite element software is deciding which solver method and solver parameters to choose, as the performance of the solver, and hence the performance of the whole simulation, depends on it.
Pseudo time-stepping is a stabilisation method that is designed to lead to a stationary solution for a wider range of initial guesses compared to traditional Newton methods. During pseudo time-stepping, a CFL number is used. This CFL number is adapted each nonlinear iteration. In COMSOL the CFL number either depends on the iteration count, or the nonlinear error estimate and previous CFL number. In this case, the CFL number is a global value, but the pseudo time-step itself is local since it also depends on the mesh cell size.

In this research, a neural network is created to predict a local CFL number for pseudo time-stepping,
such that the network accelerates convergence compared to the two CFL numbers for pseudo time-stepping in COMSOL and is generalisable. The convergence is assumed to be accelerated if the
network predictions result in convergence in fewer nonlinear iterations compared to solvers that use
each one of the two CFL numbers in COMSOL. The network is assumed to be generalisable if it is able to accelerate the convergence for different laminar flow problems in COMSOL on which the network has not been trained.
A network with 2 hidden layers with each 16 neurons is trained on local data from element patches in order to make it generalizable. The element patch data points contain information of the central triangular element vertices, and of the vertices of the three adjacent elements. The local data consist of the velocities, pressure and residuals, as well as the cell Reynolds number and the element edge lengths. The loss of this network is the root mean squared error between the network output and a previously computed CFL target. This target is the optimised value for the local CFL number such
that the difference between the solution obtained from the pseudo time-step and the exact solution is
minimised.
The network predictions were able to accelerate convergence compared to to the two CFL numbers in COMSOL for several different 2D laminar flow simulations. Therefore, the conclusion is that a network trained on optimised local CFL numbers is able to generalize well and accelerate the existing pseudo time-stepping method.

The goal of this work is to evaluate the aptness of generative adversarial networks (GANs) for use as surrogate reduced order fluid models. In contrast to previously published work, the focus is placed on analyzing the specific effect of adversarial training, by comparing GAN outcomes with those from an identical generator network trained directly on ground truth (using an L1 loss). A dataset of 10000 simulated examples of stationary flow through a 2D sudden expansion geometry containing a polygonal obstacle was created, alongside two additional datasets for testing generalization. The simulation data was interpolated to a regular image grid, and the neural networks were trained to predict the velocity field based on an image encoding the geometry.
The gathered experimental data show clearly that adversarial training cannot reach the same accuracy as direct training. This was found to be true on unseen examples from the training distribution, as well as on geometries of unfamiliar type. On the other hand, GAN outcomes tend to "appear" more realistic, and exhibit a lower continuity residual. The qualitative differences were highlighted by considering bifurcation scenarios which were purposefully included in the data set. When the bifurcation parameter is at its critical value, two very different flow scenarios can occur essentially randomly. In such cases, the GAN essentially predicts just one of the possible flow outcomes, whereas the directly trained model outputs a superposition of both. This exemplifies a fundamental difference in prediction behavior.
It is shown that these results can be well understood as a direct consequence of the different cost functions used during training. Furthermore, it is demonstrated that by using a sum of both types of loss functions (adversarial & L1), advantages of both models can be combined.
In other results, the data show that the discriminator output cannot provide a reliable indication of the accuracy of a prediction, since no robust correlation between them was found. Also, it was observed that the discriminator appeared to dominate the adversarial game, and it was shown that improving this balance could lead to better results. Moreover, an investigation into predicting pressure alongside the velocity field was conducted. Results showed that adding an additional channel for pressure to the network architecture can achieve this goal, but is not necessary, as calculating the pressure field from the velocity output produces results of similar quality. In terms of resources, GAN training required a relative increase in computational time by approx. 50%. Additionally, GAN models were also found to take many more training steps to reach convergence. Similarly, for a fixed number of steps, results showed that directly trained models can benefit more from larger datasets. In conclusion, GAN models are likely not the right choice for reduced order modeling in scientific contexts due to their lower accuracy. However, they could hold potential for use in creating visual effects or as an added regularizer during training.