Inverse problems governed by partial differential equations (PDEs) are ill-posed, and responsible use of their solutions requires quantifying the uncertainty in recovered parameters. Neural operator methods for inverse problems offer fast surrogates for classical solvers, but placing posteriors over network weights is intractable at scale. This thesis extends the Inverse Generative Neural Operator (IGNO) to full Bayesian posterior sampling by adding a normalising flow prior term to the inversion objective and replacing gradient-based optimisation with the No-U-Turn Sampler (NUTS). The extension requires no retraining of any network component. We evaluate the method on four inverse problems spanning Darcy flow, electrical impedance tomography (EIT), and the viscous Burgers equation. On in-domain test instances, the posterior achieves 93% to 100% empirical coverage at the 95% nominal level across all four benchmarks and responds appropriately to changes in observation noise and sensor count. The posterior mean matches or improves on the maximum a posteriori (MAP) point estimate in every case. A Laplace approximation baseline, which fits a Gaussian posterior at the MAP estimate, fails on two of the four problems and does not consistently outperform NUTS on the two where it converges. Because the posterior formulation separates data, physics, and prior into additive terms, physical constraints can be incorporated during sampling alongside the data likelihood. Including PDE residuals as a virtual likelihood is most beneficial when observations alone leave the posterior under-determined, as demonstrated by EIT, where boundary-only measurements provide no direct information about the interior conductivity. The uncertainty estimates are unreliable for out-of-distribution coefficient fields. The learned prior pulls the posterior toward the training distribution, producing credible intervals that can be both narrow and wrong. ... Read more

In this thesis, we study spectral bias, the tendency of gradient-based training to learn the low-frequency part of a target before its high-frequency part. We work in a setting simple enough to analyse explicitly: regression on the unit circle with a shallow
ReLU network. In the infinite-width limit, the residual dynamics are governed by the Neural Tangent Kernel. Under the uniform measure on the circle this kernel depends only on the angle between two points, so the associated operator is a convolution and the Fourier modes are its eigenfunctions, each decaying at a rate set by its eigenvalue, and the lower the frequency, the larger the eigenvalue, so low frequencies are learned first.

Away from this idealised limit the picture degrades only gradually. On a fixed low-frequency subspace, both finite sampling and frozen finite width keep the operator close to the continuum Fourier prediction, with error of order O(n^(-1/2)) in the sample size n and O(m^(-1/2)) in the width m. The description breaks only once the kernel is allowed to evolve during training. At small width the evolving kernel reaches a lower loss by strengthening its lowest-frequency components, even as its alignment with the Fourier basis fails to improve. This reinforces the low-frequency bias rather than approximating the fixed-kernel dynamics. A formal theory of this evolving-kernel regime remains the main open problem. ... Read more

Machine-learned surrogate time integrators promise large speed-ups over classical solvers, yet their performance is usually reported as a single aggregate error, leaving open the question of when they remain stable. This thesis determines the empirical stability limits of graph neural network (GNN) surrogates for the two-dimensional advection–diffusion equation on unstructured meshes with periodic boundary conditions, expressed directly in the dimensionless CFL and Fourier numbers.

Surrogates are trained to minimise the one-step error on fixed velocity and diffusion fields and evaluated autoregressively on unseen fields over horizons eight times the training window. Two families are compared at a fixed message passing budget: single-scale models, and multiscale models organised as V-cycles over predetermined coarsened graphs. For each model, a piecewise-linear fit of the final rollout error against the CFL and Fourier numbers yields empirical stability limits, defined by a blow-up threshold.

Within these limits the surrogates reproduce the finite element reference accurately on both seen and unseen fields and show no abrupt change beyond the training horizon, although the diffusion-dominated regime is consistently harder than the advection-dominated one. The single-scale CFL limit tracks the number of message-passing blocks and lies slightly above it. Adding coarse levels at a fixed total message passing layer budget broadens the advective stability range, decisively at the largest stride, but a two-level hierarchy trades diffusive stability and in-region accuracy for this gain. Only a three-level V-cycle removes the penalty, attaining zero blow-ups on both axes, and deeper models show no oversmoothing.

The diffusion-side limits carry large variance, traced partly to the dissipative backward-Euler reference, and should be read as indicative. The work delivers a concrete operational range for GNN surrogates and identifies how multiscale models can extend it. ... Read more

Operator learning has recently emerged as an effective approach for approximating mappings between function spaces. In this work, we study operator learning using wavelet representations (OLWavelet). It represents functions via discrete wavelet transforms (DWT), uses a neural network to learn mappings between the resulting wavelet coefficients, and reconstructs the output functions from the predicted coefficients to learn operators between functions. Several representative operators are considered, including 1D translation operators, operators mapping different source terms of 1D Poisson equations to the corresponding solutions, operators mapping different initial
conditions of an 1D Diffusion euqation and an 1D Burgers’ equation to their terminal solutions, and operators approximating functions with piecewise functions on dyadic partitions. We analyze the sources of error in the proposed framework by decomposing the total error into the neural network generalization error and the reconstruction error. This perspective provides insight into how different components of the model contribute to the final prediction accuracy. In a series of numerical experiments, we compare models trained with different numbers of DWT coefficients for function representation, motivated by the fact that functions can be well approximated using only a subset of DWT coefficients, which also reduces computational cost. The experimental results show that our model achieves higher prediction accuracy than comparable models based on Fourier transforms on certain tasks. Moreover, we observe a non-monotonic relationship between the model’s prediction accuracy and the number of DWT coefficients used. In addition, experiments show that using a single neural network to learn the mappings among all wavelet coefficients is often more accurate than using multiple neural networks to separately learn the mappings of wavelet coefficients at different decomposition levels. Overall, the study demonstrates the effectiveness of the proposed model and analyzes the sources of its errors, thereby revealing its strengths and limitations. ... Read more

The real-time optimization of district heating (DH) networks is computationally demanding due to the strong interdependence between their state variables. This thesis investigates whether a Reduced Order Model (ROM) can improve the computational efficiency of state estimation in DH networks. Existing ROM approaches typically rely on simplified physical assumptions or require manual, topology-dependent modifications to the model architecture, limiting their applicability and preventing fully automated retraining.

To address these limitations, this study proposes a method based exclusively on Full Order Model (FOM) data that follows a uniform training pipeline applicable to any DH network without requiring manual analysis of the network topology or case-specific architectural adjustments. A hybrid framework based on Proper Orthogonal Decomposition (POD) is developed, in which POD extracts dominant spatial modes from high-dimensional FOM data, while a feedforward neural network predicts the corresponding temporal coefficients from compressed input features. The ROM output is subsequently used as an initial guess for the FOM state iteration procedure, thereby preserving physical consistency.

The approach is evaluated on two realistic DH networks of different scales. In both cases, the ROM achieves total relative reconstruction errors below 5% (4.8% for the smaller network and 3.6% for the larger network), with prediction times below 0.1 seconds compared to approximately 100 seconds for a single FOM iteration. For the smaller network, integrating the ROM into the optimization workflow results in a 1.17× speed-up while producing decision variables nearly identical to those obtained with the FOM. This improvement arises from skipping the first FOM iteration, reducing the number of iterations required for convergence, and updating fewer time steps per iteration. For the larger network, the ROM maintains high predictive accuracy but performs less reliably during optimization, likely due to limited training data for rarely activated backup sources. Overall, the results demonstrate that hybrid POD-based ROMs can significantly improve the computational efficiency of DH network state estimation and optimization, provided that the training dataset adequately represents all relevant operational regimes. ... Read more

This thesis addresses the limitations of the classical condition number-based Conjugate Gradient (CG) iteration bound in solving high-contrast heterogeneous scalar elliptic problems, particularly when employing two-level Schwarz preconditioners. The classical bound, which relies solely on the condition number of the system matrix, fails to accurately predict the convergence behavior in scenarios where the eigenspectrum of the preconditioned system exhibits pronounced clustering and spectral gaps. Motivated by this observation, the thesis develops and analyzes sharpened CG iteration bounds that incorporate detailed spectral information, offering a more nuanced and descriptive understanding of convergence.

Building on foundational work in spectral analysis and iterative solvers, the thesis introduces novel multi-cluster and tail-cluster bounds for the CG method. These bounds are derived through a combination of theoretical analysis and practical algorithms for partitioning eigenspectra, and are validated both analytically and numerically. The new bounds utilize key spectral characteristics, such as cluster condition numbers and spectral width, to more accurately estimate the number of iterations required for convergence. Numerical experiments demonstrate that the sharpened bounds can be up to 1000 times tighter than the classical bound and are effective in distinguishing the robustness of different Schwarz preconditioners.

Despite their improved accuracy, the practical application of these bounds for a priori iteration estimation is challenged by the need for detailed spectral information, which is often unavailable in the early stages of iterative solvers. The thesis discusses heuristic approaches for leveraging partial spectral data and highlights the dependency of bound accuracy on the choice of coefficient functions and preconditioners.

In conclusion, the sharpened CG iteration bounds developed in this work provide a significant advancement in predictive performance analysis for high-contrast elliptic problems. Future research directions include refining cluster partitioning algorithms, improving a priori spectral estimation, and extending the applicability of these bounds to more complex problems and preconditioners. ... Read more

Physics-informed neural networks (PINNs) provide a powerful framework for solving differential equations but often encounter difficulties when addressing high-frequency solutions. Finite basis physics-informed neural networks (FBPINNs) improve PINN performance through uniform overlapping domain decomposition, yet they may still struggle with problems involving non-uniform frequency solutions. In this work, we introduce a novel framework called adaptive domain decomposition-based FBPINNs (Adaptive DD FBPINNs), which incorporates partition of unity networks (POUnets) to learn domain partitions that adaptively decompose the domain in a data-driven manner. This dynamic decomposition significantly enhances the accuracy and efficiency of PDE solvers, particularly for problems with high-frequency components and complex geometries. Furthermore, the framework integrates a residual-based adaptive distribution (RAD) resampling strategy that concentrates training on regions with high residuals, further boosting performance. Experimental results demonstrate that the Adaptive DD FBPINN outperforms standard FBPINN in terms of accuracy, providing a flexible and robust solution for both regular and complex-shaped domains, while efficiently enforcing Dirichlet boundary conditions as hard constraints. Overall, this work provides an exploratory contribution, presenting a promising approach for adaptively learning partitions by combining data-driven POUnets and FBPINNs, which can be further generalized to complex-shaped domains. ... Read more

Solving large-scale linear systems derived from partial differential equations (PDEs) is an important problem in the field of scientific computing. Classical stationary iterative methods are effective at eliminating high-frequency components of the error, but struggle with low-frequency components. Deep learning-based solvers like the Deep Operator Network (DeepONet) are excellent at learning low-frequency functions but suffer from the issue of spectral bias. The Hybrid Iterative Numerical Transferable Solver (HINTS) framework was recently proposed to combine these complementary strengths. However, the original HINTS framework has a significant convergence slowdown in later iterations. This thesis reveals that this problem is primarily caused by two limitations: (1) the accumulation of mid-frequency components in the error due to the different spectral preferences between classical stationary methods and the DeepONet, and (2) a distribution shift between the low-frequency-dominated training data and the mid-frequency-dominated residuals encountered during the the iterative process in the HINTS framework.

To address these limitations, this thesis introduces two enhancement strategies. First, we propose Gradient-Enhanced HINTS (GE-HINTS), a method that incorporates first-order derivative information into the DeepONet's loss function. Motivated by the anti-frequency principle, this approach mitigates the model's spectral bias, and thus improve the performance of HINTS. Second, we develop "HINTS-in-the-loop" training strategies, which makes the DeepONet model aware of the true residual distributions it will encounter during inference. This is achieved through both an offline data augmentation strategy and an online, end-to-end differentiable training loop that optimizes the solver's multi-step performance.

Numerical experiments on benchmark problems demonstrated the effectiveness of our proposed methods. Both GE-HINTS and the HINTS-in-the-loop strategies significantly accelerate the convergence of the single-level HINTS solver. Overall, this thesis provides
both mechanistic understanding and practical strategies for accelerating the HINTS framework. We hope these insights will aid researchers seeking effective hybrid iterative solvers and will contribute to further progress in this area. ... Read more

High-dimensional data imputation is a critical challenge in semiconductor metrology, where secondary measurements are often purposely omitted to optimize throughput. This thesis examines the Missing By Design (MBD) framework—an industrially motivated scenario in which data are systematically uncollected to reduce measurement overhead—and investigates a range of imputation solutions tailored to the particular complexities of wafer reflectivity and overlay. After establishing the physical, rank-deficient nature of wafer metrology data through singular-value decompositions and principal component analyses, we explore several classes of methods: linear regressions and matrix-completion techniques for baseline comparisons; deep neural-network regression (MLP) to capture nonlinearities; a contrastive-learning adaptation of CLIP for pairwise matching of primary–secondary measurements; and novel Bridge Models that refine coarse CLIP estimates with localized residual translations. Additionally, we integrate overlay-based domain constraints into CLIP via domain-guided neural network regularization (DG), ensuring physically coherent tool-to-tool (T2T) predictions. Comprehensive experiments on proprietary wafer datasets confirm that linear approaches including regressions and matrix completion methods, despite capturing the low rank structure of the data, underperform in downstream overlay and T2T prediction due to subtle nonlinear relationships. Deep neural networks offer strong reconstruction accuracy, yet demand extensive hyperparameter tuning and deeper network structures than contrastive alternatives such as CLIP-like approaches, which yield architecturally efficient, instance-based retrievals, but can lack the precision needed for rigorous overlay alignment. DG regularization, as an extension of the CLIP framework, considerably enhances T2T consistency and reduces raw reconstruction error. Meanwhile, the Bridge Model combines a CLIP-derived coarse imputation with a smaller learnable residual map between encoder domains, bridging global pairwise alignment and localized corrections for improved reconstruction and downstream tasks. Overall, this thesis presents a flexible suite of tools that advance high-dimensional MBD imputation in wafer metrology, offering valuable insights and a robust methodological foundation for future industrial applications. ... Read more

This thesis investigates how preconditioned Krylov subspace methods perform and scale under shared-memory parallelism. The focus is on the Conjugate Gradient (CG) method for symmetric positive definite systems and the Generalized Minimal Residual (GMRES) method for non-symmetric systems. Both solvers are implemented in PyKokkos and applied to finite element discretisation generated with NGSolve. We look at a scalar Laplace problem, a Stokes-like vector problem, and a steady Stokes flow around a NACA,2412 airfoil.For CG, we study Jacobi and symmetric Gauss–Seidel (SGS) preconditioning, strong and weak scaling up to 16 threads, and kernel-level timings. As the mesh is refined, the iteration count grows in line with the increasing condition number of the stiffness matrix. Jacobi reduces the iteration count only slightly but is cheap and fully parallel, leading to runtimes similar to, and sometimes slightly better than, unpreconditioned CG. SGS roughly halves the iteration count, but its forward/backward sweeps are largely sequential, which limits speed-up on many cores and can make SGS slower overall despite faster convergence.For GMRES we analyse the influence of the restart parameter, preconditioning, and polynomial order. Higher-order vector elements lead to more off-diagonal entries in the system matrix, here scalar Jacobi becomes too weak and can even make restarted GMRES slower than using no preconditioner. SGS remains effective in terms of iterations, but this comes with the same parallel limitations as in CG. And overall GMRES shows poor strong and weak scaling on the tested CPUs. For the NACA Stokes system, Jacobi and SGS preconditioning fails, whereas a block preconditioner that respects the velocity–pressure structure shows rapid convergence.Overall, the results show that good performance on shared-memory architectures requires preconditioners that both respect the block structure of the PDE and are highly parallelizable.
Github: https://github.com/Hugoreijersen/Krylov-Subspace-Methods.git ... Read more

The curse of dimensionality poses a fundamental challenge in autonomous negotiations: as the number of issues and their interdependencies increase, exhaustive evaluation of the outcome space quickly becomes infeasible. This thesis addresses this problem by introducing a surrogate-based method that approximates uncertain hypercubic constraint-based utility functions with quadratic polynomials. An autonomous negotiation agent can then search for high-utility outcomes in this surrogate model. The research objective was to investigate how efficiently an autonomous negotiation agent can identify high-utility bids with this approach, and how this approach compares to linear approximations and established benchmark agents.

The main contributions of this thesis are threefold. First, it introduces a probabilistic complexity measure for these hypercubic functions, capturing how parameters such as dimensionality, constraint width, the number of constraints, and the number of issues interact to shape the function's complexity. Second, it develops a novel agent that leverages a regression model with quadratic basis functions to construct a surrogate model of a hypercubic constraint-based utility function. Third, it evaluates the agent through extensive experiments, demonstrating how performance scales with complexity. Following the steps outlined in this thesis, the performance of surrogate models can be directly compared.

The results demonstrate that the surrogate-based method is a promising approach, as the agent constructed in this thesis outperforms the agents from the 2014 Automated Negotiating Agent Competition which used similar scenarios as those considered in this thesis. These agents all have in common that they directly search the utility function as opposed to a surrogate model of it. Furthermore, the results indicate that simple basis functions, such as quadratic ones, enable the agent to reach the global maximum of its utility function in low-complexity hypercubic cases, with performance scaling reasonably well up to medium complexity. Beyond this point, however, performance deteriorates rapidly, clearly signaling the need for more expressive surrogate models. ... Read more

Accurate modeling of vacuum dynamics in Trailing Suction Hopper Dredgers (TSHDs) is critical for optimizing suction production and mitigating sensor anomalies. This study proposes a data-driven, physics-guided operator learning framework to estimate the vacuum pressure loss parameter θ, a variable derived from physical principles in dredging operations. Leveraging a modified Deep Operator Network (DeepONet), we introduce attention-based interactions between branches and the trunk network to capture complex dependencies in the sensor data. A local trunk mechanism is introduced to preserve temporal locality across dredging trips.
Due to the nature of a lagging density sensor, we integrate a real-time rolling mean error correction mechanism. This addresses training biases for refined predictions, as well as offering an anomaly detection mechanism. The model is trained and validated on real-world vessel data, including synthetic simulations of vacuum processes, and evaluated using trip-wise and global metrics. Experimental results show that the proposed architecture significantly outperforms the rolling mean baseline setups and the classical DeepONet across accuracy metrics such as the root mean square error (RMSE).
This work demonstrates the value of combining domain knowledge with operator learning techniques in maritime engineering. The proposed framework offers a scalable framework, allowing application across entire fleets for real-time suction production estimation and anomaly detection, contributing to efficient dredging operations. ... Read more

Today, machine learning has an accelerated impact in quantitative finance. Current models require large amounts of data, which can be expensive. A notable area of research, physics-informed neural networks (PINNs), has proven to be effective in approximating problems that are described by partial differential equations (PDEs). During training, the PDE is embedded in the loss function and evaluated at the residual points. This allows these types of neural networks to solve problems where data is scarce or noisy. Recent studies have shown that the method for sampling residual points has a great influence on training efficiency. Residual-based adaptive distribution (RAD) sampling is the adaptive sampling method used throughout this paper. This research applies PINNs with RAD sampling to solve the Black-Scholes PDE. Here, the Black-Scholes model is used to determine the price of options in a financial market. The fundamental goal of this paper is to study the difference in training performance between non-adaptive and RAD sampling. The types of options that are being considered in this study, are the European call options and the American put options. The results shown suggest that both types of options benefit from using RAD sampling compared to non-adaptive sampling. With a loss decrease of 39.33\%, American put options improve more using RAD sampling than European call options. Although European call options still show a decrease in loss of 7.57\%. ... Read more

Physics-Informed Neural Networks(PINNs) have emerged as a potent, versatile solution to solving both forward and inverse problems regarding partial differential equations(PDEs), accomplished through integrating laws of physics into the learning process. The applications of this new approach are endless, as these types of equations appear across numerous fields: fluids mechanics, quantum mechanics, electrochemistry and many others. Ever since their conception, researchers have continuously improved the flexibility and performance of PINNs through advancements in the architecture of neural networks, optimization algorithms, creative sampling methods and many more. As computational power and the interest of researchers grow, the revolutionary potential of PINNs is closer to fulfillment than ever. This paper aims to examine a small part of this evolutionary process, specifically the performance and flexibility of different activation functions used in the training of the PINN, as well as potential problems this approach could solve. ... Read more

Self-Adaptive Physics-Informed Neural Networks (SA-PINNs) are a variation of traditional Physics-Informed Neural Networks (PINNs) designed to solve the challenges of solving ”stiff” partial differential equations (PDEs). By using adaptive weighting, SA-PINNs are able to focus their attention on areas of the domain with higher errors, therefore improving accuracy. This work investigates the roles of individual loss components, namely residuals, boundary conditions, and initial conditions, in the performance of SA-PINNs. ... Read more

Physics-Informed Neural Networks (PINNs) are intended to solve complex problems that obey physical rules or laws but have noisy or little data. These problems are encountered in a wide range of fields including for instance bioengineering, fluid mechanics, meta-material design and high-dimensional partial differential equations (PDEs). Whilst PINNs show promising results, they often fail to converge in the presence of higher frequency components; a problem known as the spectral bias. Multiple studies have explored ways to overcome or minimize spectral bias specifically for PINNs. This paper builds on previous studies by investigating the impact of different gradient descent methods on the spectral bias. ... Read more

This thesis concerns the implementation of parallel Schwarz domain decomposition using node-level parallelism, focusing on the parallel Schwarz method in comparison with the Jacobi iterative method. The study goes into the complexities of domain decomposition methods for solving partial differential equations, which are essential in fields such as fluid dynamics, solid mechanics, quantum mechanics, and financial mathematics. The research examines the convergence and performance of these methods within a parallel computing framework. A large portion of the work involves the comparison of varying configurations of block sizes and overlaps within the use of the block Jacobi iterative method, used for the implementation of the parallel Schwarz method. Numerical experiments are conducted for a stationary heat problem on a 2-dimensional grid with 256 points in each direction. The results show optimal performance for small block sizes, attributed to the use of a dense solver for the subdomains. Larger blocks and larger overlaps show superior convergence properties, up to the limit of an overlap of half the block size. The efficiency of the parallel Schwarz method remains high for an increasing number of threads unlike the standard Jacobi iteration, showing it is better suitable to a parallel environment. ... Read more

Burn injuries present a significant global health challenge. Among the most severe long-term consequences are contractures, which can lead to functional impairments and disfigurement. Understanding and predicting the evolution of post-burn wounds is crucial for developing effective treatment strategies. Traditional mathematical models, while accurate, are often computationally expensive and time-consuming, limiting their practical application. Recent advancements in machine learning, particularly in deep learning, offer promising alternatives for accelerating these predictions. This study investigates the use of a deep operator network (DeepONet), a type of neural operator, as a surrogate model for finite element simulations for predicting post-burn wound evolution. We trained DeepONets on various wound shapes, enhancing the architecture by incorporating initial wound shape information and applying sine augmentation to enforce boundary conditions. The most sophisticated model achieved an R² score of 0.9960, indicating strong predictive accuracy. Additionally, the model generalised well to convex combinations of basic shapes, with an R² score of 0.9944, and provided reliable predictions over an extended period of up to one year. These findings suggest that DeepONets can effectively serve as a surrogate for traditional finite element methods in simulating post-burn wound evolution, with potential applications in medical treatment planning. ... Read more

Understanding multiphase flows is critical in nuclear engineering, particularly for processes such as coolant dynamics in nuclear reactors and safety scenario analyses involving different fluid phases. Numerical simulations are a valuable tool for studying these phenomena, especially when experimental approaches are impractical due to cost or safety concerns. While direct numerical simulations (DNS) offer detailed insights, their computational expense makes them impractical for turbulent flows, necessitating the use of turbulence models for efficiency.

This thesis introduces a novel machine learning framework designed to improve Reynolds-averaged Navier-Stokes models in turbulent stratified gas-liquid flows while employing the Boussinesq approximation. The framework encompasses two methods for turbulent viscosity field inversion and introduces correction terms in the turbulence model equations to ensure an accurate prediction of the turbulent viscosity field. Through sparse symbolic regression, the framework consistently discovers models that improve the accuracy of the baseline RANS model, even in untrained flow scenarios, though further testing is needed for varied flow regimes.

Key findings include the superior performance of sparse symbolic regression models over neural network (NN) models in improving the baseline RANS model accuracy. Notably, LASSO and elastic net techniques yielded the most successful models, significantly reducing baseline errors. However, these models did not surpass the Egorov damping approach in terms of accuracy, indicating the need for further refinement.

The developed models were numerically stable and robust, which is important for practical use. However, a main limitation is that the models' accuracy during training did not always correlate with the results when coupled with the RANS equations. Moreover, data from more varied flow conditions is needed to properly assess the generalizability of the models.

Overall, this research highlights the potential of data-driven turbulence modelling to enhance two-phase flow simulations, marking a significant step forward while also identifying areas for future improvement and exploration. ... Read more