Inland and short-sea container shipping in Northwestern Europe relies on manual planning by experienced logistics operators. This process, while effective for routine operations, is time-consuming, difficult to scale, and limited in its ability to globally optimize fleet utilization. The underlying scheduling problem is classified as a Heterogeneous Vehicle Routing Problem with Pickup and Delivery and Time Windows (VRPPDTW), incorporating domain-specific constraints such as minimum call sizes, forbidden terminals, terminal opening hours, and mandatory vessel breaks.

This thesis presents Orion, a constraint-based optimization solver built on the Google OR-Tools Routing Library, designed to automate and improve this vessel scheduling process. The problem is formulated using Constraint Programming, which allows complex business rules to be expressed as logical predicates rather than linearized inequalities. The solver pipeline includes a data preprocessing stage that aggregates individual container orders into compound orders, reducing problem size by approximately 90\% while preserving solution quality.

The evaluation follows a three-phase methodology. First, a feasibility analysis filters 14 construction heuristics down to two viable candidates: Local Cheapest Insertion and Parallel Cheapest Insertion. Second, a parameter tuning phase identifies Local Cheapest Insertion combined with Tabu Search as the best-performing configuration, achieving an Average Relative Percentage Deviation of 2.72\% across all scenarios. Third, a benchmarking phase compares Orion against human planners and the company's existing Simulated Annealing solver (Baseline SA) on six real-world scenarios from three distinct logistics operators.

The results show that Orion achieves travel cost reductions of 6.7\% to 32.8\% compared to human planners on five of six scenarios and outperforms Baseline SA's average result on five of six scenarios. A key structural advantage is determinism: Orion produces identical results across repeated runs, whereas Baseline SA exhibits variance of up to 37.5\% between its best and worst runs. Convergence analysis indicates that 96--99.97\% of objective improvement occurs within the first 30 seconds, making a 5-minute time budget sufficient for operational use.

The thesis concludes with recommendations for production deployment and identifies future research directions, including dynamic water level constraints, improved rolling-horizon replanning, custom fleet reduction operators, and hub-based consolidation strategies.

For most organizations, the majority of greenhouse gas emissions are Scope 3 emissions embedded in geographically dispersed supply chains. In such settings, environmental and economic impacts, as well as operating conditions, are uncertain, and decisions are sequential, meaning that early commitments constrain later operational choices. Yet current optimization practices do not integrate a full life cycle perspective in such conditions. In response, this thesis develops a stochastic Closed-Loop Supply Chain Life Cycle Optimization (SCLCO) framework that embeds the algebraic, matrix-based structure of life cycle assessment directly within a multistage stochastic mixed-integer program. Rather than assessing impacts ex post, the model minimizes expected environmental and economic impacts through sequential decisions while representing uncertainty in supplier-dependent life cycle inventory data and economic drivers, as well as stochastic demand and return processes. The framework is illustrated using a university campus circular furniture procurement case study. Applying the stochsatic SCLCO to a framework agreement yields decision-relevant insights for supplier selection under joint environmental–economic objectives. The resulting problem is solved using Sample Average Approximation. Trade-offs are explored via weighted objectives and summarized using a Pareto frontier; robustness is assessed through optimality-gap analysis and value-of-information metrics; and outcomes are examined using contribution, uncertainty, and sensitivity analyses. Results indicate that economic outcomes are driven primarily by demand uncertainty, whereas environmental outcomes are dominated by uncertainty in manufacturing-stage inventory data. Accordingly, this work offers a decision-support framework for decarbonization efforts throughout supply chains.

Type systems are a tool for preventing software errors, by classifying (sub)terms according to how they are evaluated. This way, mistakes can be caught at compile-time, ruling out the existence of entire classes of mistakes altogether. Using a programming language with a strong type system to develop critical software can dramatically reduce the prevalence and impact of bugs.
In light of the potentially enormous impact of bugs, it is important that we can trust a type system to be succesful in preventing errors. A key property of type systems that reflects this criterium is type soundness, which establishes that “well-typed programs cannot go wrong”. That is, programs that are deemed safe by the type system should not exhibit certain wrong behaviour when executed. To gain trust in a type system’s ability to prevent errors, we can give a formal system of both a language’s type system and semantics, and mathematically verify that the type system is sound with respect to the defined semantics. While this provides airtight evidence that a type system is succesful in ruling out certain mistakes, the formal specification and verification of programming languages requires a formidable amount of time and expertise on behalf of the language designer, and is therefore infeasible in most cases....

Optimisation problems are all around us and play a critical role in the outcomes of various sectors of society including scheduling, logistics, network design, and resource allocation. In this thesis, we look at a subset of problems where some or all the choices to be made can only take on values belonging to a discrete set of values – a limitation which increases the difficulty of finding a solution in most cases. We handle this thesis in two parts: first zooming in on a particular class of problems – scheduling – and then on a particular solution method – decision diagrams.

In Part I of this dissertation, we consider the problem of scheduling in manufacturing systems. This popular discrete optimisation problem has been studied extensively; however, advancements in modern manufacturing systems present new challenges and opportunities. A major driver of the changes in manufacturing systems is the integration of the physical processes with computation, networking, and automated control capabilities. Thus, the domain of activities that can directly be decided upon and actuated from software has expanded. We look at one such activity in Chapter 3 namely,
sequence-dependent maintenance and propose solution methods that directly integrate maintenance and production planning.

Part II of this thesis looks into decision diagrams as a solution method for discrete optimisation problems. Decision diagrams have existed since the 1950s and were originally introduced as a means to represent boolean functions. Since then they have been used in different fields such as in circuit verification, knowledge representation, and most recently, operations research. While decision diagrams have already been shown to be very promising methods for solving optimisation problems, we push the field forward as follows. In Chapter 5, we propose a decision diagram model for the kind of scheduling problems tackled in Part I of this thesis. We further perform a comparative study of the consequences of heuristic decisions made during the decision diagram compilation process in Chapter 6 and integrate reinforcement learning with decision-diagram-based branch-and-bound in Chapter 7. In Chapter 8 we go further into considering uncertainty in optimisation problems. We focus on the paradigm of finding the best solution while accepting some level of risk, i.e., chance constrained optimisation and present a chance constrained decision diagram formulation. We further provide theoretical guarantees for instances with normally distributed variables.

The contributions of this thesis cover different aspects of solving discrete optimisation problems with a focus on scheduling and logistic applications. The hope is that these advances further the adoption of state of the art research in solving optimisation problems in real-world contexts.

The goal of reinforcement learning is to train agents to perform tasks under little supervision. Tasks are specified by a reward function and transition function, which state how much reward the agent gets for its action in a state, and how the environment state changes based on the action the agent took. Typically reinforcement learning assumes no prior knowledge over the reward and transition function,meaning that agents need to explore the environment and learn essentially through trial and error. Model-free methods attempt to learn which actions lead to good outcomes without modeling the reward or environments itself. Efficiently selecting that actions are promising is an active research direction which can greatly reduce the number of total interactions needed for an agent to learn the task, potentially opening the door to new applications where trials or simulations are expensive or compute is limited.
Uncertainty quantification is a central mechanism in such efficient exploration methods. Provided with an estimate of how certain the agent is about the outcome of an action, it can intelligently weigh whether it is worth exploring. The Bayesian paradigm is one method to quantify uncertainty in machine learning. It models the uncertainty with a probability distributions over models, specifying how likely a model is based on the data the agent has collected.
We adopt a Bayesian point of view in model-free reinforcement learning, and develop a deeper understanding on when Bayesian reinforcement learning methods can be expected to work well and challenges that remain. To this end, in Chapter 2 we propose training ensembles through Sequential Monte Carlo, obtaining a sample from the posterior distribution of a deep Q-learning agent. We observe that agents are able to perform directed exploration, although not necessarily more efficiently than standard ensembles in every environment. Furthermore, in Chapter 3 we theoretically analyze existing Bayesian Deep model-Free Reinforcement Learning methods, and unify them into a single theoretical framework we call Epistemic Bellman Operators. We prove that these operators are contractions, establishing convergence of derived algorithms in a simplified setting. Finally, in Chapter 4 we analyze the likelihood and prior assumptions
in existing Bayesian deep model-free reinforcement learning methods, and find through statistical tests that the standard likelihood assumptions are violated on every benchmark we tested. We also find that we can improve performance of Bayesian model free reinforcement learning methods by picking different priors based on empirical data from unrelated tasks, which transfer to new environments.
This dissertation establishes several desirable properties of Bayesian Deep model free reinforcement learning, but also raises some key issues, most notably misspecification in Chapter 4. We hope our findings convince other Bayesian reinforcement learning researchers to give more attention to assumptions about priors and likelihoods.

Buildings, as major global energy consumers, can help mitigate the impact of growing renewable energy in smart grids through demand-side management (DSM). Smart energy management of buildings requires advanced control schemes that can cope with economic objectives, environmental uncertainties, occupant comfort, physical constraints, and external communication signals. Robust optimization (RO) and model predictive control (MPC) provide systematic and effective frameworks for this purpose. Beyond building energy management, RO and MPC methods are also fundamental to a broad range of engineering applications, such as chemical process planning, transportation, robotics, etc. This thesis focuses on RO and MPC design for linear systems as well as their applications in building energy management.  The research is organized into three topics:

• MPC designs for building energy management to enable energy-flexible DSM and improve environmental sustainability (Chapters 2–4).
• data-driven RO designs and algorithmic solutions for linear models to reduce conservatism and improve computational efficiency (Chapters 5 & 6).
• a distributionally robust MPC design for constrained linear systems to robustify control performance against additive model uncertainties and disturbances (Chapter 7).

Air transport has enormous impact on economic, social, and environmental factors worldwide. According to the International Air Transport Association significant year on year increases can be noticed recently, in both passenger and cargo traffic. However, with this increasing demand for air transport, airports are faced with a rising trend in congestion, delays, and other problematic inefficiencies. Although increasing demand is a factor in these challenges, airline or airport causes, such as ground handling, remain the second largest cause of delays. This highlights the need for efficient and optimal scheduling of ground handling processes to minimise delays, and thereby the negative impact this might have on the airlines or airports. Most existing studies address only simplified sub-problems of Airport Ground Handling (AGH), by relaxing constraints or by leaving them out, rather than tackling the complete problem. Furthermore, these approaches tend to focus on single objective optimisation of AGH, while in practice there can be many (conflicting) objectives that need to be optimised simultaneously. This research explores the possibility of extending a neural model, which is trained to optimise instances of AGH on a single objective, with a generic learning-based approach that approximates the Pareto set for multi-objective optimisation, and applying further hypothetical improvements to this combined model. The implemented models are compared against each other, heuristic approaches for multi-objective combinatorial optimisation, and against the original single-objective neural model.

In large-scale engineering environments, efficient issue tracking is essential for timely problem resolution and knowledge reuse. However, manual classification and association of issue reports present scalability challenges, further complicated by inconsistent annotations and the absence of semantic linking mechanisms. This project investigates the application of Natural Language Processing and Artificial Intelligence to automate multi-label classification and discover meaningful semantic associations between technical issues. Over 70 model configurations were evaluated on a real-world industrial dataset, comparing classical models with transformer-based and deep learning approaches. DistilBERT achieved the highest Recall@5 (0.93), indicating strong performance in identifying relevant categories. Classical methods, such as TF-IDF combined with Logistic Regression, also performed well, offering a computationally efficient and interpretable option. For association discovery, approaches including lexical retrieval, embedding-based similarity, clustering-based filtering, and topic modelling were assessed using both quantitative metrics and expert review. Lexical (BM25) and embedding-based (SBERT + Cosine Similarity) methods offer complementary strengths, retrieving overlapping yet distinct sets of associations. Associations identified by both models were rated as useful in over 70% of cases by domain experts, suggesting that agreement between methods may serve as an indicator of relevance. While Copilot provided consistent relevance assessments, its ratings were often higher than those provided by human evaluators and did not always reflect their detailed assessments. These findings highlight the potential of combining lexical and semantic methods with human-in-the-loop validation to support scalable and accurate industrial applicability.

Bayesian Deep Q-Networks (BDQN) have demonstrated superior exploration capabilities and performance in complex environments such as Atari games, yet their behavior in other simpler settings and their sensitivity to hyperparameters remain understudied. This work evaluates BDQN in both contextual bandit and reinforcement learning tasks, compares it against the standard ϵ-greedy exploration strategy and analyzes its hyperparameter sensitivity. Our results indicate that BDQN outperforms ϵ-greedy DQN in exploration-heavy environments, particularly Deep Sea with sparse rewards, but performs comparably in simpler tasks where exploration is less critical. Sensitivity analysis reveals that the forgetting factor (α) plays a central role in modulating
exploration, while other hyperparameters such as batch size also impact performance to varying degrees. These findings suggest BDQN is a promising strategy for complex tasks requiring persistent exploration, though it introduces additional tuning complexity.

This thesis investigates the gap between the theoretical ideals and practical realities of distributed exchange protocols in peer-to-peer sharing economies. While classical economic models and stylised trading mechanisms have been extensively studied, their assumptions often overlook the complexities and behavioural nuances of real-world agent-based markets. Conversely, more empirically grounded approaches tend to be highly case-specific, limiting their generalisability and disconnecting them from established economic concepts.
The central aim of this work is to develop and analyse an algorithmic economic model that more faithfully captures the dynamics of decentralised resource sharing — balancing fairness, efficiency, and resistance to manipulation, while retaining a useful level of abstraction.
In light of this purpose, the thesis performs a comparative theoretical analysis of centralised and distributed exchange mechanisms, and introduces a novel exchange market model that incorporates heterogeneous strategies, bounded rationality, and asynchronous timing. A series of numerical experiments evaluates the performance and robustness of different protocols under these features of sharing economies.
The results show that distributed, mixed-strategy protocols can achieve stable and desirable outcomes, however their success is sensitive to population diversity, limited information, and strategic behaviour. These findings highlight the importance of integrating behavioural aspects into protocol design, and provide insights towards building more robust models for the sharing economy.

This thesis presents a comprehensive, heuristic cost-driven framework for optimizing database table configuration in Amazon Redshift focusing on distribution styles, sort keys and column encodings. Unlike existing approaches that treat optimization parameters independently, this research develops a sequential optimization methodology that captures complex interdependencies between configuration choices and their performance impacts across different data scales.

The study addresses four research questions examining individual parameter optimization strategies and their integrated effects on system performance. The experimental evaluation employs two datasets: a primary table containing 300 million data records across 23 columns where optimization is performed, and a secondary join table with 117.5 million data records across 12 columns that remains unchanged. Scale-dependent analysis is conducted using subsets of 10 million and 100 million data records selected from the primary dataset to enable controlled comparison across different data volumes.

Key findings demonstrate that table configuration optimization in Amazon Redshift exhibits pronounced scale-dependent performance characteristics across the experimental datasets, with three distinct performance regimes identified: a small-scale regime (10M data records) characterized by query-type dependent optimization effectiveness, a medium-scale regime (100M data records) showing optimization trade-off transitions, and a large-scale regime (300M data records) dominated by I/O and storage optimizations. The research reveals mixed optimization outcomes, with performance improvements ranging from 21 percent CPU reduction at small scales to 62 percent I/O improvement at large scales, while demonstrating that optimization strategies effective at one scale can become counterproductive at another. Overall, the framework shows variable success in parameter selection for distribution style, sort key and encoding selection.

The research identifies fundamental challenges in optimizing Amazon Redshift table configurations where internal algorithms remain opaque and optimization benefits exhibit non-linear scaling patterns across the tested different data volumes. While the framework provides valuable insights into scale-dependent optimization patterns, the mixed results highlight the complexity of achieving consistent performance improvements across different scales and query types. These findings challenge assumptions about uniform optimization benefits and emphasize the need for empirical validation approaches in cloud database optimization, providing practical insights for database administrators and theoretical foundations for developing adaptive optimization systems.

This thesis presents a novel framework for solving the Multi-Skill Resource-Constrained Multi-Modal Project Scheduling Problem with maximum time lags, addressing the challenges of scalability, deadline adherence, and uncertainty in job durations. The research is conducted through a case study with the maintenance department of a large European airline, using real-life maintenance scheduling data. To improve scalability, the framework integrates batching techniques that segment the scheduling horizon and a priority-rule-based heuristic to hot start the solver, significantly reducing computational runtimes for large problem instances. A range of objective functions, including single and multi-objective formulations, are explored to evaluate their impact on scheduling performance. The results demonstrate
that multi-objective formulations provide the best balance between throughput and deadline adherence and consistently outperform a priority-based heuristic. A clear trade-off is observed between optimizing for maximum tardiness and average tardiness, where minimizing maximum tardiness improves deadline adherence at the cost of lower throughput, while minimizing average tardiness has a more consistent throughput but allows slightly more deadline misses. To address job duration uncertainty, adaptive buffering strategies based on historical job performance are introduced and shown to outperform static buffers by tailoring slack times to individual job characteristics. In the examined case study, the combination of an adaptive buffering strategy with a multi-objective function combining makespan and
average weighted tardiness offers the most effective trade-off between robustness and efficiency. Overall, the framework proves to be scalable, adaptable, and well-suited to real-world scheduling environments with high variability and complex constraints.

Effectively solving the Nurse Rostering Problem enhances nurse moral and leads to improved patient care. While the use of ejection chains has shown promise in previous studies, studying their impact on real-life instances from two Dutch hospitals further deepens our understanding of their practical utility. This thesis begins by discussing the Nurse Rostering Problem (NRP) as presented in literature, highlighting the lack of research conducted in real-life settings. We then describe the context of Dutch hospitals by defining the relevant hard and soft constraints. Afterwards, a review of previous NRP solutions is provided, covering various solvers and techniques, including ejection chains. Ejection chains are an approach that combines multiple local search moves into a larger one, where each move forces (ejects) the next move. These moves are executed consecutively, forming a chain. Two recent ejection chain approaches were identified for further investigation, and a novel approach is also proposed.

Afterwards, the concept of the three ejection chains is explained. The first ejection chain, InfeasibleEC, is inspired by Kingston, and explores regions of the search space that contain infeasible rosters. It starts with a swap that introduces a single hard constraint violation, but results in lower roster penalty. Then the ejection chain proceeds with a series of repairs, where each repair fixes the previous violation, but may introduce a single new violation. Four different approaches to explore the search space were implemented. The second ejection chain described is EmulateEC by Curtois et al. This chain emulates human schedule planners by performing a series of vertical swaps. Each swap improves the penalty for one nurse but simultaneously increases the penalty for a different nurse. Thus, the next swap aims to improve the nurse whose penalty was worsened in the previous step. This series of swaps is repeated until a better roster is found. The third ejection chain, RuinRecreateEC, repeatedly performs ruin and recreate operations in sequence. The idea is that each ruin and recreate operation makes a relatively small change to the roster, and by repeating the process multiple times, the final roster undergoes substantial diversification while still retaining much of the structure of the original roster.

All three ejection chains underwent individual parameter tuning using a sequential greedy tuning strategy to identify suitable parameter configurations based on two instances from the same hospital. Once tuned, the chains were further tested by rerunning the chains on the initial instances as well as on additional instances from a second hospital to assess their robustness. We analysed the breakdown of the penalty based on the different soft constraints, and investigated the impact of the chains on finding better rosters throughout the running time. We could not find improvements on specific soft constraints penalties, nor on finding better rosters faster. A significance test was conducted to determine whether the chains significantly reduce the penalty of the final rosters. There was insufficient evidence to conclude a statistically significant improvement.

Deep Reinforcement Learning has achieved superhuman performance in many tasks, such as robotic control or autonomous driving. Algorithms in Deep Reinforcement Learning still suffer from a sample efficiency problem, where, in many cases, millions of samples are needed to achieve good performance. Recently, Bayesian uncertainty-based algorithms have gained traction. This work focuses on providing a better understanding of the behaviour of Langevin Monte Carlo algorithms for Bayesian posterior approximation applied on top of Q-learning. This research builds on top of already existing algorithms, aiming to provide a better understanding of the underlying mechanics that drive them. We provide empirical experimentation with different hyperparameters in three different environments. Our results suggest that hyperparameters that were previously thought not to have a big impact on the algorithms are crucial for deep exploration.

We present a large-scale empirical study of Bootstrapped DQN (BDQN) and Randomized-Prior BDQN (RP-BDQN) in the DeepSea environment, aimed at characterizing their scaling properties. Our primary contribution is a unified scaling law that accurately models the probability of reward discovery as a function of task hardness and ensemble size. This law is parameterized by a method-dependent effectiveness factor, $\psi$.Under this framework, RP-BDQN demonstrates substantially higher effectiveness ($\psi \approx 0.87$) compared to BDQN ($\psi \approx 0.80$), enabling it to solve more challenging tasks.Our analysis reveals that this advantage stems from RP-BDQN's sustained ensemble diversity, which mitigates the posterior collapse observed in BDQN.Furthermore, we demonstrate diminishing returns in performance for ensemble sizes $K>10$. These results offer practical guidance for ensemble configuration and raise new theoretical questions surrounding the effectiveness parameter $\psi$.

Efficient exploration is a major issue in reinforcement learning, particularly in environments with sparse rewards. In these environments, traditional methods like e-greedy fail to efficiently reach an optimal policy. A new method proposed by Fortunato, et al. Fortunato, et al. showed promise by improving efficiency on RL tasks such as Atari games, by driving exploration with learned perturbations of the network weights. Three different types of settings were investigated in order to test the robustness of a contextual bandit implemented with this proposed method: 1. ContextualBandit-v2; a bandit with multiple predefined functions mapping the 1-dimensional continuous input to the reward, 2. MNISTBandit-v0; a bandit rewarding correct identification of MNIST dataset images, and 3. NNBandit-v0; a bandit with the reward being determined by a neural network. Furthermore, non-stationary variants of environment 1. and 3. were tested. A slight variation in hyperparameter sensitivity between environments was observed and a generally optimal set was determined. Overall, NoisyNet-DQNs (Deep Q-Networks) achieved performance comparable to regular DQNs, though often slightly lower. In the high-dimensional stationary MNISTBandit-v0 environment, NoisyNet-DQN converged to an optimal policy slightly faster, at the cost of a larger variation in performance.

This paper examines how different exploration strategies affect the learning behavior and trading performance of reinforcement learning (RL) agents in a custom foreign exchange (forex) environment. By holding all other components constant—including model architecture, features, and reward function—the study isolates the role of exploration in deep Q-learning. Three strategies were compared: Epsilon-Greedy, Boltzmann, and Max-Boltzmann. The hybrid Max-Boltzmann approach delivered the most stable and profitable outcomes, suggesting that weighted, value-aware exploration can be beneficial in highrisk domains. The results also highlight the impact of non-Markovian structure in financial environments and limitations of equity-based rewards. Beyond empirical results, this work contributes a modular, reproducible RL framework for trading, and opens new questions about the suitability of exploration techniques in environments with asymmetric risk and irreversible actions.

Algorithmic trading already dominates modern financial markets, yet most live systems still rely on fixed heuristics that falter when conditions change. Deep reinforcement learning agents promise adaptive decision making, but their behaviour is driven entirely by the reward function - a design choice that remains poorly researched. Given the critical importance of reward function design in RL problems, this paper investigates the impact of different choices of reward functions while trading in the Forex market and focusing on the EUR/USD pair. We explore different rewarding methods such as profit‑only, risk‑adjusted, multi‑objective, imitation‑learning based, and self‑rewarding mechanisms. Overall, we demonstrate the impact of careful reward engineering and whether it can boost performance and efficiency, highlighting reward design as a critical and previously under‑examined part of RL for deploying reliable trading models.

This study explores how different features impact a Reinforcement Learning agent's performance in forex trading. Using a Deep Q-Network (DQN) agent and EUR/USD data from 2022-2024, we found that performance is highly sensitive to the information provided. Key findings show that for feature types like momentum and volatility, a single indicator outperformed a combination of them, as the latter tended to introduce noise. Including information about the agent's own status, such as its current trade duration, was beneficial. Counter-intuitively, providing more historical data consistently worsened performance, leading to overfitting where the agent memorized training data rather than learning general strategies. The main conclusion is that creating an effective state representation is a trade-off; the complexity of the input data must match the learning algorithm's ability to process it without overfitting.