document

Reasoning about MDPs Abstractly: Bayesian Policy Search with Uncertain Prior Knowledge

Molhoek, Jord (author)

Many real-world problems fall in the category of sequential decision-making under uncertainty; Markov Decision Processes (MDPs) are a common method for modeling such problems. To solve an MDP, one could start from scratch or one could already have an idea of what good policies look like. Furthermore, there could be uncertainty in this idea. In...

master thesis 2024

document

Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach

Tseremoglou, I. (author), Santos, Bruno F. (author)

In the Condition-Based Maintenance (CBM) context, the definition of optimal maintenance plans for an aircraft fleet depends on an efficient integration of : (i) the probabilistic predictions of the health condition of the components and (ii) the stochastic arrival of the corrective maintenance tasks, together with consideration of the...

journal article 2024

document

Assessing the Optimality of Decentralized Inspection and Maintenance Policies for Stochastically Degrading Engineering Systems

Bhustali, P. (author), Andriotis, C. (author)

Long-term inspection and maintenance (I&M) planning, a multi-stage stochastic optimization problem, can be efficiently formulated as a partially observable Markov decision process (POMDP). How- ever, within this context, single-agent approaches do not scale well for large multi-component systems since the joint state, action and observation...

conference paper 2023

document

Investigating Inverse Reinforcement Learning from Human Behavior: Effect of Demonstrations with Temporal Biases on Learning Rewards using Inverse Reinforcement Learning

Zatezalo, Mateja (author)

Inverse Reinforcement Learning (IRL) is a machine learning technique used for learning rewards from the behavior of an expert agent. With complex agents, such as humans, the maximized reward may not be easily retrievable. This is because humans are prone to cognitive biases. Cognitive biases are a form of deviation from rationality that affects...

bachelor thesis 2023

document

Incorporating climate change in I&M planning for engineering structures - A Deep Reinforcement Learning Framework

Knoppert, Sammie (author)

In the last decades, climate change is causing our environment to change rapidly, unprecedented in recent history. Civil engineering structures are dependent on the deteriorating environment they are situated in. Changes can cause an increase in loading due to, for example, extreme weather events or alter the structure’s resistance by, for...

master thesis 2023

document

On the road from Model-Based Dynamical Programming to Model-Free Reinforcement Learning: A sample efficient approach

Mur Uribe, Pol (author)

This thesis introduces a new method, called Mixed Iteration, for controlling Markov Decision Processes when partial information is known about the dynamics of the Markov Decision Process. The algorithm uses sampling to calculate the expectation of partially known dynamics in stochastic environments. Its goal is to lower the number of iterations...

master thesis 2023

document

A Comparative Study of Optimization Models for Condition-Based Maintenance Scheduling of an Aircraft Fleet

Tseremoglou, I. (author), van Kessel, Paul J. (author), Santos, Bruno F. (author)

Condition-based maintenance (CBM) scheduling of an aircraft fleet in a disruptive environment while considering health prognostics for a set of systems is a very complex combinatorial problem, which is becoming more challenging in light of the uncertainty included in health prognostics. This type of problem falls under the broad category of...

journal article 2023

document

Interval Markov Decision Processes with Continuous Action-Spaces

Delimpaltadakis, Giannis (author), Lahijanian, Morteza (author), Mazo, M. (author), Laurenti, L. (author)

Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with...

conference paper 2023

document

Distributionally Robust Strategy Synthesis for Switched Stochastic Systems

Gracia, Ibon (author), Boskos, D. (author), Laurenti, L. (author), Mazo, M. (author)

We present a novel framework for formal control of uncertain discrete-time switched stochastic systems against probabilistic reach-avoid specifications. In particular, we consider stochastic systems with additive noise, whose distribution lies in an ambiguity set of distributions that are ε−close to a nominal one according to the Wasserstein...

conference paper 2023

document

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Morato, P. G. (author), Andriotis, C. (author), Papakonstantinou, K. G. (author), Rigo, P. (author)

In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I...

journal article 2023

document

Dynamic Programming based basicity control of an experimental smelting furnace prototype

Vitanov, George (author)

This thesis discusses the chemical composition (basicity) control problem of HIsarna, an experimental iron furnace which operates with 30% less CO2 emissions than its traditional blast furnace counterparts. The control challenge is keeping the basicity of the plant in a narrow operating region. A mass balance model of the plant was constructed -...

master thesis 2022

document

Generalized Models of Sequential Decision-Making under Uncertainty

Neustroev, G. (author)

Sequential decision-making under uncertainty is an important branch of artificial intelligence research with a plethora of real-life applications. In this thesis, we generalize two fundamental properties of the decision-making process. First, we show that the theory on planning methods for finite spaces can be extended to infinite but countable...

doctoral thesis 2022

document

Optimal maintenance of deteriorating systems integrating deep reinforcement learning and Bayesian inference

Lathourakis, Christos (author)

An issue of utmost significance constitutes the maintenance of engineering systems exposed to corrosive environments, e.g. coastal and marine environments, highly acidic environments, etc. The most beneficial sequence of maintenance decisions, i.e. the one that corresponds to the minimum maintenance cost, can be sought as the solution to an...

master thesis 2022

document

Using Decision Trees produced by Generative Adversarial Imitation Learning to give insight into black box Reinforcement Learning models

Meijer, Caspar (author)

Machine learning models are increasingly being used in fields that have a direct impact on the lives of humans. Often these machine learning models are black-box models and they lack transparency and trust which is holding back the implementation. To increase transparency and trust this research investigates whether imitation learning,...

bachelor thesis 2022

document

REAL Reinforcement Learning: Planning with adversarial models

Foffano, Daniele (author)

Model-Based Reinforcement Learning (MBRL) algorithms solve sequential decision-making problems, usually formalised as Markov Decision Processes, using a model of the environment dynamics to compute the optimal policy. When dealing with complex environments, the environment dynamics are frequently approximated with function approximators (such as...

master thesis 2022

document

Unlocking the Flexibility of District Heating Pipeline Energy Storage with Reinforcement Learning

Stepanovic, K. (author), Wu, J. (author), Everhardt, Rob (author), de Weerdt, M.M. (author)

abstract 2022

document

A Cross-Field Review of State Abstraction for Markov Decision Processes

Congeduti, E. (author), Oliehoek, F.A. (author)

Complex real-world systems pose a significant challenge to decision making: an agent needs to explore a large environment, deal with incomplete or noisy information, generalize the experience and learn from feedback to act optimally. These processes demand vast representation capacity, thus putting a burden on the agent’s limited computational...

conference paper 2022

document

Formal Control Synthesis for Stochastic Neural Network Dynamic Models

Adams, S.J.L. (author), Lahijanian, Morteza (author), Laurenti, L. (author)

Neural networks (NNs) are emerging as powerful tools to represent the dynamics of control systems with complicated physics or black-box components. Due to complexity of NNs, however, existing methods are unable to synthesize complex behaviors with guarantees for NN dynamic models (NNDMs). This letter introduces a control synthesis framework for...

journal article 2022

document

A Markovian decision model of adaptive cancer treatment and quality of life

Bayer, Péter (author), Brown, Joel S. (author), Dubbeldam, J.L.A. (author), Broom, Mark (author)

This paper develops and analyzes a Markov chain model for the treatment of cancer. Cancer therapy is modeled as the patient's Markov Decision Problem, with the objective of maximizing the patient's discounted expected quality of life years. Patients make decisions on the duration of therapy based on the progression of the disease as well as...

journal article 2022

document

Optimizing deep reinforcement learning policies for deteriorating systems considering ordered action structuring and value of information

Andriotis, C. (author), Papakonstantinou, K.G. (author)

Inspection and maintenance (I&M) optimization entails many sources of computational complexity, among others, due to high-dimensional decision and state variables in multi-component systems, long planning horizons, stochasticity of objectives and constraints, and inherent uncertainties in measurements and models. This paper studies how the...

conference paper 2022