document

Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach

Tseremoglou, I. (author), Santos, Bruno F. (author)

In the Condition-Based Maintenance (CBM) context, the definition of optimal maintenance plans for an aircraft fleet depends on an efficient integration of : (i) the probabilistic predictions of the health condition of the components and (ii) the stochastic arrival of the corrective maintenance tasks, together with consideration of the...

journal article 2024

document

A Comparative Study of Optimization Models for Condition-Based Maintenance Scheduling of an Aircraft Fleet

Tseremoglou, I. (author), van Kessel, Paul J. (author), Santos, Bruno F. (author)

Condition-based maintenance (CBM) scheduling of an aircraft fleet in a disruptive environment while considering health prognostics for a set of systems is a very complex combinatorial problem, which is becoming more challenging in light of the uncertainty included in health prognostics. This type of problem falls under the broad category of...

journal article 2023

document

Inference and dynamic decision-making for deteriorating systems with probabilistic dependencies through Bayesian networks and deep reinforcement learning

Morato, P. G. (author), Andriotis, C. (author), Papakonstantinou, K. G. (author), Rigo, P. (author)

In the context of modern engineering, environmental, and societal concerns, there is an increasing demand for methods able to identify rational management strategies for civil engineering systems, minimizing structural failure risks while optimally planning inspection and maintenance (I&M) processes. Most available methods simplify the I...

journal article 2023

document

Interval Markov Decision Processes with Continuous Action-Spaces

Delimpaltadakis, Giannis (author), Lahijanian, Morteza (author), Mazo, M. (author), Laurenti, L. (author)

Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with...

conference paper 2023

document

Distributionally Robust Strategy Synthesis for Switched Stochastic Systems

Gracia, Ibon (author), Boskos, D. (author), Laurenti, L. (author), Mazo, M. (author)

We present a novel framework for formal control of uncertain discrete-time switched stochastic systems against probabilistic reach-avoid specifications. In particular, we consider stochastic systems with additive noise, whose distribution lies in an ambiguity set of distributions that are ε−close to a nominal one according to the Wasserstein...

conference paper 2023

document

Generalized Models of Sequential Decision-Making under Uncertainty

Neustroev, G. (author)

Sequential decision-making under uncertainty is an important branch of artificial intelligence research with a plethora of real-life applications. In this thesis, we generalize two fundamental properties of the decision-making process. First, we show that the theory on planning methods for finite spaces can be extended to infinite but countable...

doctoral thesis 2022

document

Migration-aware Network Services with Edge Computing

Mukhopadhyay, Atri (author), Iosifidis, G. (author), Ruffini, Marco (author)

The development of Multi-access edge computing (MEC) has resulted from the requirement for supporting next generation mobile services, which need high capacity, high reliability and low latency. The key issue in such MEC architectures is to decide which edge nodes will be employed for serving the needs of the different end users. Here, we...

journal article 2022

document

Optimizing deep reinforcement learning policies for deteriorating systems considering ordered action structuring and value of information

Andriotis, C. (author), Papakonstantinou, K.G. (author)

Inspection and maintenance (I&M) optimization entails many sources of computational complexity, among others, due to high-dimensional decision and state variables in multi-component systems, long planning horizons, stochasticity of objectives and constraints, and inherent uncertainties in measurements and models. This paper studies how the...

conference paper 2022

document

Unlocking the Flexibility of District Heating Pipeline Energy Storage with Reinforcement Learning

Stepanovic, K. (author), Wu, J. (author), Everhardt, Rob (author), de Weerdt, M.M. (author)

abstract 2022

document

A Cross-Field Review of State Abstraction for Markov Decision Processes

Congeduti, E. (author), Oliehoek, F.A. (author)

Complex real-world systems pose a significant challenge to decision making: an agent needs to explore a large environment, deal with incomplete or noisy information, generalize the experience and learn from feedback to act optimally. These processes demand vast representation capacity, thus putting a burden on the agent’s limited computational...

conference paper 2022

document

Large-Scale Wildfire Mitigation Through Deep Reinforcement Learning

Altamimi, Abdulelah (author), Lagoa, Constantino (author), Borges, José G. (author), McDill, Marc E. (author), Andriotis, C. (author), Papakonstantinou, K. G. (author)

Forest management can be seen as a sequential decision-making problem to determine an optimal scheduling policy, e.g., harvest, thinning, or do-nothing, that can mitigate the risks of wildfire. Markov Decision Processes (MDPs) offer an efficient mathematical framework for optimizing forest management policies. However, computing optimal MDP...

journal article 2022

document

Unlocking the Flexibility of District Heating Pipeline Energy Storage with Reinforcement Learning

Stepanovic, K. (author), Wu, J. (author), Everhardt, Rob (author), de Weerdt, M.M. (author)

The integration of pipeline energy storage in the control of a district heating system can lead to profit gain, for example by adjusting the electricity production of a combined heat and power (CHP) unit to the fluctuating electricity price. The uncertainty from the environment, the computational complexity of an accurate model, and the scarcity...

journal article 2022

document

Formal Control Synthesis for Stochastic Neural Network Dynamic Models

Adams, S.J.L. (author), Lahijanian, Morteza (author), Laurenti, L. (author)

Neural networks (NNs) are emerging as powerful tools to represent the dynamics of control systems with complicated physics or black-box components. Due to complexity of NNs, however, existing methods are unable to synthesize complex behaviors with guarantees for NN dynamic models (NNDMs). This letter introduces a control synthesis framework for...

journal article 2022

document

A Markovian decision model of adaptive cancer treatment and quality of life

Bayer, Péter (author), Brown, Joel S. (author), Dubbeldam, J.L.A. (author), Broom, Mark (author)

This paper develops and analyzes a Markov chain model for the treatment of cancer. Cancer therapy is modeled as the patient's Markov Decision Problem, with the objective of maximizing the patient's discounted expected quality of life years. Patients make decisions on the duration of therapy based on the progression of the disease as well as...

journal article 2022

document

Approximately Optimal Resource Management for Multi-Function Radar: Algorithmic Solutions Using a Generic Framework

Schöpe, M.I. (author)

Recent advances in Multi-Function Radar (MFR) systems led to an increase in their degrees of freedom. As a result, modern MFR systems are capable of adjusting many parameters during runtime. An automatic adaptation of the radar system to changing situations, like weather conditions, interference, or target maneuvers, is often mentioned in the...

doctoral thesis 2021

document

The Intersection of Planning and Learning

Moerland, T.M. (author)

Intelligent sequential decision making is a key challenge in artificial intelligence. The problem, commonly formalized as a Markov Decision Process, is studied in two different research communities: planning and reinforcement learning. Departing from a fundamentally different assumption about the type of access to the environment, both research...

doctoral thesis 2021

document

A Constrained POMDP Formulation and Algorithmic Solution for Radar Resource Management in Multi-Target Tracking

Schöpe, M.I. (author), Driessen, J.N. (author), Yarovoy, Alexander (author)

The radar resource management problem in a multitarget tracking scenario is considered. The problem is solved using a dynamic budget balancing algorithm. It models the different sensor tasks as partially observable Markov decision processes and solves them by applying a combination of Lagrangian relaxation and policy rollout. The algorithm has a...

journal article 2021

document

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Andriotis, C. (author), Papakonstantinou, K. G. (author)

Determination of inspection and maintenance policies for minimizing long-term risks and costs in deteriorating engineering environments constitutes a complex optimization problem. Major computational challenges include the (i) curse of dimensionality, due to exponential scaling of state/action set cardinalities with the number of components; ...

journal article 2021

document

Strategy synthesis for partially-known switched stochastic systems

Jackson, John (author), Laurenti, L. (author), Frew, Eric (author), Lahijanian, Morteza (author)

We present a data-driven framework for strategy synthesis for partially-known switched stochastic systems. The properties of the system are specified using linear temporal logic (LTL) over finite traces (LTLf), which is as expressive as LTL and enables interpretations over finite behaviors. The framework first learns the unknown dynamics via...

conference paper 2021

document

Radar Resource Management for Multi-Target Tracking Using Model Predictive Control

de Boer, Thies (author), Schöpe, M.I. (author), Driessen, J.N. (author)

The radar resource management problem in a multi-target tracking scenario is considered. Partially observable Markov decision processes (POMDPs) are used to describe each tracking task. Model predictive control is applied to solve the POMDPs in a non-myopic way. As a result, the computational complexity compared to stochastic optimization...

conference paper 2021