Reasoning about MDPs Abstractly: Bayesian Policy Search with Uncertain Prior Knowledge

Many real-world problems fall in the category of sequential decision-making under uncertainty; Markov Decision Processes (MDPs) are a common method for modeling such problems. To solve an MDP, one could start from scratch or one could already have an idea of what good policies look like. Furthermore, there could be uncertainty in this idea. In...

Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach

In the Condition-Based Maintenance (CBM) context, the definition of optimal maintenance plans for an aircraft fleet depends on an efficient integration of : (i) the probabilistic predictions of the health condition of the components and (ii) the stochastic arrival of the corrective maintenance tasks, together with consideration of the...

Contingency Games for Multi-Agent Interaction

Contingency planning, wherein an agent generates a set of possible plans conditioned on the outcome of an uncertain event, is an increasingly popular way for robots to act under uncertainty. In this work we take a game-theoretic perspective on contingency planning, tailored to multi-agent scenarios in which a robot's actions impact the...

On Game-Theoretic Planning with Unknown Opponents' Objectives

Many autonomous navigation tasks require mobile robots to operate in dynamic environments involving interactions between agents. Developing interaction-aware motion planning algorithms that enable safe and intelligent interactions remains challenging. Dynamic game theory renders a powerful mathematical framework to model these interactions...

A Comparative Study of Optimization Models for Condition-Based Maintenance Scheduling of an Aircraft Fleet

Condition-based maintenance (CBM) scheduling of an aircraft fleet in a disruptive environment while considering health prognostics for a set of systems is a very complex combinatorial problem, which is becoming more challenging in light of the uncertainty included in health prognostics. This type of problem falls under the broad category of...

Interval Markov Decision Processes with Continuous Action-Spaces

Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with...

Probabilistic Motion Planning for Multi-Robot Systems

Planning safe motions for multi-robot systems is crucial for deploying them in real-world applications such as target tracking, environmental monitoring, and multi-view cinematography. Traditional approaches mainly solve the multi-robot motion planning problem in a deterministic manner, where the robot states and system models are perfectly...

Decentralized probabilistic multi-robot collision avoidance using buffered uncertainty-aware Voronoi cells

In this paper, we present a decentralized and communication-free collision avoidance approach for multi-robot systems that accounts for both robot localization and sensing uncertainties. The approach relies on the computation of an uncertainty-aware safe region for each robot to navigate among other robots and static obstacles in the...

Collective Decision Making through Self-regulation: Mechanisms and Algorithms for Self-regulation in Decision-Theoretic Planning

This thesis explores the potential of self-regulation in collective decision making to align interests and optimise joint performance. Demonstrated in the domain of road maintenance planning, this research contributes novel incentive mechanisms and algorithmic techniques to incite self-regulation and coordinate agent interactions, paired with a...

The Complexity of Stability: In Parallel Processor Scheduling

In dynamic scheduling, some of the information about the scheduling problem is learnt during the execution of the schedule. Because of this, creating a low cost schedule that does not have to be changed is impossible in most cases. When changes are made during the execution, focussing solely on the performance of the schedule may result in very...

Planning under Uncertainty in Constrained and Partially Observable Environments

Developing intelligent decision making systems in the real world requires planning algorithms which are able to deal with sources of uncertainty and constraints. An example can be found in smart distribution grids, in which planning can be used to decide when electric vehicles charge their batteries, such that the capacity limits of lines are...

Interactive Learning and Decision Making: Foundations, Insights & Challenges

Designing "teams of intelligent agents that successfully coordinate and learn about their complex environments inhabited by other agents (such as humans)" is one of the major goals of AI, and it is the challenge that I aim to address in my research. In this paper I give an overview of some of the foundations, insights and challenges in this...

Learning and Optimizing Probabilistic Models for Planning under Uncertainty

Decision-theoretic planning techniques are increasingly being used to obtain (optimal) plans for domains involving uncertainty, which may be present in the form of the controlling agent's actions, its percepts, or exogenous factors in the domain. These techniques build on detailed probabilistic models of the underlying system, for which Markov...

Bounding the probability of resource constraint violations in multi-agent MDPs

Multi-agent planning problems with constraints on global resource consumption occur in several domains. Existing algorithms for solving Multi-agent Markov Decision Processes can compute policies that meet a resource constraint in expectation, but these policies provide no guarantees on the probability that a resource constraint violation will...

Planning Under Uncertainty for Aggregated Electric Vehicle Charging with Renewable Energy Supply

Renewable energy sources introduce uncertainty regarding generated power in smart grids. For instance, power that is generated by wind turbines is time-varying and dependent on the weather. Electric vehicles will become increasingly important in the development of smart grids with a high penetration of renewables, because their flexibility makes...

Planning under Uncertainty for Aggregated Electric Vehicle Charging using Markov Decision Processes

The increasing penetration of renewable energy sources and electric vehicles raises important challenges related to the operation of electricity grids. For instance, the amount of power generated by wind turbines is time-varying and dependent on the weather, which makes it hard to match flexible electric vehicle demand and uncertain wind power...

Planning under Uncertainty with Weighted State Scenarios

In many planning domains external factors are hard to model using a compact Markovian state. However, long-term dependencies between consecutive states of an environment might exist, which can be exploited during planning. In this paper we propose a scenario representation which enables agents to reason about sequences of future states. We show...