Condition-Based Maintenance scheduling of an aircraft fleet under partial observability: A Deep Reinforcement Learning approach

In the Condition-Based Maintenance (CBM) context, the definition of optimal maintenance plans for an aircraft fleet depends on an efficient integration of : (i) the probabilistic predictions of the health condition of the components and (ii) the stochastic arrival of the corrective maintenance tasks, together with consideration of the...

A Comparative Study of Optimization Models for Condition-Based Maintenance Scheduling of an Aircraft Fleet

Condition-based maintenance (CBM) scheduling of an aircraft fleet in a disruptive environment while considering health prognostics for a set of systems is a very complex combinatorial problem, which is becoming more challenging in light of the uncertainty included in health prognostics. This type of problem falls under the broad category of...

BADDr: Bayes-Adaptive Deep Dropout RL for POMDPs

While reinforcement learning (RL) has made great advances in scalability, exploration and partial observability are still active research topics. In contrast, Bayesian RL (BRL) provides a principled answer to both state estimation and the exploration-exploitation trade-off, but struggles to scale. To tackle this challenge, BRL frameworks with...

Approximately Optimal Radar Resource Allocation: Joint Radar Communication for Automotive Applications

Due to the rising number of wireless device users, it is expected that there will be a scarcity in the spectrum. The will especially true for the Automotive Spectrum between 77 and 81 GHz. In this thesis, we apply Sensor Management to the Joint Radar Communication scenario. We develop an algorithm that can allocate resources to both sensing and...

Radar Resource Management for Multi-Target Tracking using Model Predictive Control

With modern multi-function radars becoming more flexible, handling the limited amount of resources of these radars becomes increasingly important. In this thesis the radar resource management (RRM) problem in a multi-target tracking scenario is considered. Partially observable Markov decision processes (POMDPs) are used to describe each tracking...

Approximately Optimal Radar Resource Management for Multi-Sensor Multi-Target Tracking: A First Step Towards a Distributed Solution for Radar Resource Management in a Sensor Network

The Radar Resource Management (RRM) problem in a multi-sensor multi-target scenario is considered. The problem is defined as a constrained optimization problem in which the predicted error covariance is minimized subject to resource budget constraints. By applying Lagrangian Relaxation (LR) the problem is decoupled into multiple sub-optimization...

Approximately Optimal Radar Resource Management for Multi-Sensor Multi-Target Tracking

Radar Resource Management in a multi-sensor multi-target scenario is considered. A dynamic resource balancing algorithm is proposed which optimizes target task parameters assuming an underlying partially observable Markov decision process (POMDP). By applying stochastic optimization methods, such as policy rollout, the POMDP is solved non...

Planning for Money Laundering Investigations

According to the United Nations, the amount of money laundered worldwide each year is an estimated 2 - 5% of global GDP (equivalent to $800 billion to $2 trillion in US dollars). This is money that criminal enterprises rely on to oper- ate. For that reason, the European Union demands that gatekeepers (banks and other obliged entities) apply...

Exploring the effects of conditioning Independent Q-Learners on the sufficient plan-time statistic for Dec-POMDPs

The Decentralized Partially Observable Markov Decision Process is a commonly used framework to formally model scenarios in which multiple agents must collaborate using local information. A key difficulty in a Dec-POMDP is that in order to coordinate successfully, an agent must decide on actions not only using its own information, but also by...

POMDP based online parameter estimation for autonomous passenger vehicles: Researching online tyre parameter estimation performance by improving the trajectory using a POMDP algorithm.

The internal model is an important piece of the control system of an autonomous driving vehicle. In order for the model to deliver accurate predictions, a valid model structure and well chosen parameters are needed. Model parameters can be highly fluctuating or complex to predict, especially when looking into tyre ground surface interaction...

Influence-Based Abstraction in Deep Reinforcement Learning

thousands, or even millions of state variables. Unfortunately, applying reinforcement learning algorithms to handle complex tasks becomes more and more challenging as the number of state variables increases. In this paper, we build on the concept of influence-based abstraction which tries to tackle such scalability issues by decomposing large...

Bayesian Reinforcement Learning in Factored POMDPs

Model-based Bayesian Reinforcement Learning (BRL) provides a principled solution to dealing with the exploration-exploitation trade-off, but such methods typically assume a fully observable environments. The few Bayesian RL methods that are applicable in partially observable domains, such as the Bayes-Adaptive POMDP (BA-POMDP), scale poorly. To...

Exploiting submodular value functions for scaling up active perception

In active perception tasks, an agent aims to select sensory actions that reduce its uncertainty about one or more hidden variables. For example, a mobile robot takes sensory actions to efficiently navigate in a new environment. While partially observable Markov decision processes (POMDPs) provide a natural model for such problems, reward...

Decision Making under Uncertainty for Automated Vehicles in Urban Situations

One of the most crucial links to building an autonomous system is the task of decision making. The ability of a vehicle to make robust decisions on its own by predicting and assessing future consequences is what makes it intelligent. This task of decision making becomes even more complex due to the fact that the real world is uncertain,...

Probabilistic Motion Planning in Uncertain and Dynamic Environments

Autonomous driving is one of the popular and advanced research fields aiming to reduce the mortality rate and improve the welfare and efficiency of commuters' lives. As one of the important research branches of self-driving cars, this thesis focuses on the motion planning problems in dynamic and uncertain environments. The challenge of motion...

Reducing the need for communication in wireless sensor networks using machine learning and planning techniques

Wireless sensor networks are commonly used to remotely and automatically monitor environments.<br/>One of the main challenges in wireless sensor networks is to use the limited available energy as efficiently as possible, to ensure longevity of the network. For such networks to survive their intended deployment period no energy may be wasted on...

Robot Search in Unknown Environments using POMDPs

Search is an important competence for a robot. It is the core task of a search and rescue robot, and many other typical robots task require some form of search. In order to be truly autonomous, the robot must be able to perform its tasks in an unknown environment. Using SLAM, the robot has to make search decisions with increasing knowledge of...