Interval Markov Decision Processes with Continuous Action-Spaces

Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with...

Distributionally Robust Strategy Synthesis for Switched Stochastic Systems

We present a novel framework for formal control of uncertain discrete-time switched stochastic systems against probabilistic reach-avoid specifications. In particular, we consider stochastic systems with additive noise, whose distribution lies in an ambiguity set of distributions that are ε−close to a nominal one according to the Wasserstein...

A Cross-Field Review of State Abstraction for Markov Decision Processes

Complex real-world systems pose a significant challenge to decision making: an agent needs to explore a large environment, deal with incomplete or noisy information, generalize the experience and learn from feedback to act optimally. These processes demand vast representation capacity, thus putting a burden on the agent’s limited computational...

Optimizing deep reinforcement learning policies for deteriorating systems considering ordered action structuring and value of information

Inspection and maintenance (I&M) optimization entails many sources of computational complexity, among others, due to high-dimensional decision and state variables in multi-component systems, long planning horizons, stochasticity of objectives and constraints, and inherent uncertainties in measurements and models. This paper studies how the...

Strategy synthesis for partially-known switched stochastic systems

We present a data-driven framework for strategy synthesis for partially-known switched stochastic systems. The properties of the system are specified using linear temporal logic (LTL) over finite traces (LTLf), which is as expressive as LTL and enables interpretations over finite behaviors. The framework first learns the unknown dynamics via...

Radar Resource Management for Multi-Target Tracking Using Model Predictive Control

The radar resource management problem in a multi-target tracking scenario is considered. Partially observable Markov decision processes (POMDPs) are used to describe each tracking task. Model predictive control is applied to solve the POMDPs in a non-myopic way. As a result, the computational complexity compared to stochastic optimization...

Multi-Task Sensor Resource Balancing Using Lagrangian Relaxation and Policy Rollout

The sensor resource management problem in a multi-object tracking scenario is considered. In order to solve it, a dynamic budget balancing algorithm is proposed which models the different sensor tasks as partially observable Markov decision processes. Those are being solved by applying a combination of Lagrangian relaxation and policy rollout....

Improving mobile video quality through predictive channel quality based buffering

Frequent variations in throughput make mobile networks a challenging environment for video streaming. Current video players deal with those variations by matching video quality to network throughput. However, this adaptation strategy results in frequent changes of video resolution and bitrate, which negatively impacts the users' streaming...

Bounding the probability of resource constraint violations in multi-agent MDPs

Multi-agent planning problems with constraints on global resource consumption occur in several domains. Existing algorithms for solving Multi-agent Markov Decision Processes can compute policies that meet a resource constraint in expectation, but these policies provide no guarantees on the probability that a resource constraint violation will...

Planning Under Uncertainty for Aggregated Electric Vehicle Charging with Renewable Energy Supply

Renewable energy sources introduce uncertainty regarding generated power in smart grids. For instance, power that is generated by wind turbines is time-varying and dependent on the weather. Electric vehicles will become increasingly important in the development of smart grids with a high penetration of renewables, because their flexibility makes...

A ship egress analysis method using spectral Markov decision processes

A decision-making framework for planning lifecycle ballast water treatment compliance

A ship egress analysis method using spectral Markov decision processes

Solving Transition-Independent Multi-agent MDPs with Sparse Interactions

In cooperative multi-agent sequential decision making under uncertainty, agents must coordinate to find an optimal joint policy that maximises joint value. Typical algorithms exploit additive structure in the value function, but in the fully-observable multi-agent MDP (MMDP) setting such structure is not present. We propose a new optimal solver...

A decision-making framework for planning lifecycle ballast water treatment compliance

This paper introduces a novel decision-making framework for planning lifecycle compliance of ballast water treatment by applying eigenvalue spectral analysis<br/>to the ship-centric Markov decision process (SC-MDP) framework. This method focuses on identifying the relationships of various decision making scenarios, and how those relationships...

A ship egress analysis method using spectral Markov decision processes

This paper introduces a means of performing a ship egress analysis by applying eigenvalue analysis to the ship-centric Markov decision process (SC-MDP) framework. This method focuses on how people egress, the decisions they make under uncertainty, and the interaction between the individuals and the layout of the vessel. The objective is to...

Experience-based model predictive control using reinforcement learning

Model predictive control (MPC) is becoming an increasingly popular method to select actions for controlling dynamic systems. TraditionallyMPC uses a model of the system to be controlled and a performance function to characterize the desired behavior of the system. The MPC agent finds actions over a finite horizon that lead the system into a...