Improving value function approximation in factored POMDPs by exploiting model structure

Conference paper (2015)

Authors

Tiago S. Veiga Lisbon Technical University
M.T.J. Spaan Algorithmics - EEMCS
Pedro U. Lima Lisbon Technical University
Research Group
Algorithmics (EEMCS) (TU Delft)
POMDP Value function approximation
More Info
expand_more

Published Date

01-01-2015

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Algorithmics

Abstract

Linear value function approximation in Markov decision processes (MDPs) has been studied extensively, but there are several challenges when applying such techniques to partially observable MDPs (POMDPs). Furthermore, the system designer often has to choose a set of basis functions. We propose an automatic method to derive a suitable set of basis functions by exploiting the structure of factored models. We experimentally show that our approximation can reduce the solution size by several orders of magnitude in large problems.