Model-Based Reinforcement Learning with State Abstraction: A Survey

Conference paper (2022)

Authors

R.A.N. Starre Interactive Intelligence - EEMCS
M. Loog Pattern Recognition and Bioinformatics - EEMCS
F.A. Oliehoek Interactive Intelligence - EEMCS
Research Group
Interactive Intelligence (EEMCS) (TU Delft)
Copyright: © 2022 R.A.N. Starre, M. Loog, F.A. Oliehoek
DOI
help_outline
https://doi.org/10.1007/978-3-031-39144-6_9
State Abstraction Model-Based RL MDPs
More Info
expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Intelligent Systems

Research Group

Interactive Intelligence

Abstract

Model-based reinforcement learning methods are promising since they can increase sample efficiency while simultaneously improving generalizability. Learning can also be made more efficient through state abstraction, which delivers more compact models. Model-based reinforcement learning methods have been combined with learning abstract models to profit from both effects. We consider a wide range of state abstractions that have been covered in the literature, from straightforward state aggregation to deep learned representations, and sketch challenges that arise when combining model-based reinforcement learning with abstraction. We further show how various methods deal with these challenges and point to open questions and opportunities for further research.