DeepKoCo

Efficient latent planning with a task-relevant Koopman representation

Conference paper (2021)

Authors

D.S. van der Heijden Learning & Autonomous Control - Mechanical, Maritime and Materials Engineering
L. Ferranti Learning & Autonomous Control - Mechanical, Maritime and Materials Engineering
J. Kober Learning & Autonomous Control - Mechanical, Maritime and Materials Engineering
R. Babuska Learning & Autonomous Control - Mechanical, Maritime and Materials Engineering
Research Group
Learning & Autonomous Control (Mechanical, Maritime and Materials Engineering) (TU Delft)
Copyright: © 2021 D.S. van der Heijden, L. Ferranti, J. Kober, R. Babuska
DOI
help_outline
https://doi.org/10.1109/IROS51168.2021.9636408
Model-predictive control Model-based reinforcement learning Koopman theory
More Info
expand_more

Published Date

2021

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Mechanical, Maritime and Materials Engineering

Department

Cognitive Robotics

Research Group

Learning & Autonomous Control

Abstract

This paper presents DeepKoCo, a novel modelbased agent that learns a latent Koopman representation from images. This representation allows DeepKoCo to plan efficiently using linear control methods, such as linear model predictive control. Compared to traditional agents, DeepKoCo learns taskrelevant dynamics, thanks to the use of a tailored lossy autoencoder network that allows DeepKoCo to learn latent dynamics that reconstruct and predict only observed costs, rather than all observed dynamics. As our results show, DeepKoCo achieves a similar final performance as traditional model-free methods on complex control tasks, while being considerably more robust to distractor dynamics, making the proposed agent more amenable for real-life applications.