Print Email Facebook Twitter Think Too Fast Nor Too Slow Title Think Too Fast Nor Too Slow: The Computational Trade-off Between Planning And Reinforcement Learning Author Moerland, T.M. (TU Delft Interactive Intelligence; Universiteit Leiden) Deichler, Anna (Student TU Delft) Baldi, S. (TU Delft Team Bart De Schutter; Southeast University) Broekens, D.J. (Universiteit Leiden) Jonker, C.M. (TU Delft Interactive Intelligence; Universiteit Leiden) Contributor Fern, Alan (editor) Gomez, Vicenc (editor) Jonsson, Anders (editor) Katz, Michael (editor) Palacios, Hector (editor) Sanner, Scott (editor) Date 2020 Abstract Planning and reinforcement learning are two key approaches to sequential decision making. Multi-step approximate real-time dynamic programming, a recently successful algorithm class of which AlphaZero [Silver et al., 2018] is an example, combines both by nesting planning within a learning loop. However, the combination of planning and learning introduces a new question: how should we balance time spend on planning, learning and acting? The importance of this trade-off has not been explicitly studied before. We show that it is actually of key importance, with computational results indicating that we should neither plan too long nor too short. Conceptually, we identify a new spectrum of planning-learning algorithms which ranges from exhaustive search (long planning) to model-free RL (no planning), with optimal performance achieved midway. To reference this document use: http://resolver.tudelft.nl/uuid:f46e64c0-a1ce-457e-bf56-748d3bb73673 Publisher Association for the Advancement of Artificial Intelligence (AAAI) Source ICAPS: PRL 2020: Proceedings of the 1st Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL) Event ICAPS 2020, 2020-10-19 → 2020-10-30, Virtual Nancy, France Bibliographical note Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. Part of collection Institutional Repository Document type book chapter Rights © 2020 T.M. Moerland, Anna Deichler, S. Baldi, D.J. Broekens, C.M. Jonker Files PDF 2005.07404_1.pdf 2.69 MB Close viewer /islandora/object/uuid:f46e64c0-a1ce-457e-bf56-748d3bb73673/datastream/OBJ/view