Generalized Optimistic Q-Learning with Provable Efficiency

Generalized Optimistic Q-Learning with Provable Efficiency

Neustroev, G. (TU Delft Algorithmics)
de Weerdt, M.M. (TU Delft Algorithmics)

An, Bo (editor)
El Fallah Seghrouchni, Amal (editor)
Sukthankar, Gita (editor)

2020-05

Reinforcement learning (RL), like any on-line learning method, inevitably faces the exploration-exploitation dilemma. When a learning algorithm requires as few data samples as possible, it is called sample efficient. The design of sample-efficient algorithms is an important area of research. Interestingly, all currently known provably efficient model-free RL algorithms utilize the same well-known principle of optimism in the face of uncertainty. We unite these existing algorithms into a single general model-free optimistic RL framework. We show how this facilitates the design of new optimistic model-free RL algorithms by simplifying the analysis of their efficiency. Finally, we propose one such new algorithm and demonstrate its performance in an experimental study.

Model-free learning
Reinforcement learning
Sample efficiency

http://resolver.tudelft.nl/uuid:6ea9bc9a-b697-40fc-8c7f-2c04947b74ce

International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS)

978-1-4503-7518-4

Proceedings of the 19th International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2020

AAMAS 2020, 2020-05-09 → 2020-05-13, Auckland, New Zealand

Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS, 1548-8403, 2020-May

Virtual/online event due to COVID-19

Institutional Repository

conference paper

© 2020 G. Neustroev, M.M. de Weerdt