Title
Synthesising Reinforcement Learning Policies Through Set-Valued Inductive Rule Learning
Author
Coppens, Youri (Vrije Universiteit Brussel)
Steckelmacher, Denis (Vrije Universiteit Brussel)
Jonker, C.M. (TU Delft Interactive Intelligence; Universiteit Leiden) ![ORCID 0000-0003-4780-7461 ORCID 0000-0003-4780-7461](/sites/all/themes/tud_repo3/img/icons/orcid_16x16.png)
Nowe, A.S.P. (Vrije Universiteit Brussel)
Contributor
Heintz, Fredrik (editor)
Milano, Michela (editor)
O’Sullivan, Barry (editor)
Date
2021
Abstract
Today’s advanced Reinforcement Learning algorithms produce black-box policies, that are often difficult to interpret and trust for a person. We introduce a policy distilling algorithm, building on the CN2 rule mining algorithm, that distills the policy into a rule-based decision system. At the core of our approach is the fact that an RL process does not just learn a policy, a mapping from states to actions, but also produces extra meta-information, such as action values indicating the quality of alternative actions. This meta-information can indicate whether more than one action is near-optimal for a certain state. We extend CN2 to make it able to leverage knowledge about equally-good actions to distill the policy into fewer rules, increasing its interpretability by a person. Then, to ensure that the rules explain a valid, non-degenerate policy, we introduce a refinement algorithm that fine-tunes the rules to obtain good performance when executed in the environment. We demonstrate the applicability of our algorithm on the Mario AI benchmark, a complex task that requires modern reinforcement learning algorithms including neural networks. The explanations we produce capture the learned policy in only a few rules, that allow a person to understand what the black-box agent learned. Source code: https://gitlab.ai.vub.ac.be/yocoppen/svcn2.
Subject
Explainable AI
Inductive rule learning
Policy distillation
Reinforcement Learning
To reference this document use:
http://resolver.tudelft.nl/uuid:1ab03460-6575-4c7e-9aaf-8b5e3b307aa9
DOI
https://doi.org/10.1007/978-3-030-73959-1_15
Publisher
Springer
Embargo date
2022-01-27
ISBN
9783030739584
Source
Trustworthy AI – Integrating Learning, Optimization and Reasoning - First International Workshop, TAILOR 2020, Revised Selected Papers
Event
1st International Workshop on Trustworthy AI – Integrating Learning, Optimization and Reasoning, TAILOR 2020 held as a part of European Conference on Artificial Intelligence, ECAI 2020, 2020-09-04 → 2020-09-05, Virtual, Online
Series
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 12641 LNAI
Bibliographical note
Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.
Part of collection
Institutional Repository
Document type
conference paper
Rights
© 2021 Youri Coppens, Denis Steckelmacher, C.M. Jonker, A.S.P. Nowe