Using Decision Trees produced by Generative Adversarial Imitation Learning to give insight into black box Reinforcement Learning models

Using Decision Trees produced by Generative Adversarial Imitation Learning to give insight into black box Reinforcement Learning models

Meijer, Caspar (TU Delft Electrical Engineering, Mathematics and Computer Science; TU Delft Algorithmics)

Lukina, A. (mentor)
Murukannaiah, P.K. (graduation committee)

Delft University of Technology

Computer Science and Engineering

CSE3000 Research Project

2022-01-28

Machine learning models are increasingly being used in fields that have a direct impact on the lives of humans. Often these machine learning models are black-box models and they lack transparency and trust which is holding back the implementation. To increase transparency and trust this research investigates whether imitation learning, specifically Generative Adversarial ImitationLearning (GAIL), can be used to give insights into the black-box models by extracting decision trees. To achieve this, an extension of GAIL was made allowing it to extract decision trees. The decision trees were then measured in terms of performance, fidelity, behavior, and interpretability in three different environments. We find that GAIL is able to extract decision trees with high fidelity and can give insightful information into the expert models. Moreover, further research can be done on more complex environments and black-box models, other surrogate models, and possibilities for more specific local insights.

Generative Adversarial Network
Generative Adversarial Imitation Learning
Reinforcement Learning (RL)
Imitation Learning
Interpretability
Explainable Reinforcement Learning
Python
Decision Trees
Markov Decision Process
Neural network
black box
Machine learning

http://resolver.tudelft.nl/uuid:dbc92b47-7fae-473a-ad53-e906ad7b2008

Student theses

bachelor thesis

© 2022 Caspar Meijer