Interpretability and performance comparisons of decision tree surrogate models produced by AGGREVATE

Abstract

Imitation learning algorithms, such as AggreVaTe, have proven successful in solving many challenging tasks accurately and efficiently. In practice, however, they have not been applied quite as much. Black box policies produced by imitation learning algorithms can not ensure the safety needed for real-world applications. This paper extends this field by outputting a decision tree surrogate model from AggreVaTe and comparing it to other imitation learning algorithms (Behavioral cloning, GAIL, DAgger, Viper) in terms of interpretability as well as performance. A modification to AggreVaTe is proposed to train decision tree policies that can be used to explain individual decision-making of the model. Three simple environments of open AI Gym have been used to compare the multiple different imitation learning algorithms. The experiments reveal that on performance, AggreVaTe overall performs better than the baseline behavioral cloning but slightly worse than GAIL, DAgger and Viper. AggreVaTe performs slightly better in terms of interpretability on these simple environments. Both of these conclusions could be explained by the fewer data points used by AggreVaTe. Further study can be done into the subjective interpretability of AggreVaTe as well as more difficult environments where the extra exploring of AggreVaTe should help with finding the best solution.