Enabling Human-In-The-Loop Interpretability Methods of Machine Learning Models

Abstract

To study how to involve the end-users in the development of machine learning explainability, this project has chosen the context of bird species identification. It intends to develop a platform where the end-users can learn bird knowledge while contributing to building the explainability of machine learning models. Among all the methods that equips machine learning models with explainability, this project adopts a framework called SECA (Semantic Concept Extraction and Analysis). In this framework, we require human-made-annotations to be made to the saliency maps of training photos to provide semantically understandable explanations to the end-users. On the other hand, we hope that the process of making annotations will also benefit the human annotators’ skills in bird species identification, in order to motivate their participation.
Two main goals of the user research were: to understand the users’ needs for learning and to know their capability in making the annotations needed by the project owners.
The user research started with qualitative and quantitative research to understand the current practices of the bird hobbyists, to define the target user groups, which were the birders with zero or little expertise.
Then, in order to link their learning needs to the capability of machine learning explanations, three prototypes were built to collect their feedback. It was found out that they didn’t care much about the justification or transparency of bird ID apps, compared to learning knowledge in distinguishing birds. Then came the annotation test when we found the participants were able to finish the annotation task with high correctness (>93% on average). And the most popular annotations of each task were 100% correct.
Finally, we built a functional, high-fidelity prototype with experiential interfaces and interactions, and tested it among 3 of the target users. They had positive feedback on the prototype’s usability and the overall workflow, which proved the feasibility of our concepts. Recommendations on usability were drawn at the end of this test. Throughout the research and design phases in this project, we have developed an approach to involve end-users in the annotation process of an explainable bird species identification model for their own benefit of fun and learning, which could potentially be applied to broader deployments