Print Email Facebook Twitter What do You Mean? Interpreting Image Classification with Crowdsourced Concept Extraction and Analysis Title What do You Mean? Interpreting Image Classification with Crowdsourced Concept Extraction and Analysis Author Balayn, A.M.A. (TU Delft Web Information Systems) SOILIS, P. (TU Delft Electrical Engineering, Mathematics and Computer Science) Lofi, C. (TU Delft Web Information Systems) Yang, J. (TU Delft Web Information Systems) Bozzon, A. (TU Delft Human-Centred Artificial Intelligence) Faculty Electrical Engineering, Mathematics and Computer Science Date 2021 Abstract Global interpretability is a vital requirement for image classification applications. Existing interpretability methods mainly explain a model behavior by identifying salient image patches, which require manual efforts from users to make sense of, and also do not typically support model validation with questions that investigate multiple visual concepts. In this paper, we introduce a scalable human-in-the-loop approach for global interpretability. Salient image areas identified by local interpretability methods are annotated with semantic concepts, which are then aggregated into a tabular representation of images to facilitate automatic statistical analysis of model behavior. We show that this approach answers interpretability needs for both model validation and exploration, and provides semantically more diverse, informative, and relevant explanations while still allowing for scalable and cost-efficient execution. Subject Concept extractionHuman computationImage classificationMachine-learning interpretability To reference this document use: http://resolver.tudelft.nl/uuid:1553d362-a9b1-4ce6-aed7-83c9b4374225 DOI https://doi.org/10.1145/3442381.3450069 ISBN 978-1-4503-8312-7 Source The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021 Event International World Wide Web Conference 2021, 2021-04-19 → 2021-04-23, Ljubljana, Slovakia Series The Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021 Part of collection Institutional Repository Document type conference paper Rights © 2021 A.M.A. Balayn, P. SOILIS, C. Lofi, J. Yang, A. Bozzon Files PDF 3442381.3450069.pdf 3.41 MB Close viewer /islandora/object/uuid:1553d362-a9b1-4ce6-aed7-83c9b4374225/datastream/OBJ/view