SparCAssist

A Model Risk Assessment Assistant Based on Sparse Generated Counterfactuals

Conference paper (2022)

Authors

Zijian Zhang L3S Research Center

Vinay Setty L3S Research Center, University of Stavanger

A. Anand Web Information Systems - , L3S Research Center

Research Group

Web Information Systems () (TU Delft)

DOI: https://doi.org/10.1145/3477495.3531677

Interpretable machine learning Counterfactual interpretation Data-annotation tools Human-in-the-loop machine learning

To reference this document use:

http://resolver.tudelft.nl/uuid:553f3377-f4bd-4b19-a0ee-974d9947f62b

More Info

expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Web Information Systems

Abstract

We introduce SparCAssist, a general-purpose risk assessment tool for the machine learning models trained for language tasks. It evaluates models' risk by inspecting their behavior on counterfactuals, namely out-of-distribution instances generated based on the given data instance. The counterfactuals are generated by replacing tokens in rational subsequences identified by ExPred, while the replacements are retrieved using HotFlip or the Masked-Language-Model-based algorithms. The main purpose of our system is to help the human annotators to assess the model's risk on deployment. The counterfactual instances generated during the assessment are the by-product and can be used to train more robust NLP models in the future.

Files

3477495.3531677.pdf

(pdf | 1.31 Mb)

- Embargo expired in 01-07-2023

Unknown license