Active Monitoring of Neural Networks

Conference paper (2021)

Authors

A. Lukina Algorithmics - EEMCS
Christian Schilling Universität Konstanz
Thomas A. Henzinger Institute of Science and Technology (IST Austria)
Research Group
Algorithmics (EEMCS) (TU Delft)
Copyright: © 2021 A. Lukina, Christian Schilling, Thomas A. Henzinger
More Info
expand_more

Published Date

2021

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Software Technology

Research Group

Algorithmics

Abstract

Neural-network classifiers are trained to achieve high prediction accuracy. However, their performance still suffers from frequently appearing inputs of unknown classes. As a component of a cyber-physical system, the classifier in this case can no longer be reliable and is typically retrained. We propose an algorithmic framework for monitoring reliability of a neural network. In contrast to static detection, a monitor wrapped in our framework operates in parallel with the classifier, communicates interpretable labeling queries to the human user, and incrementally adapts to their feedback.