Debugging Machine Learning on Time-series Data

Debugging Machine Learning on Time-series Data

Zheng, Meng (TU Delft Electrical Engineering, Mathematics and Computer Science)

Yang, J. (mentor)
Wang, Q. (mentor)
Dauwels, J.H.G. (mentor)

Delft University of Technology

Electrical Engineering

2022-08-29

Machine learning models are so-called a "black box," which means people can not easily observe the relationship between the output and input or explain the reason for such results. In recent years, much work has been done on interpretable machine-learning, such as Shapley values, counterfactual explanations, partial dependence plots, or saliency maps . However, methods based on such works target models trained with 2-dimensional static data, and trim work has been done or implemented on debugging models with timeseries data. This research targets identifying what a model with a time-series dataset really knows and what it doesn’t know. To further evaluate such a method, LightDigit, which Hao Liu proposes, will be implemented.
LightDigit is an embedded-based Machine Learning project which focuses on achieving touch-free interaction with public touchscreens and buttons. This project used 3X3 photodiodes. The key idea is to recognize the shadowed area by hand while using finger-writing digits above the screen and capture the time series data.
We will feed these time series data and labels to train the deep learning model. By iterating multiple epochs,
the model shall be able to predict the corresponding digit.
Through Hao Liu’s work, still, the model can achieve an accuracy of 91 %. To further improve this work, it
could be essential to know what this model knows and what it should show. After gaining this knowledge, it will be possible for further research to be done on improving the performance of this model. Agathe Balayn provided a human-in-the-loop approach to knowing what a model knows. This work proposed the SECA framework, which firstly uses the Local interpretability method to generate a saliency map, then saliency map are provided for annotators to annotate, and finally process these data and gain the result,
However, Agathe’s work focus on static data, like images. This project will migrate the SECA framework on time-series data, and take LightDigit as a use case to determine what such a time-series-based model really
knows and what such a model should know.

Interpretable machine learning
debugging
Human in the loop

http://resolver.tudelft.nl/uuid:6bb69ca2-06d0-457f-b0ef-0d1b87caf9f3

Student theses

master thesis

© 2022 Meng Zheng