Towards Understanding Machine Learning Testing in Practise

Towards Understanding Machine Learning Testing in Practise

Shome, A. (TU Delft Software Engineering)
Cruz, Luis (TU Delft Software Engineering)
van Deursen, A. (TU Delft Software Technology)

Software Technology

2023

Visualisations drive all aspects of the Machine Learning (ML) Development Cycle but remain a vastly untapped resource by the research community. ML testing is a highly interactive and cognitive process which demands a human-in-the-loop approach. Besides writing tests for the code base, bulk of the evaluation requires application of domain expertise to generate and interpret visualisations. To gain a deeper insight into the process of testing ML systems, we propose to study visualisations of ML pipelines by mining Jupyter notebooks. We propose a two prong approach in conducting the analysis. First, gather general insights and trends using a qualitative study of a smaller sample of notebooks. And then use the knowledge gained from the qualitative study to design an empirical study using a larger sample of notebooks. Computational notebooks provide a rich source of information in three formats - text, code and images. We hope to utilise existing work in image analysis and Natural Language Processing for text and code, to analyse the information present in notebooks. We hope to gain a new perspective into program comprehension and debugging in the context of ML testing.

AI Engineering
Computational Notebooks
Data Mining
Image Analysis
Machine Learning Testing
Natural Language Processing
NLP for Code

http://resolver.tudelft.nl/uuid:6f391be2-a267-48c3-878b-3164bfeb7279

https://doi.org/10.1109/CAIN58948.2023.00028

IEEE

2024-01-01

9798350301137

Proceedings - 2023 IEEE/ACM 2nd International Conference on AI Engineering - Software Engineering for AI, CAIN 2023

2nd IEEE/ACM International Conference on AI Engineering - Software Engineering for AI, CAIN 2023, 2023-05-15 → 2023-05-16, Melbourne, Australia

Proceedings - 2023 IEEE/ACM 2nd International Conference on AI Engineering - Software Engineering for AI, CAIN 2023

Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2023 A. Shome, Luis Cruz, A. van Deursen