Dealing with competing requirements surrounding machine learning performance and explainability

Dealing with competing requirements surrounding machine learning performance and explainability: Designing a decision support framework for choosing specific machine learning models while dealing with competing model requirements; A case study for the RvIG

Beens, Tim (TU Delft Technology, Policy and Management; TU Delft Multi Actor Systems)

van der Voort, H.G. (mentor)
Metoui, N. (graduation committee)
Mulder, Nita (graduation committee)
Mostert, Chris (graduation committee)

Delft University of Technology

Engineering and Policy Analysis

2022-08-29

For the future demand prediction of identification documents the National Office for Identity Data is interested in a new prediction model based on machine learning techniques. Due to the existence of many different machine learning algorithms and the often competing model requirements regarding model performance and explainability it can be difficult to determine which machine learning algorithm to use. This research aims to find a solution to help decision-makers determine what machine learning algorithms they should use while having to account for competing model requirements. This is done by performing a case study for the National Office for Identity Data in which the use of machine learning for a new prototype prediction model is explored. Additionally, a novel explainability assessment table is designed through which the explored machine learning algorithms are assessed on their explainability. By combining the results from the case study and the explainability assessment analysis a final design is created in the form of a decision support framework. This framework aims to support decision-makers in their process of finding a suitable machine learning algorithm when dealing with competing model requirements. Future researchers are recommended to either extend upon the decision support framework by making it more accessible for people with a less technical background or to analyse how a more standardised definition of model explainability can be developed and implemented in the field of machine learning.

Machine learning
Competing model requirements
Decision support framework
Case study
Multivariate time-series analysis

http://resolver.tudelft.nl/uuid:3e4c4d45-7410-4b07-a00f-2d1b8a5b7531

Student theses

master thesis

© 2022 Tim Beens