Dealing with competing requirements surrounding machine learning performance and explainability

Abstract

For the future demand prediction of identification documents the National Office for Identity Data is interested in a new prediction model based on machine learning techniques. Due to the existence of many different machine learning algorithms and the often competing model requirements regarding model performance and explainability it can be difficult to determine which machine learning algorithm to use. This research aims to find a solution to help decision-makers determine what machine learning algorithms they should use while having to account for competing model requirements. This is done by performing a case study for the National Office for Identity Data in which the use of machine learning for a new prototype prediction model is explored. Additionally, a novel explainability assessment table is designed through which the explored machine learning algorithms are assessed on their explainability. By combining the results from the case study and the explainability assessment analysis a final design is created in the form of a decision support framework. This framework aims to support decision-makers in their process of finding a suitable machine learning algorithm when dealing with competing model requirements. Future researchers are recommended to either extend upon the decision support framework by making it more accessible for people with a less technical background or to analyse how a more standardised definition of model explainability can be developed and implemented in the field of machine learning.