Reducing Manual Labour in Forensic Microtrace Recognition with Deep Learning

Abstract

Forensic microtrace investigation relies on a time- and labour-intensive process of manually analysing samples via microscopy. To aid forensic experts in their investigations, an image recognition model for microtrace localisation and classification is needed. This work investigates the trace recognition accuracy that can be achieved by analysing images captured with automated microscopy through deep learning. Fibres, hairs, skin, glass and sand are pixel-wise classified in microscopy scans of tape-lift samples. As deep learning requires extensive amounts of annotated training data, pretraining is investigated to minimise the required annotation workload. ImageNet pretraining, pretraining with self-supervised learning and a sequential application of these approaches are tested. It is found that pretrained models are able to reduce the required annotated data twofold compared to models trained from scratch, while retaining the prediction accuracy. While the ImageNet-pretrained models outperform the self-supervised-pretrained models, the highest accuracy is achieved by combining the two approaches. With this, traces are recognised with a mean intersection over union of 0.56 when training on only 2.2 dm2 of annotated tape lift scans.