Hierarchical Prediction of Registration Misalignment using a Convolutional LSTM
Application to Chest CT Scans
Hessam Sokooti (Leiden University Medical Center)
Sahar Yousefi (Leiden University Medical Center)
Mohamed S. Elmahdy (Leiden University Medical Center)
Boudewijn P.F. Lelieveldt (TU Delft - Pattern Recognition and Bioinformatics, Leiden University Medical Center)
Marius Staring (TU Delft - Pattern Recognition and Bioinformatics, Leiden University Medical Center)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
In this paper we propose a supervised method to predict registration misalignment using convolutional neural networks (CNNs). This task is casted to a classification problem with multiple classes of misalignment: 'correct' 0-3 mm, 'poor' 3-6 mm and 'wrong' over 6 mm. Rather than a direct prediction, we propose a hierarchical approach, where the prediction is gradually refined from coarse to fine. Our solution is based on a convolutional Long Short-Term Memory (LSTM), using hierarchical misalignment predictions on three resolutions of the image pair, leveraging the intrinsic strengths of an LSTM for this problem. The convolutional LSTM is trained on a set of artificially generated image pairs obtained from artificial displacement vector fields (DVFs). Results on chest CT scans show that incorporating multi-resolution information, and the hierarchical use via an LSTM for this, leads to overall better F1 scores, with fewer misclassifications in a well-tuned registration setup. The final system yields an accuracy of 87.1%, and an average F1 score of 66.4% aggregated in two independent chest CT scan studies.
help_outline