Hierarchical Prediction of Registration Misalignment using a Convolutional LSTM

Hierarchical Prediction of Registration Misalignment using a Convolutional LSTM: Application to Chest CT Scans

Sokooti, Hessam (Leiden University Medical Center)
Yousefi, Sahar (Leiden University Medical Center)
Elmahdy, Mohamed S. (Leiden University Medical Center)
Lelieveldt, B.P.F. (TU Delft Pattern Recognition and Bioinformatics; Leiden University Medical Center)
Staring, M. (TU Delft Pattern Recognition and Bioinformatics; Leiden University Medical Center)

2021

In this paper we propose a supervised method to predict registration misalignment using convolutional neural networks (CNNs). This task is casted to a classification problem with multiple classes of misalignment: 'correct' 0-3 mm, 'poor' 3-6 mm and 'wrong' over 6 mm. Rather than a direct prediction, we propose a hierarchical approach, where the prediction is gradually refined from coarse to fine. Our solution is based on a convolutional Long Short-Term Memory (LSTM), using hierarchical misalignment predictions on three resolutions of the image pair, leveraging the intrinsic strengths of an LSTM for this problem. The convolutional LSTM is trained on a set of artificially generated image pairs obtained from artificial displacement vector fields (DVFs). Results on chest CT scans show that incorporating multi-resolution information, and the hierarchical use via an LSTM for this, leads to overall better F1 scores, with fewer misclassifications in a well-tuned registration setup. The final system yields an accuracy of 87.1%, and an average F1 score of 66.4% aggregated in two independent chest CT scan studies.

Biomedical imaging
Computed tomography
convolutional neural networks
Decoding
Feature extraction
hierarchical classification
Image registration
image registration
Image resolution
registration misalignment
Task analysis

http://resolver.tudelft.nl/uuid:50ce781d-296c-4ea3-baf0-5bb7add3cc28

https://doi.org/10.1109/ACCESS.2021.3074124

2169-3536

IEEE Access, 9, 62008-62020

Institutional Repository

journal article

© 2021 Hessam Sokooti, Sahar Yousefi, Mohamed S. Elmahdy, B.P.F. Lelieveldt, M. Staring