Deep Learning for 4D Longitudinal Segmentation of MRI Brain Tissues and Glioma

Abstract

Glioma is a kind of slow-growing brain tumor which may result in severe seizures. Currently a major tool used to detect and diagnose the glioma is MRI scan. To better analyze the medical image, segmentation is usually conducted as a basic step for further processing, which partitions an integrate image into multiple physically meaningful regions by annotating objects and boundaries. Deep learning based segmentation methods have attracted significant interest due to their high efficiency and strong generalization ability. With the increasing demands of high-quality segmentation of bio-tissues in medical region, plenty of innovative approaches were proposed to expand the boundary of segmentation capability of deep learning models by taking the spatial or temporal constraints of bio-structure into consideration. Although longitudinal segmentation in 2D natural image sequences has made a lot of success, the potential of deep learning network in segmenting a series of chronological 3DMRI images in terms of improving consistency remains unclear. This thesis aims to investigate whether deep learning models are able to increase segmentation accuracy as well as consistency in longitudinal 3D images, specifically focusing on introducing Recurrent Neural Network(RNN) to 3D Convolutional Neural Network(CNN) for 4D segmentation. In addition to the implementation of several U-Net variants as CNN backbone, three types of longitudinal connection strategies are proposed. A hierarchical workflow is followed to create the optimal version of longitudinal network based on combining multiple CNN variants and connection strategies. The evaluation of the 4D network shows that segmentation accuracy of the longitudinal model is limited by its CNN backbone and temporal information can partially improve the segmentation consistency with regard to maintaining the highest proportion of normal tissue unchangeable over time.