Tissue Characterization by Deep Learning in Medical Hyperspectral Images

Abstract

Hyperspectral imaging (HSI) is a promising imaging modality in medical applications, especially for non-invasive and non-contact disease diagnosis and image-guided surgery. Encoding both spatial and spectral information, it can detect subtle changes in the biochemical and morphological properties of a tissue, revealing the early progression of a pathological condition like cancer. Previous medical hyperspectral image analysis approaches depended on handcrafted features or feature extraction principle, requiring considerable domain expertise. To overcome this, automatic feature learning approaches like convolutional neural networks (CNN), previously used in tasks like classification, detection and segmentation in medical images were applied to hyperspectral data, although in a limited number of research studies. This thesis was proposed to review the state-of-the-art in medical hyperspectral image analysis, identify the limitations in current methods, and present a proof-of-concept for using limited hyperspectral image data in CNN-driven tissue characterization.



The goal of this thesis is to characterize, using CNNs, ex vivo head and neck (tongue) tissue of patients affected by tumors. While previous work in this field implemented patch-based classification of tissue, in this thesis, a pixel-wise classification approach was proposed to obtain a smooth and continuous segmentation of hyperspectral images. To this end, two types of CNN models were trained from scratch using limited labelled training data, one to automatically learn the spectral features present in the hyperspectral data and the other to learn the combined spectral-spatial features from the hyperspectral data. Performance of four different trained models was evaluated by using a leave-one-out testing scheme, with the spectral-spatial learning approach with larger input spatial dimensions outperforming the other considered approaches.