Deep learning is a growing field of research and and so is the application of deep learning to the analysis of medical images. Convolutional neural networks are used to diagnose diseases, determine risk of disease development, finding the exact area of abnormalities and and so on. Mammography is an imaging technique, which aims at the early detection of breast cancer. Invasive Lobular Carcinoma (ILC) is a type of breast cancer with properties that make it less visible on mammography. This study compares six models which apply convolutional neural networks to detect breast cancer, on its ability to detect ILC. Furthermore, transfer learning with ILC and healthy images is applied to one of these models to improve the performance on ILC data. For the evaluated breast cancer detection models, the performance on ILC data is worse than for a dataset which includes all breast cancer types and more healthy images. The model that transfer learning is applied to performs better on ILC data after transfer learning than before, with an increase of 0.09 for the AUC value. Additional analyses of the results show that women with high breast density have a lower chance of getting a correct ILC diagnosis from the model than women with low beast density and this also holds for other types of breast cancer. Lastly, the model outcomes are compared to radiologist reviews, to determine the additional value of models to the routine screening performed by radiologists. Within the images that are labeled as healthy by the radiologists, a model could be applied to detect tumor that have been missed by radiologists. When a specificity of 89% was allowed, 23% of the missed tumors could have been detected by the original GMIC model. In this way, the models used in this study and other deep learning models that are in development now, can contribute to breast cancer detection from mammography, and ILC specifically.

Patients with 1p/19q co-deleted low grade glioma (LGGs) have better prognosis and react better to certain treatments than patients with intact 1p/19q LGG. Currently, information about the 1p/19q co-deletion status is obtained by means of an invasive procedure called biopsy. As an alternative, non-invasive techniques to extract this information from medical images are being studied. Recent research suggests that local binary patterns (LBPs), a textural image descriptor, are an important feature which can predict the 1p/19q co-deletion from MRI scans. In this project we report the effect of including LBP information in a convolutional neural network (CNN) to predict the 1p/19q co-deletion status in patients suffering from a presumed LGG using pre-operative MRI scans. A combination of convolutional filters was designed and included in the CNN, resulting into local binary convolutional neural networks (LBCNNs). Three LBP descriptors, each of them representing a different textural scale, were studied, as well as the combination of the three. A default CNN without LBPs was also studied. To validate the designed filters and to study more sophisticated LBPs images like the uniform LBPs, pre-computed LBP images were directly input to the CNN. An in-house multi-institution MRI dataset consisting of 284 patients who had undergone a biopsy or resection before the treatment, and with available pre-operative T1-weighted post contrast and T2-weighted scans was used to train the different network architectures. An independent dataset consisting of 129 patients was used to validate the results. The performance of the LBCNNs was compared to the performance of the CNN. The performance of the CNN and LBCNNs was similar, reporting an area under the receiver operating characteristic curve (AUC) ranging from 0.816 to 0.872 for the different architectures. These findings suggest that the CNN can extract information relative to LBPs by itself. In addition, pre-computed uniform LBPs report similar metrics (AUC: 0.819), suggesting that they do not add new information.

Purpose Manufacturer’s predictions of ablation zone dimensions are the current directives for treatment planning in thermal ablation, while they are mostly based on ex vivo experiments making its reliability questionable. The aim of this study is to determine the correspondence in dimensions, volume, shape and overlap of the manufactures’ predicted with the clinically realized ablation zones following thermal ablation of hepatocellular carcinoma (HCC). The secondary objective is to determine the effect of tumor- and liver characteristics on this correspondence. Methods Data was retrospectively collected from two prospective studies. A registered pre-ablation and post-ablation computed tomography scan with liver, tumor and ablation zone segmentations were available for analysis. Needle position reconstruction was performed based on image visual assessment (e.g., gas formation, tumor location and subcapsular hemorrhage) using in-house developed software. The dimensions of the predicted ablation zone were derived from the manufacturer’s chart corresponding to treatment settings used during ablation. The long axis diameter (LAD), short axis diameter (SAD) and volume of the realized and predicted ablation zone were compared. The overlap was determined using the Dice similarity coefficient (DSC) and the average surface deviation between the realized and predicted ablation zone. The effect of tumor location, vascular proximity and liver cirrhosis on the overlap was quantified using the DSC and average surface deviation. Results Nineteen patients and 21 ablations were included for analysis. The median realized volume did not significantly differ from the predicted volume, 25.7 cm3 and 22.6 cm3 respectively (p = 0.526). The median LAD and SAD of the realized ablation zone differed significantly from the manufacturer’s prediction (51.9 mm vs 40.0 mm, p<0.001 and 36.9 mm vs 35.0 mm, p<0.001 respectively). The predicted ablation zone corresponds to the realized ablation zone with a mean DSC of 0.73 and mean average surface deviation of 3.04 mm. Tumor location and vascular proximity did not affect the overlap between the realized and predicted ablation zone. The effect of liver cirrhosis was not assessed due to a low sample size of HCC in a non-cirrhotic liver (n=2). Conclusion The manufacturer’s predicted volume of liver ablation zones corresponds well to the clinically realized ablation volume. However, the LAD and SAD are underestimated by the manufacturers. The shape and overlap of the predicted and realized ablation zone were sufficient. Further studies evaluating the effect of tumor- and liver characteristics on the correspondence of the predicted with the realized ablation zone with a larger patient cohort is needed.

Glioma is a kind of slow-growing brain tumor which may result in severe seizures. Currently a major tool used to detect and diagnose the glioma is MRI scan. To better analyze the medical image, segmentation is usually conducted as a basic step for further processing, which partitions an integrate image into multiple physically meaningful regions by annotating objects and boundaries. Deep learning based segmentation methods have attracted significant interest due to their high efficiency and strong generalization ability. With the increasing demands of high-quality segmentation of bio-tissues in medical region, plenty of innovative approaches were proposed to expand the boundary of segmentation capability of deep learning models by taking the spatial or temporal constraints of bio-structure into consideration. Although longitudinal segmentation in 2D natural image sequences has made a lot of success, the potential of deep learning network in segmenting a series of chronological 3DMRI images in terms of improving consistency remains unclear. This thesis aims to investigate whether deep learning models are able to increase segmentation accuracy as well as consistency in longitudinal 3D images, specifically focusing on introducing Recurrent Neural Network(RNN) to 3D Convolutional Neural Network(CNN) for 4D segmentation. In addition to the implementation of several U-Net variants as CNN backbone, three types of longitudinal connection strategies are proposed. A hierarchical workflow is followed to create the optimal version of longitudinal network based on combining multiple CNN variants and connection strategies. The evaluation of the 4D network shows that segmentation accuracy of the longitudinal model is limited by its CNN backbone and temporal information can partially improve the segmentation consistency with regard to maintaining the highest proportion of normal tissue unchangeable over time.

Background: Histopathological growth patterns (HGP) are a biomarker for predicting survival and systemic treatment effectiveness in colorectal liver metastasis (CRLM). Currently, HGP assessment in CRLM requires the resection specimen. Predicting the HGP from preoperative medical imaging could allow more personalised care and better outcomes. Methods: 252 patients underwent CRLM resection between 2004 and 2018 without receiving any systemic treatment. Patients were characterised as having either pure desmoplastic growth (dHGP) or any other type of growth pattern combination (non-dHGP) (21% dHGP; 79% non-dHGP). These categories were chosen because pure desmoplastic growth is predictive of better overall survival. regions of interest were automatically extracted using a UNet based segmentation model. These ROIs were passed to a radiomics model and a deep learning model to classify between dHGP/non-dHGP and predict the fraction of dHGP. Results: The best-performing classification method was the radiomic approach achieving an AUC of 0.67 (95% CI: 0.58-0.78), whereas the best-performance deep learning model achieved an average AUC value of 0.59 (95% CI: 0.53-0.65). Additionally, regression predicting the fraction of dHGP failed, with the predicted values showing no significant correlation with the actual value. Conclusions: Radiomics can be used to assess HGP, however further improvements in predictive performance are needed before these methods can be applied.

Primary liver cancer is a commonly diagnosed cancer and accurate diagnosis is crucial for treatment planning. To differentiate between malignant and benign liver tumors, contrast-enhanced MRI is typically used as it provides information over multiple contrast phases. However, diagnosis based on MRI is challenging. In this study, automatic classification is used to distinguish common primary liver tumors. Imaging data from 102 patients with malignant (hepatocellular carcinoma) and benign (focal nodular hyperplasia and hepatocellular adenoma) primary liver tumors was used for binary classification through radiomics and deep learning approaches. The radiomics method was applied with the use of the open-source toolbox WORC. The deep learning model was based on the ResNet-10 architecture. The data input consisted of individual and combined phases of contrast-enhanced T1-weighted and T2-weighted MRI. The highest performance values were found for the radiomics approach that combined the precontrast, arterial, portal venous, and delayed contrast phases together with T2-weighted MRI, with an AUC of 0.92. The deep learning model scored an AUC of 0.83 with this data input, however substantial overfitting occurred due to the limited sample size. In conclusion, the radiomics classifiers based on combined contrast-enhanced T1-weighted and T2-weighted MRI can differentiate malignant from benign primary liver tumors with limited data samples. The classification task is too complex with the given data when using a ResNet-10 model and should be applied to an extended dataset.

The periconceptional period, encompassing the embryonic phase, is a critical window where a majority of reproductive failures, pregnancy complications, and adverse pregnancy outcomes arise. The Carnegie staging system comprises 23 stages which are based on embryonic morphological development. This allows for the assessment of normal and abnormal embryonic development during this critical period. In-utero Carnegie staging using three-dimensional (3D) ultrasound scans visualized with virtual reality offers valuable insights but is currently a time-consuming manual process. To address this, we propose a deep learning approach for Carnegie staging in 3D ultrasound scans. We used a dataset comprising 1413 3D ultrasound scans from the Rotterdam Periconceptional Cohort, annotated with Carnegie stages spanning from stages 13 to 23, including fetal subjects. Various training strategies were explored. We compared a metric regression approach, which considers the ordered nature of the Carnegie stages by treating the Carnegie stages as a continuous variable, with a multi-class classification approach, treating stages as independent categories. Additionally, we evaluated the influence of using a loss function accommodating the categorical nature of the Carnegie stages in the metric regression approach and examined the impact of incorporating embryonic size in the model input. Ultimately, a regression approach using the Mean Squared Error (MSE) loss function emerged as the optimal choice. This model achieved a classification accuracy of 0.59 and a Root Mean Squared Error (RMSE) of 0.62 on the test set. This performance is comparable to an intermediate human rater, which achieved an accuracy of 0.63 and a RMSE of 0.65. Our findings represent a significant step towards the development of an automated Carnegie staging method, offering the potential for a more comprehensive evaluation of the critical embryonic phase in the clinic.

Semantic Segmentation of medical images are used to improve diagnosis and treatment. In recent years, the application of machine learning methods are increasingly used. However, the design of these models is difficult and time-consuming. In this thesis, we investigated the automation of this process using an Automated Machine Learning (AutoML) algorithm called Bayesian Optimization with HyperBand (BOHB). We found several hyperparameters significantly influencing the performance and confirmed that BOHB is applicable to these kind of problems. However, we did not find a significant difference between BOHB and Random Search on a small search, stressing the need of matching the search space to the search strategy. Finally, our approach was only slightly worse than the current state-of-the-art, which shows that AutoML has potential in this field.

Hypertrophic cardiomyopathy (HCM) is known as a frequent, genetic cardiovascular disease, often caused by mutations of sarcomere protein genes. HCM is primarily characterized by the presence of an increased left ventricular wall thickness, i.e. left ventricular hypertrophy (LVH). However, the disease appears to be asymptomatic in some patients, which makes it a diagnostic challenge. Mutation carriers of HCM who have not yet developed LVH are called genotype-positive left ventricular hypertrophy-negative (G+/LVH-) patients. The primary aim of this study was to investigate whether a radiomics model is able to distinguish between G+/LVH- patients and healthy controls, based on cardiac magnetic resonance (CMR) images. In total three datasets are analysed. A development dataset was used to develop different radiomics models and to evaluate the performance of the models. The models were validated on both the prospective validation dataset and external validation dataset. G+/LVH- patients had to be known to carry a class 4 (likely pathogenic) or class 5 (pathogenic) gene mutation for HCM and a maximum left ventricular wall thickness of <13mm. Endocardial and epicardial borders were manually and automatically segmented on long-axis view (2-chamber (2CH), 3-chamber (3CH), and 4-chamber (4CH)) and short-axis (SA) view in both end-diastolic (ED) and end-systolic (ES) phase. From these segmentation 555 features including shape, intensity and texture were extracted. Evaluation of radiomics models was performed through a 100x stratified random-split cross-validation in development dataset. Next, the models were validated on prospective validation dataset and external validation dataset. The radiomics model with best performance developed on development dataset had a mean area under the receiver operating characteristic curve (AUC) of 0.89. A similar performance in prospective validation was found (mean AUC of 0.89), while a lower performance was found in external validation dataset (mean AUC of 0.63). In addition, the radiomics models performed with automatic segmentation showed all a decrease in performance; mean AUC of 0.75, 0.77 and 0.50 in development dataset, prospective validation dataset and external validation dataset, respectively. Our radiomics models using CMR images can non-invasively distinguish between +/LVH- patients and healthy controls on both development dataset and prospective validation dataset. However, it was not able to distinguish on external validation dataset.