Resting-state fMRI (rs-fMRI) has become an important imaging modality and is commonly used to study intrinsic brain networks. These networks can be obtained by decomposing rs-fMRI data into components, using independent component analysis (ICA). Recently, these ICA components have been used as inputs for neural networks to learn complex relations between the intrinsic networks of the brain and mental disorders or demographic variables. Instead of training a non-linear classifier on these linearly decomposed components, this work asks whether unsupervised representation learning can lead to linearly separable representations for multiple downstream tasks.

We propose to apply non-linear representation learning to voxelwise rs-fMRI data. Learning the non-linear representations is done using two versions of a variational autoencoder (VAE). The first version is a vanilla VAE with 3D residual blocks in both its encoder and decoder. The second version is based on the identifiable VAE and uses a time-dependent prior. The models train to reconstruct the original input data from latent variables it infers. Three predictive models then evaluate the predictive power of the latent variables on an age regression, a sex classification, and a schizophrenia classification task. Each of the predictive models performs predictions for each of the three tasks. The predictive models are a support vector machine (SVM), a k-nearest neighbor (k-NN) model, and a long short-term memory (LSTM) neural network.

We show that our method performs exceptionally well on the age regression and sex classification tasks without any supervision. These results imply that VAEs can model predictive variations in their latent spaces for demographic variables. The models, however, do not do well on the schizophrenia classification task, even when the models are pretrained. Despite the lower performance on the schizophrenia classification task, the overall results are encouraging and pave the way for future work on voxelwise representation learning.

In clinical practice, as a first approximation, the severity of an abnormality on an image is often determined by measuring its volume. Researchers often first segment this abnormality with a neural network trained by voxel-wise labels and thereafter extract the volume. Instead of this indirect two steps approach, we propose to train neural networks directly using the volumes as image-level label and predict the volume directly. Using image-level labels to train automatic abnormality prediction could decrease the labeling burden for clinical experts, which is both expensive and time consuming. In this report, a neural network that consisted of a segmentation part and an appended regression part was compared with the indirect segmentation approach. It was investigated if networks trained with image-level labels have the same performance of image-level prediction as networks trained with voxel-wise labels. The neural networks were trained on a large local dataset to quantify white matter hyperintensity (WMH) burden from brain MRI, and their performance was evaluated on a held-out test set. Furthermore, generalization properties were compared by applying the trained networks on four independent public datasets. The networks trained with image-level labels achieved volume quantification that was slightly better than their counterpart on the held-out test set. The attention maps of these networks showed that the networks were able to focus on the surroundings of the WMH, and hence learned meaningful image features. Nevertheless, the attention maps were not suitable to achieve a compatible segmentation. In terms of generalization towards external datasets, the advantage of weak labels for volume quantification did not hold as there was no significant difference between the performance of the label types. The results suggest that neural networks optimized with image-level labels were able to directly predict WMH volume as well as neural networks trained with voxel-wise labels. Subsequently, we also studied networks that were optimized on both image-level and voxel-wise labels. Those networks reached a lower performance, which suggested that the tasks and their image features learned were not similar enough.

Objective: Recent studies have suggested an association between age-related hearing loss and cognitive decline. Yet, the underlying mechanism explaining this relation remains unknown. In this regard, several studies investigated gray matter (GM) differences in age-related hearing loss but presented inconsistent results regarding the association and regions involved. To our knowledge, a data-driven approach for exploring this association has not been performed. Therefore, we aimed to investigate possible GM differences and regions involved in age-related hearing loss using conventional multivariable linear regression and deep learning. Methods: Within the population-based Rotterdam Study, 2070 participants (mean age: 65.5 years) underwent pure-tone audiometry to quantify hearing thresholds (hearing loss [> 40 dB], n=205; normal-hearing controls [< 20 dB], n=822). Magnetic resonance (MR) imaging was performed to obtain GM volumes of the superior temporal and precentral gyrus, and GM modulated images. Using multivariable linear regression we investigated the associations between age-related hearing loss and GM volume in the superior temporal and precentral gyrus. A convolutional neural network (CNN) was trained to classify hearing loss and normal-hearing controls based on GM modulated images of the whole brain and the region around the superior temporal gyrus. Visualization of relevant features for the classification was performed with gradient-weighted activation mapping (Grad-CAM).Results: We found that age-related hearing loss was significantly associated with smaller GM volumes in the right hemisphere of both the superior temporal gyrus (difference in standardized brain volume per dB increase: -0.006 [95$\%$ CI: -0.010, -0.003]) and precentral gyrus (difference: -0.005 [95$\%$ CI: -0.008, -0.001]). The CNN classification performance ranged between 0.89 and 0.96 area under the receiver-operating characteristic curves. Analysis of relevant features for the classification showed that features were not specific to the superior temporal gyrus or primary auditory cortex, but appeared across the whole brain. Furthermore, we noticed that misclassified subjects were significantly related to age. Conclusion: This study shows that age-related hearing loss is related to both GM volume in the superior temporal and precentral gyrus. Moreover, relevant features for the classification of age-related hearing loss were observed across the whole brain. These results may be explained by a third factor affecting both hearing loss and neurodegeneration. As age likely is the third factor involved, a longitudinal study design or age-matched groups are required in further studies on age-related hearing loss.

The gap between predicted brain age and chronological age could serve as biomarker for early-stage neurodegeneration and as potentially as a risk indicator for dementia. We assess the utility of this age gap as a risk biomarker for incident dementia in a general elderly population. The brain age is estimated from longitudinal brain magnetic resonance imaging (MRI) data using deep learning models. From the population-based Rotterdam Study, 5656 dementia-free and stroke-free participants (mean age 64.67±9.82, 54.73% women) underwent brain MRI at 1.5T, including three-dimensional (3D) T1-weighted sequence, between 2006 and 2015. All participants were followed for incident dementia until 2016. During 6.66±2.46 years of follow-up, 159 subjects developed dementia. The entire dataset was split into control (N=5497) and incident dementia (N=159) groups. We then built a convolutional neural network (CNN) model trained on the control group to predict brain age based on brain MRI. Model prediction performance was measured in mean absolute error MAE=4.45±3.59 years of brain age prediction. Reproducibility of prediction was tested using the intra-class correlation coefficient ICC=0.97 (95% confidence interval CI=0.96-0.98), computed on a subset of 80 subjects. Hereafter, we investigated the gap between model predicted age and chronological age of the incident dementia group data, compared to control group. Logistic regressions and Cox proportional hazards models were used to assess the association of the age gap with incident dementia, adjusted for years of education, ApoE4 allele carriership, GM and intracranial volume. These models showed that the age gap was significantly related to incident dementia (odds ratio OR=1.11 and 95% confidence intervals CI=1.05-1.16; hazard ratio HR=1.11 and 95% CI=1.06-1.15, respectively). Additionally, we computed the attention maps of CNN, which shows the importance of brain regions for age prediction. These were particularly focused on the amygdalae and hippocampi. We show that the gap between predicted and chronological brain age is a biomarker, associated with a risk of dementia development. This suggests that it can potentially be used as a complimentary biomarker for early-stage dementia risk screening.

Patients with 1p/19q co-deleted low grade glioma (LGGs) have better prognosis and react better to certain treatments than patients with intact 1p/19q LGG. Currently, information about the 1p/19q co-deletion status is obtained by means of an invasive procedure called biopsy. As an alternative, non-invasive techniques to extract this information from medical images are being studied. Recent research suggests that local binary patterns (LBPs), a textural image descriptor, are an important feature which can predict the 1p/19q co-deletion from MRI scans. In this project we report the effect of including LBP information in a convolutional neural network (CNN) to predict the 1p/19q co-deletion status in patients suffering from a presumed LGG using pre-operative MRI scans. A combination of convolutional filters was designed and included in the CNN, resulting into local binary convolutional neural networks (LBCNNs). Three LBP descriptors, each of them representing a different textural scale, were studied, as well as the combination of the three. A default CNN without LBPs was also studied. To validate the designed filters and to study more sophisticated LBPs images like the uniform LBPs, pre-computed LBP images were directly input to the CNN. An in-house multi-institution MRI dataset consisting of 284 patients who had undergone a biopsy or resection before the treatment, and with available pre-operative T1-weighted post contrast and T2-weighted scans was used to train the different network architectures. An independent dataset consisting of 129 patients was used to validate the results. The performance of the LBCNNs was compared to the performance of the CNN. The performance of the CNN and LBCNNs was similar, reporting an area under the receiver operating characteristic curve (AUC) ranging from 0.816 to 0.872 for the different architectures. These findings suggest that the CNN can extract information relative to LBPs by itself. In addition, pre-computed uniform LBPs report similar metrics (AUC: 0.819), suggesting that they do not add new information.

Cardiac magnetic resonance (CMR) is used extensively in the diagnosis and management of cardiovascular disease. Deep learning methods have proven to deliver segmentation results comparable to human experts in CMR imaging, however, no successful attempts have been made at fully automated diagnosis. This has been contributed to a lack of sufficiently large datasets required for end-to-end learning of diagnoses. Here we propose to exploit the excellent results obtained in segmentation by jointly training with diagnosis in a multitask learning setting. We hypothesize that segmentation has a regularizing effect on learning and promotes learning of features relevant for diagnosis. Results show a three-fold reduction of the classification error to 0.12 compared to a baseline without segmentation, both results are obtained by training on just 75 cases in a dataset (ACDC) that is equally distributed over 5 classes.

Prostate cancer is a disease with very high prevalence and mortality in the western world. An early accurate diagnosis can increase treatment efficiency. Current diagnosing techniques consist in systematic biopsy sampling. Radiomics can infer tumor's phenotypic differentiations from medical images, providing an accurate guide for biopsy sampling and making personalized treatment plans possible. Radiomics are various features that are extracted from medical images. Subsequently they are applied to train machine learning models that distinguish between healthy or cancerous tissue.
In this thesis a software routine that extracts the most commonly reported MRI prostate cancer radiomic features was built. Then, several classification methods were tried. Results were validated on T2 MRI patient images with confirmed histopathology from two different clinics.

The brain’s white matter mainly consists of (myelinated) axons that connect different parts of the brain. Diffusion-weighted MRI (DW-MRI) is a technique that is particularly suited to image this white matter. The MRI signal in DW-MRI is sensitized to diffusion of water in the microstructure by introducing strong bipolar gradients in the MRI pulse sequence. By measuring the diffusion in different directions, the local diffusion profile of water molecules is obtained which reflects microstructural characteristics of the white matter.
The focus of this thesis is on the analysis of conventional DW-MRI data acquired in the context of the Rotterdam Scan Study. This is a prospective population-based cohort study with more than 10.000 participants to investigate causes of neurological disease in elderly people. Conventional DW-MRI is defined as diffusion data acquired with a single diffusion-weighting factor and a small number of diffusion-sensitizing gradient orientations. The objectives of this thesis are (1) to enhance our insight in the relation between tissue structure and the DW-MRI signal from conventional DW-MRI sequences, and (2) to develop methods to quantify diffusion properties in the brain as accurately and precisely as possible based on conventional DW-MRI data.
To gain insight into the relation between tissue structure and the DW-MRI signal, simulated DW-MRI signals based on Monte Carlo simulations of spins between randomly packed cylinders are compared to experimentally acquired data from a hardware phantom. The hardware phantom consists of solid fibers and acts as a model for the extra-axonal diffusion. The simulated DW-MRI signal is in good agreement with the experimentally acquired data. Furthermore, simulations show that the DW-MRI signal from spins between randomly packed cylinders is relatively independent of the cylinder diameter for b-values up to 1500 s/mm². For b-values higher than 1500 s/mm², substrates with a smaller cylinder diameter yield a larger attenuation of the diffusion-weighted signal (chapter 2).
Conventional DW-MRI data is commonly analyzed with a technique known as diffusion tensor imaging. Here, thewater diffusion profile is modelled by a 3D Gaussian diffusion profile. However, in white matter structures in close proximity to the cerebrospinal fluid (CSF) the use of the single diffusion tensor model is inappropriate. A novel framework is introduced to analyze white matter structures adjacent to the CSF. In this framework a constrained two-compartment diffusion model is fit to the data in which the CSF is explicitly modeled with a free water diffusion compartment. The proposed diffusion statistics are shown to be relatively independent of partial volume effects with CSF and are applied to study ageing in the fornix, a small white matter structure bordering the CSF (chapter 3).
A significant part of the white matter constitutes of ‘crossing fibers’, whereby two or more white matter tracts contribute to the DW-MRI signal in a voxel. The single diffusion tensor model cannot adequately describe the data in such voxels. To solve this issue a fiber orientation atlas and a model complexity atlas were used to analyze conventional DW-MRI data with a simple crossing fibers model, namely the ball-and-sticks model. It is shown that the application of a fiber orientation atlas and a model complexity atlas can significantly improve the reproducibility and sensitivity of diffusion statistics in a voxel-based analysis (chapter 4).
Finally, a framework is proposed that aims to specifically improve the analysis of longitudinal DW-MRI data. In this framework the ball-and-sticks model is fit simultaneously to multiple scans of the same subject. The orientations of the sticks are constrained to be the same over different scans, while all other parameters are estimated separately for each scan. The use of this framework is shown to increase the precision of estimated ball-and-sticks model parameters in longitudinal DW-MRI studies (chapter 5).
In conclusion, this thesis describes frameworks to enhance the accuracy or precision of estimated diffusion properties of the white matter by applying sophisticated diffusion models to conventional DW-MRI data. We anticipate that many diffusion MRI studies may benefit from the work described in this thesis.