Imaging Mass Spectrometry (IMS) data calls for computational methods to perform factorizations efficiently that facilitate interpretable components. Despite the widespread use of Nonnegative Matrix Factorization (NMF) for IMS-related dimensionality reduction, conventional NMF algorithms are not up to par to scale for extremely large datasets. To remedy this problem, this thesis explores Randomized Nonnegative Matrix Factorization as an efficient alternative to compute IMS data on whole-body mouse pup tissue data.

In this thesis, the author demonstrates how Randomized NMF inherits its two-stage framework for computation from Randomized SVD. The main bottleneck between Randomized NMF replicating the success of its predecessor is the lower-dimensional basis used to compress the data matrix. Although the basis has been conventionally effective for Randomized SVD with provable guarantees, the lack of entry-wise nonnegativity constraint requires existing methods to compromise the projection quality and interpretability and rely on the relaxation of nonnegativity constraints within the update rules. Therefore, this work motivates and explores the use of alternative bases that directly satisfy entry-wise nonnegativity.

The global internet protocol (IP) traffic is rising exponentially due to the increased number of bandwidth-intensive services like video-on-demand and cloud computing. To meet the demands of growing traffic, the data rates of fiber optic communication systems (FOCSs) need to be increased. In this regard, digital signal processing (DSP), which already plays a powerful role in the modern FOCSs, is being explored. Increasing the net data rate of FOCSs requires compensation for nonlinear impairments that can arise from the Kerr effect during the propagation of signal through fiber as well as from the non-ideal responses of transceiver hardware components. Furthermore, cutbacks in net data rates due to training overheads, i.e., the non-information carrying part of transmitted data consumed by DSP algorithms; need to be reduced. In this dissertation, we propose novel DSP approaches to address these problems of great practical interest.

The Kerr nonlinear effects add phase shifts to the signal, which are dependent on its instantaneous power. These phase distortions occur simultaneously with the dispersion effect of the fiber, which spreads signal pulses in time. The interplay is complicated and makes compensation of distortions challenging. The nonlinear Fourier transform(NFT), which offers immunity from the distortions of the Kerr effect, received great interest in recent years. The lossless nonlinear Schrödinger equation (NLSE), which models signal propagation in an ideal lossless optical fiber, belongs to a class of nonlinear partial differential equations known as integrable equations. These integrable equations can be solved exactly by NFT. Similar to the Fourier transform that translates a linear dispersive propagation in the time domain into phase delays in the signal spectrum, the NFT translates the nonlinear evolution of the signal governed by the lossless NLSE into trivial multiplications in the nonlinear Fourier spectrum of the signal. The NFT is exact for lossless fiber channels. In the presence of loss, the integrability property is violated. In lossy propagation, signal power reduces as it propagates. This in turn reduces the strength of the nonlinear effects along the length of the fiber. As practical fibers are lossy, the path-average approximation is often used to apply NFT on lossy fiber channels. In this approximation, the variation in the Kerr nonlinear effects due to the reduction in signal power is accounted as the variations in the Kerr-nonlinearity parameter of the fiber. Then, by approximating the varying Kerr-nonlinearity parameter with its average value over a span, a lossless fiber model is obtained. This approximation has errors associated with it which sacrifices the performance. We developed a NFT-based transmission system that is exact even in the presence of fiber loss. The proposed design eliminates errors due to loss, thus improving performance over the design that uses path-average approximation…

In medical research, Imaging Mass Spectrometry (IMS) is a powerful tool that facilitates the spatial mapping of biomolecules in tissue samples, contributing to the identification of disease biomarkers and the analysis of drug effects. While offering valuable insights, IMS data is often large and complex, presenting challenges in data interpretation. Traditional techniques such as principal component analysis (PCA) and non-negative matrix factorization (NMF) have been employed to address these issues, but they fall short due to their inability to fully capture the complex, mixed nature of IMS data. Deep non-negative matrix factorization (deep NMF) offers a promising solution by employing a multi-layer architecture, allowing for dimensionality reduction while improving interpretability.

Deep NMF can decompose IMS data into multiple structures, helping to uncover complex biochemical interactions. This thesis explores the application of deep NMF to IMS data, deriving update algorithms for both the Frobenius norm and generalized Kullback-Leibler (gKL) divergence, and testing these methods on MALDI-TOF IMS data. Results show that the deep multiplicative update rule (MUR) outperforms non-deep methods in terms of interpretability and surpasses multi-layer structures in both reconstruction error and interpretability. Computational challenges remain in tuning algorithm parameters and achieving convergence in deep Alternating Direction Method of Multipliers (ADMM) NMF.

This report investigates the use of Generative Adversarial Nets (GANs) specifically for over-
sampling Imaging Mass Spectrometry spectra. IMS is a technique used to measure the spatial
distribution of molecules, which is valuable in fields like oncology and biomarker discovery.
GANs, on the other hand, are a class of machine learning frameworks where two neural net-
works, the generator, and the discriminator, are trained simultaneously through adversarial
processes. The generator creates synthetic data, while the discriminator tries to distinguish
between real and synthetic data.
GANs-based oversampling aims to increase classifier performance by adding data to classes
that are underrepresented in the original data. Synthetic oversampling is especially relevant
in IMS data as the measuring technique is destructive, making acquiring more real samples
impossible. GANs have been shown to outperform other oversampling techniques such as
SMOTE on various datasets. Applying GANs directly to the dataset proved unsuccessful in
this oversampling task.
Different possible causes of the limited performance of the GANs are studied leading to
improved experiment results using spectra reduced in dimension and the Wasserstein GANs
with gradient penalty. Even though with these changes to the experiment the GANs appear
to generate more realistic data, using this data for oversampling does not increase overall
classifier performance. Rather, it steers the classifier to overfitting towards the minority
classes.
This report demonstrates that applying the designed GANs for oversampling minority classes
on this dataset does increase classifier performance. However, it is shown that GANs can be
trained on IMS data and that GANs might be of use for applications with IMS data besides
oversampling.

The main aim of the project was to develop a novel algorithm that enable two-dimensional feature detection in an extremely sparse environment of Ion Mobility Imaging Mass Spectrometry (IM-IMS) measurements. For this, 2D Wavelet Transform Maxima is proposed. This led to construction of wavelet chains (ridges) in the wavelet transform space. Current thresholding methods for these chains were based on global thresholding of wavelet coefficients. Therefore, a thresholding method known as effective length-based thresholding is introduced. Here, length-based thresholding is modified to take the local nature of noise into account. The design parameters of the algorithm were evaluated using a synthetic IM-IMS data sample, and the performance of the algorithm was quantified using F-Score. The parameters that were studied were (i) width of the wavelet function (along mobility dimension), (ii) effective length, (iii) penalty factor used for calculation of local noise. Fair performance was obtained by choosing effective length equal to 4 and width equal to 1% of mobility dimension. There were a significant number of false positives still being detected along dominant mobilograms . The performance of the algorithm was compared with an existing feature detection algorithm on a real-world IM-IMS data sample and was found to be better in terms of number of features being detected.

Convolutional Neural Networks (CNNs) have emerged primarily from research focusing on image classification tasks and as a result, most of the well-motivated design choices found in literature are relevant to computer vision applications. CNNs' application on Imaging Mass Spectrometry (IMS) data is quite recent and involves new challenges, such as taking into account their unique structure (e.g. both spatial and spectral dimensions).

In this thesis, we suggest a 1-D CNN architecture that extracts local features along the spectral dimension. The aim is to investigate if CNNs improve the classification accuracy compared to other classic Machine Learning (ML) methods such as linear models. Furthermore, we explore Neural Networks (NNs) that employ the novel Sharpened Cosine Similarity (SCS) as a feature extraction method, opposed to convolution. We call those networks SCS-NN in correspondence to the Convolutional-NN (CNN). To evaluate these methods, we implement our pipeline for various IMS datasets, with different characteristics and classification tasks, using several performance metrics such as balanced accuracy and F1 score.

Moreover, we provide a detailed description of the methodology pipeline used for the CNN architecture design. The suggested methodology is the Tree-structured Parzen Estimator (TPE) algorithm, a Bayesian optimization technique for automated architecture selection. By implementing TPE, we manage to explore and exploit efficiently a complex and large hyperparameter configuration space and automatically select optimal hyperparameters (such as number of convolutional layers, kernel size, strides, learning rates etc.). This automated approach reduces time consumption, errors, and the need for specialized knowledge in biology and biochemistry that would be associated with manual design. In addition to developing a pipeline for designing, training and evaluating a CNN for IMS data classification, we also apply a model agnostic interpretation methodology based on SHapley Additive exPlanations (SHAP) and provide SHAP score maps that visualize the importance of features in the spatial dimension of the IMS datacube.

In this thesis, we present and analyse the automated selection of 1-D CNN architectures for IMS data classification based on the TPE algorithm. Furthermore, we investigate a novel alternative to convolution, SCS, and evaluate its strengths and weaknesses in IMS data classification. The experimental results show that the TPE-generated CNN architectures outperform all the other applied classifiers. Finally, our interpretation of the CNN models reveals that accuracy performance alone might not be a sufficient criterion to trust the model's output.

Macro X-ray fluorescence (MA-XRF) is a recently developed technology allowing to obtain elemental information from cultural heritage objects. This information can, for example, be used to identify pigments used in a painting. Yet, the extended period of time it takes to scan an object is a major issue within MA-XRf. For instance, it took about 60 days to scan the Ghent Altarpiece. The long scanning time is a consequence of the necessary dwell time per pixel to create a robustly interpretable spectrum: the higher the dwell time, the higher the signalto-noise ratio (SNR), hence, the easier to detect elements. This thesis explores a possible solution for this problem using a denoising algorithm that increases the signal-to-noise ratio post-acquisition by exploiting the similarity between neighbouring pixels and spectra. To this end, a customized method of wavelet filter bank denoising is proposed. Current thresholding methods used in wavelet filter bank denoising are not suitable for filtering MA-XRF data, therefore, a novel thresholding method is introduced. Here, the widely used universal thresholding method is used as a basis, for which the formula for calculating the standard deviation of the detail coefficients of a channel is altered. Several design parameters of wavelet filter bank denoising were evaluated using a synthetic dataset, for which the performance quality indicators root mean square error (RMSE), mean absolute error (MAE) and SNR were determined. The parameters for which we optimized were the mother wavelet, the number of decomposition levels, and the number of neighbouring channels used for determining the standard deviation σ for thresholding. Good performance was obtained with the haar, db2, and coif1 wavelets, all at 3 levels of decomposition. A suitable number of neighbouring channels depended on the decomposition level and was determined to be 3 (on each side of the channel). Herewith, the signal-to-noise ratio was improved for both the average pixel spectra and the sum spectrum. The filtered synthetic dataset simulated to have a dwell time of 0.5 seconds had a SNR approximately equal to the raw synthetic dataset simulated to have a dwell time of 0.75 seconds. Hence, the algorithm succeeded in lowering the necessary dwell time. A case study of a daguerreotype was used to test the proposed denoising algorithm.

Imaging Mass Spectrometry (IMS) is a powerful technique capable of extracting unlabeled spatial and chemical information from a biological tissue sample. Ever-increasing technological advancements have resulted in rapid growth of IMS data set sizes, scaling quadratically with the increasingly refined spatial resolution of its ion images. Dimensionality reduction techniques aim to make large data sets practically approachable by reducing the number of dimensions in the data while retaining as much information as possible. Nonlinear Dimensionality Reduction (NLDR) methods attempt to uncover an underlying nonlinear manifold or structure in the data by constructing a Low-Dimensional (LD) feature space with variables that are nonlinear combinations of the original features (i.e. mass-to-charge ratio (m/z) bins in IMS measurements). The t-Distributed Stochastic Neighbor Embedding (t-SNE) has been a common approach in NLDR methods, using force-directed graphs. Uniform Manifold Approximation and Projection (UMAP) works in a similar manner, but leans on a more versatile mathematical foundation in topology, is generally faster, and the method tends to scale better than t-SNE.

In this thesis, we introduce UMAPLUS, an extension of UMAP that takes not only spectral, but also spatial information into account. Our experiments show that UMAPLUS is capable of extracting the structure of a synthetic IMS data set better than UMAP. Besides UMAP’s standard spectral and random initializations, we introduce a new spatial initialization that provides more intuitive insight into the LD embedding relative to the spatial image domain. Standard UMAP entails several parameters that influence consistency, reliability, and quality of its LD embedding. Thus far, the influence of these parameters on IMS data sets has been barely investigated, and previous studies have consistently applied the default settings of UMAP. In this thesis, a Data-Driven UMAP (DD-UMAP) is constructed, which includes optimization of UMAP parameters in a data-driven manner. Several distance metrics are investigated, with cosine similarity providing the most robust results. A naive 1-D optimization procedure is compared to a multivariate Bayesian optimization approach, capable of optimizing multiple parameters simultaneously. Using an evaluation function partly based on the cost function of UMAP and the addition of spatial information, DD-UMAP is able to estimate and utilize an optimized input parameter set for UMAP in an unsupervised manner and on unlabeled IMS data sets. This is demonstrated on generated synthetic data, and two real world IMS data sets; a full mouse pup and a murine kidney.

Recently, many advancements have been made in accelerated MRI reconstruction with the use of neural networks. Such deep learning methods learn a suitable MRI prior distribution from large sets of training data. For MRI images acquired with an uncommon scanning sequence, large datasets required for training are not available. Additionally, deep learning methods do not generalize well to unseen data. Therefore, in this research, a framework is proposed for accelerated MRI reconstruction trained with simulated data. The framework uses a Recurrent Inference Machine (RIM). The RIM is a deep learning framework that learns an iterative inference process. The RIM framework has been chosen as it is designed to learn the inversion process itself rather than the image statistics. Therefore, RIMs have a low tendency to overfit, and a high capacity to generalize to unseen data. The framework is evaluated by reconstructing undersampled data of in-vivo brain MRI images and comparing them with zero-filled reconstructions, reconstructions of an identical framework trained with in-vivo data and ESPIRiT reconstructions. The comparison shows that the framework does partly learn the inference process; however, the reconstructions still contain artefacts and the reconstruction of the framework trained with in-vivo data and the ESPIRiT method are of higher quality. For simulated data to replace in-vivo data for the training of the RIM, the simulated data has to be more similar to the in-vivo data.

Modern imaging modalities across many application domains increasingly acquire a large number of very high-dimensional measurements, commonly collecting hundreds to millions of variables per spatial resolution element. That high-dimensional nature can severely challenge traditional (often Euclidean distance based) approaches to noise and dimensionality reduction. Furthermore, statistical analysis of such data is often hampered by the curse of dimensionality, concomitant with large numbers of pixels and channels, by the growing abundance of low signal-to-noise ratio (SNR) measurements, and by the detrimental effects of noise accumulation. It is therefore necessary to find efficient means of reducing high-dimensional (and large data size) measurement sets to a lower-dimensional representation while incurring minimal information loss. Addressing this challenge is essential (a) to enable processing of the massively multivariate measurement sets acquired by several promising imaging technologies today (commonly hundreds of gigabytes to terabytes per experiment), (b) to avoid that computational analysis becomes a bottleneck for the development of new instrumental capabilities, with hardware setups theoretically capable of yielding terabyte to petabyte imaging but currently considered impractical, and (c) to enable archiving massively multivariate measurement sets, often required to be stored for several years and containing information suitable for additional science projects. This thesis focuses on this challenge specifically. First, it explores structured and regularized matrix decomposition methods based on the l₁-norm, e.g. building on principal component pursuit (PCP), to address this challenge. Second, it delivers custom implementations of these methods. Third, it develops and implements an application-driven framework for automatic setting of hyperparameters. Finally, it applies and compares these methods using several spectral imaging case studies spanning both very small scales, in molecular imaging of organic tissue, as well as very large scales, in the spectroscopic imaging of Outer space.

Retrieving actionable information from large datasets is increasingly computationally expensive due to the current trend of ever-increasing dataset sizes. Reducing dataset sizes with dimensionality reduction techniques is often necessary for statistical analysis techniques, such as classification, to be computationally feasible. Most dimensionality reduction methods do not require any additional information to accomplish their task. However, datasets used for classification, for example, are accompanied by a set of class-labels as well. This extra information can improve dimensionality reduction techniques by explicitly preserving features that explain differences between classes. A field where high-dimensional and large datasets are standard is Imaging Mass Spectrometry (IMS), a technique that simultaneously records the abundance and spatial location of molecules throughout biological tissue samples. Classification has been applied to IMS datasets for a wide range of scenarios, including the diagnosis of disease, distinguishing between tumour types for personalized treatment, and identifying biomarkers. A recently introduced dimensionality reduction method called Soft Discriminant Map (SDM), designed to incorporate class information and prevent overfitting when used on high-dimensional datasets, is a promising candidate to reduce the size and dimensionality of IMS datasets. However, SDM currently requires manual setting of a free parameter β that influences class separation in the newly constructed feature-space. This thesis explores the use of SDM on IMS datasets in classification use cases and proposes a framework to set β in a data-driven way: Data-Driven Soft Discriminant Map (DD-SDM). Furthermore, the sensitivity of the classification performance to changes in β is examined. DD-SDM is compared to similar state-of-the-art dimensionality reduction methods in terms of classification performance. The performed experiments show that DD-SDM successfully finds a value for β where the classification performance is on par with, or in some scenarios better than, state-of-the-art dimensionality reduction methods while using fewer features. Setting β either too low or too high results in a suboptimal feature space and worsens classification performance. Golden section search, the search strategy used to find the optimal β in DD-SDM, succeeds in finding the optimal β in fewer iterations than more naive methods. With the use of an artificial dataset in combination with a novel evaluation metric, the Peak Conservation Score (PCS), the distinctive ability of DD-SDM to discard features that are common between classes and to actively select for discriminative features is demonstrated. The DD-SDM framework is furthermore applied to real-world IMS measurements of rat brain and mouse kidney tissue.

Nowadays, video surveillance and motion detection system are widely used in various environments. With the relatively low-price cameras and highly automated monitoring system, video and image analysis on road, highway and skies becomes realistic. The key process in the analysis is to separate the useful information such as moving foreground objects from the original video sequence where Robust Principal Component Analysis (RPCA) plays an important role in extracting the foreground objects. RPCA have been widely used in data analysis and dimension reduction with applications in image recovery, information clustering and computer vision. But one drawback of RPCA lies in the fact that it does not guarantee the nonnegativity of pixels. It is important to have nonnegative foreground object since negative pixels that are not in the range between 0 and 255 are meaningless and the foreground objects are thus not visible. State-of-the-art methods do not consider the nonnegativity of the foreground object in their algorithms. This thesis focuses on the problem of extracting foreground moving object from background scenes and guarantee the nonnegativity of foreground object. This thesis proposes a method that combines RPCA and Nonnegative Matrix Factorization (NMF). It ensures the pixels that constitute the foreground object is nonnegative by using the basic model of RPCA and nonnegative components that NMF provides. The efficacy of the proposed algorithms is tested on publicly available dataset. Experiment shows in detail how the proposed algorithms achieve in recovering the foreground object with high true positive rate. Together with RPCA algorithm, the performance of recovery is compared and their advantages and disadvantages are discussed.

In the field of biomedical imaging, images often report a combination of biologically induced variation, usually the goal of the imaging process (e.g. outlining an anatomical region or disease pattern), and non-biological variation, such as instrument or acquisition method-induced noise patterns.
Since some medical decisions are made based on imaging, separating the biological signal from noise is of significant importance (e.g. accelerates decision-making, reducing the chance of misdiagnosis).
Some non-biological variations that span a wide range of imaging modalities include e.g. viewport stitching artifacts, slice-to-slice interference, aliasing, and Gibbs-phenomena.
From a signal processing perspective, many of these can be modeled as quasiperiodic patterns.
Thus, removal of quasiperiodic patterns while preserving the underlying medical information is the main focus of this thesis.

Although in modern instruments, many forms of non-biological variation can be attenuated to be invisible to the naked eye, machine learning algorithms which are often used for classification of disease and segmentation of biological samples may be susceptible to even minor variations and noise patterns.
Development of entirely data-driven, unsupervised denoising techniques can potentially increase the effectiveness and reliability of such algorithms.
Furthermore, under certain image transformations, such as different color spaces, the Fourier and wavelet transform, and factorizations, such as principal component analysis and non-negative matrix factorization, as well as combinations of these, non-biological patterns can get amplified and become so prominent that much of the underlying biological information is concealed.
Removing these quasiperiodic patterns using current state-of-the-art algorithms still requires manual parameter tuning and prior expert knowledge, which is an impractical and possibly unnecessary expectation towards healthcare professionals.
The goal of this M.Sc. thesis is to develop an automated, data-driven framework, that is able to reliably identify, quantify, and eliminate quasiperiodic patterns within the images while retaining as much biological information as possible.

In this framework, named Quasiperiodic Image Denoising (QID), two novel algorithms are implemented, both operating in the Fourier domain.
One algorithm is based on robust principal component analysis (QID-RPCA) and the other uses the normalized median of absolute differences (QID-MADN).
The methods used to achieve unsupervised, data-driven denoising are described in detail.
This includes the use of histogram equalization for radial binning, an automated, sparsity-based approach to choosing the optimal aggregation level, and noise component attenuation based on radial frequency patterns.
The methodology is demonstrated through three case studies.
First, a synthetic dataset is used to compare the performance of the novel algorithms to the current state-of-the-art solutions.
Second, the performance is evaluated on two real-world datasets processed using a number of methods, e.g. factorization and different color spaces. One of these datasets is based on a microscopy image of a transversal section of a mouse brain and the other one is based on a microscopy image of a coronal section of a rat kidney.
Finally, a real-world, raw dataset is denoised consisting of a set of high-resolution fluorescent microscopy images of a human kidney.
Results indicate that the novel algorithms have higher denoising performance than previous approaches in the literature with notable improvements achieved for low-frequency corruptions.

The ability to locate specific objects within images is an essential step in various computer vision based engineering applications. Image segmentation is the task of dividing an image into "segments" that are uniform as well as homogeneous with respect to some characteristics, for example grey tone or texture as in Haralick et al. This thesis seeks to perform image segmentation using a Deep Learning (DL) approach in the area of warehouse automation, specifically focusing on an order picking use case of Vanderlande Industries (VI). Generally in literature, DL algorithms for image segmentation are split into two main classes: algorithms for RGB images and algorithms for RGB-D images. RGB stands for the Red, Green, and Blue values of a pixel. RGB-D stands for the Red, Green, Blue, and Depth values of a pixel. The depth value in this case differs from the RGB values in that it does not give a value for a colour intensity, but rather it gives a value for physical distance between the camera and the object it is capturing. The challenge addressed by this thesis focuses on whether the introduction of depth data results in a substantially better performance than using RGB-only images, based on a data-set provided by VI. Also, this thesis looks into the maximum allowed deviation along the X-axis in the registration of the depth data to the RGB images. Two networks from literature were investigated and implemented in MATLAB for this purpose: the SegNet architecture proposed by Badrinarayanan et al. and the FuseNet architecture proposed by Hazirbas et al. Through experiments we have found that, for this use case, the introduction of complementary depth data leads to an improvement over the use of RGB-only images. We also find that, for this use case, the maximum allowed deviation along the X-axis in the registration of the depth data to the RGB images is approximately equal to 1.67 millimetres. The results in this thesis seem to indicate that investing in acquiring an additional depth band does have a positive effect on the accuracy of image segmentation for order picking in warehouse automation.

Imaging Mass Spectrometry (IMS) is a spectral imaging technique, which enables detection of the spatial distribution of molecules by collecting a mass spectrum for every pixel across a tissue sample. As such, IMS enables the detection of disease-introduced anomalies in tissue samples as well as the gaining of deeper insight on a molecular level into biological processes. The dimensionality of IMS data is high, considering that every bin (or ion) along a mass spectrum represents a separate image and the number of pixels per image is relatively high. Manual analysis of the data suffers from this high dimensionality as visualization becomes increasingly difficult. Furthermore, analysis of such large datasets becomes problematic or infeasible for computational techniques both in time and computational resources. Moreover, the dimensionality of current IMS measurements hampers new applications capturing even more data. Linear dimensionality reduction methods, such as Principal Component Analysis (PCA) and Nonnegative Matrix Factorization (NMF), seek to reduce these datasets to a set of (principal) components. These components span an underlying feature subspace within the original measurement space. Rank estimation determines the quantity of such components, estimating the number needed to represent the original dataset in a lower-dimensional space while incurring minimal information loss. In the context of IMS, this task is typically performed without the use of domain-specific knowledge. Intensity-aware rank estimation seeks to utilize domain knowledge - in the form of an ion intensity threshold - to help estimate the rank. This threshold emerges naturally from IMS, due to prior knowledge on instrument and ionization process inaccuracies in the low ion intensity region. The ion intensity threshold defines a lower bound for which variations in measurements are reliable. Establishing an intensity-aware version of rank estimation requires the threshold, defined in the original measurement space, to be linked to the abstract feature subspace, defined by NMF or PCA, where the rank estimation takes place. This connection is nontrivial to make and is, therefore, a central topic of this thesis. Furthermore, intensity-aware rank estimation requires the abstract subspace to represent the majority of the information above the threshold in the first set of components, which is not guaranteed in pure NMF and PCA formulations. In this thesis, we demonstrate threshold-aware rank estimation and residual-fraction rank estimation which make rank estimation for PCA intensity-aware. Threshold-aware rank estimation applies a histogram transformation to the intensities in the original measurement space to emphasize threshold-exceeding intensities. Consecutively, we estimate the rank based on the percentage of explained variance. Residual-fraction rank estimation uses untransformed measurements but instead estimates rank based on the ratio of the above- and below-threshold residuals. We demonstrate that both rank estimations are able to find the correct rank in a synthetic dataset. With threshold-aware rank estimation applied to an IMS dataset, we show that the transformation before application of PCA leads to a lower overall estimate of rank based on a percentage of the explained variance. With residual-fraction rank estimation applied to an IMS dataset, we show that we can obtain rank estimates based on the structure of dataset close to cross-validation rank estimates for the same dataset.

Hyperspectral imaging (HSI) is a promising imaging modality in medical applications, especially for non-invasive and non-contact disease diagnosis and image-guided surgery. Encoding both spatial and spectral information, it can detect subtle changes in the biochemical and morphological properties of a tissue, revealing the early progression of a pathological condition like cancer. Previous medical hyperspectral image analysis approaches depended on handcrafted features or feature extraction principle, requiring considerable domain expertise. To overcome this, automatic feature learning approaches like convolutional neural networks (CNN), previously used in tasks like classification, detection and segmentation in medical images were applied to hyperspectral data, although in a limited number of research studies. This thesis was proposed to review the state-of-the-art in medical hyperspectral image analysis, identify the limitations in current methods, and present a proof-of-concept for using limited hyperspectral image data in CNN-driven tissue characterization.



The goal of this thesis is to characterize, using CNNs, ex vivo head and neck (tongue) tissue of patients affected by tumors. While previous work in this field implemented patch-based classification of tissue, in this thesis, a pixel-wise classification approach was proposed to obtain a smooth and continuous segmentation of hyperspectral images. To this end, two types of CNN models were trained from scratch using limited labelled training data, one to automatically learn the spectral features present in the hyperspectral data and the other to learn the combined spectral-spatial features from the hyperspectral data. Performance of four different trained models was evaluated by using a leave-one-out testing scheme, with the spectral-spatial learning approach with larger input spatial dimensions outperforming the other considered approaches.

Imaging mass spectrometry (IMS) is a multiplexed chemical imaging technique that enables the spatially targeted molecular mapping of biological samples at cellular resolutions. Within a single experiment, IMS can measure the spatial distribution and relative concentration of thousands of distinct molecular species across the surface of a tissue sample. The large size and high-dimensionality of IMS datasets, which can consist of hundreds of thousands of pixels and hundreds to thousands of molecular ions tracked per pixel, have made computational approaches necessary for effective analysis. This thesis focuses primarily on biomarker discovery in IMS data using supervised machine learning algorithms. Biomarker discovery is the identification of molecular markers that enable the recognition of a specific biological state, for example recognizing diseased tissue from healthy tissue. Biomarkers are increasingly used in biology and medicine for diagnostic and prognostic purposes, as well as for driving the development of new drugs and therapies. Traditionally, the focus has been on maximizing the predictive performance of supervised machine learning models, without necessarily examining the models' internal decision-making processes. Yet, in order to generate insight into the underlying chemical mechanism of disease or drug action, we must go beyond the scope of just prediction and learn how these empirically trained models make their decisions and who are the primary chemical drivers of this prediction process. Machine learning model interpretability is the ability to explain a model's predictions, and can practically be translated into the ability to explicitly report the relative predictive importance of each of the dataset's features. When analyzing IMS data, interpretability is crucial for understanding how the spatial distribution and relative concentration of certain molecular features relate to the labeling of pixels into different physiological classes. The key to our data-driven approach to biomarker discovery in IMS data is to establish (in relation to a specific biomedical recognition task) a means of ranking the molecular features of supervised machine learning models according to their respective predictive importance scores. Ensuring model interpretability and feature ranking in supervised machine learning allows empirical model building to be used as a filtering mechanism to rapidly determine, among thousands of features, those features that exert a large amount of relevance to a specific class determination. With regards to biology, the top-ranking features can help empirically highlight important molecular drivers in the biological process under examination, and can help generate new hypotheses. In terms of translational medicine, such top-ranking features can yield a shortlist of candidate biomarkers worthy of further clinical investigation. Three different classifiers, namely logistic regression, random forests, and support vector machines, are implemented and their performance is compared in terms of accuracy, precision, recall, scale invariance, sensitivity to noise, and computational efficiency. Subsequently, several approaches to explaining these classifiers' predictions are implemented and investigated: model-specific interpretability methods are tied to intrinsically interpretable classifiers, such as generalized linear models and decision trees, whereas model-agnostic interpretability methods can also explain the predictions of black-box models, such as support vector machines with nonlinear kernels or deep neural networks. In addition to three model-specific methods, we present two post-hoc model-agnostic interpretability methods: permutation importance and Shapley importance. Our implementation of Shapley importance, based on Shapley values from cooperative game theory, is novel. Having observed a variability between the rankings of different interpretability methods, we investigate improving the inter-method reliability of feature rankings by decorrelating the features prior to training the classifiers. We also propose a robust ensemble approach to interpretability that aggregates the importance scores attributed to each feature by different model-specific interpretability methods. We demonstrate our methodology on two biomedical case studies: one MALDI-FTICR IMS dataset taken from the coronal section of a rat brain, and one MALDI-TOF IMS dataset taken from the sagittal section of a mouse-pup.

Visual surveillance technologies are increasingly being used to monitor public spaces. These technologies process the recordings of surveillance cameras. Such recordings contain depictions of human actions such as "running", "waving", and "aggression". In the field of computer vision, automated detection of human actions in videos is known as action detection. Recently, deep learning models have been proposed for the task of action detection. Deep learning models for this task can be grouped into single-frame models and multi-frame models. Single-frame models detect actions using individual frames of videos whereas multi-frame models detect actions using sequences of frames.
This thesis proposes to use multi-frame models as compared to single-frame models for action detection in surveillance videos. To compare multi-frame and single-frame models, we implement the ACT-detector. The ACT-detector is a deep learning model that takes as input a sequence of K frames and outputs tubelets (labeled sequences of bounding boxes). We train and evaluate ACT for various values of K on the VIRAT dataset. In our comparison, K=1 serves as the single-frame model and K>1 as the multi-frame models. When compared qualitatively, we find that multi-frame models have less missed detections. When compared quantitatively, we find that multi-frame models outperform single-frame models in performance measures such as classification accuracy, MABO, frame-mAP, and video-mAP.
To assess whether the improvements of multi-frame models yield purely from the increased number of frames, or also from the temporal order encoded by those frames, we experiment with training multi-frame models on unordered sequences of frames, i.e., sequences for which the frames are shuffled in time. When compared qualitatively, we find that multi-frame models have less precise localization when trained on unordered sequences. When compared quantitatively, we find that multi-frame models perform worse when trained on unordered sequences, indicating that multi-frame models learn temporal dynamics of actions. Nevertheless, even when trained on unordered sequences, multi-frame models outperform single-frame models for action detection in surveillance videos.

Multivariate images are built up by measuring multiple features or variables simultaneously while recording a measurement’s location. An example of such images is Imaging Mass Spectrometry (IMS) data. IMS is a technique for recording the mass-over-charge ratio of molecules in (biological) samples while also recording the molecules’ spatial location. High dimensionality in multivariate images (e.g. many recorded features per pixel) often makes direct human interpretation infeasible and computational analysis impractical. For this reason, unsupervised and data-driven factorization is often applied prior to any exploration of the data, with the goal of reducing its dimensionality. However, one of the more promising factorization methods for multivariate images, Maximum Autocorrelation Factorization (MAF), still depends on some input from the user. Unlike most factorization methods, that focus solely on the spectral content of the observations, MAF also utilizes the spatial structure of the input data to generate matrix factors that try to capture both spatial and spectral patterns in the data. The factors of MAF represent the content of a matrix, ranked according to spatial autocorrelation, i.e. how rapidly they vary spatially. The idea for application of MAF in a bioimaging context is that naturally occurring, signals tend to form larger, more uniform areas and therefore change more slowly spatially, compared to noisy, non-biological measurement patterns. Since MAF factors are ordered according to autocorrelation, noisy measurement (with low autocorrelation) tend to get demoted in the order of factors, effectively separating them from biological data components (with positive autocorrelation). The goal of this thesis is to build upon the MAF algorithm and remove the current need for user input, making an extension of MAF. This novel factorization method is named Extended Maximum Autocorrelation Factorization (EMAF). Similarly to MAF, EMAF is invariant to linear transformations, utilizes spatial and spectral information of the dataset to determine its factors, and produces uncorrelated factors at all distances under certain conditions. The EMAF algorithm is fully unsupervised and produces factors ranked according to spatial autocorrelation. Unlike MAF, EMAF does not unnecessarily promote spatial artifacts oriented in one particular direction over other directions. The exact formulation of EMAF, derivations of the mentioned traits, and practical examples of EMAF are found in the following thesis. Preliminary experiments show that EMAF returns factors of improved quality compared to MAF.

Knowledge on adversaries during military missions at sea heavily influences decision making, making identification of unknown vessels an important task. Identification of surrounding vessels based on visual data offers an alternative to AIS information (Automatic Identification System), the current standard in vessel identification, which can be spoofed. One visual approach employs human expertise and manually identifies vessels guided by a ship catalog. In order to minimize or potentially eliminate human error and performance limitations, there is strong interest in developing an automated vessel classification pipeline. One such pipeline is currently being developed at TNO, capable of classifying over 500 separate classes. A crucial part of the classification pipeline is retrieving an accurate contour of a vessel from a digital image.
To address this important challenge, this thesis proposes an advanced deep learning pipeline to automatically segment the vessel image into background (e.g. sky and sea) and the object of interest (a vessel). Deep learning models based on Fully Convolutional Neural Networks (FCNs) have achieved high performance on the task of semantic segmentation. Several networks such as CRF-RNN, PSPNet, DeepLab and Mask R-CNN are employed to determine a baseline performance. We will focus on identifying the cause of poor or failing segmentations and aim to construct a robust network capable of handling these challenges. By sampling disturbances, caused by ship distance and camera noise, augmented data sets are built to tune networks to input from on-site images. Additionally, experiments are done to evaluate the influence of different levels of disturbances.
Previous approaches implementing the CRF-RNN network achieved top 1 and top 5 classification accuracies of 31.1% and 44.0% respectively. Employing the DeepLab network, trained to convergence on artificial noise augmented data, we report top 1 and top 5 accuracy of 68.9% and 88.8% respectively. Additionally, implementing an ensemble of classifiers, performance is increased to 73.0% and 91.7% for top 1 and top 5 accuracy respectively. This best result is comparable to the classification results with human annotated ship silhouettes. The human performance accuracy is 73.4% on top 1, and 91.3% on top 5 classification performance. Finally, we show that training on a collection of different levels of image disturbances results in a network that is robust against increasing disturbance in images, while retaining performance on clean images.