Convolutional Neural Networks (CNNs) have emerged primarily from research focusing on image classification tasks and as a result, most of the well-motivated design choices found in literature are relevant to computer vision applications. CNNs' application on Imaging Mass Spectrometry (IMS) data is quite recent and involves new challenges, such as taking into account their unique structure (e.g. both spatial and spectral dimensions).

In this thesis, we suggest a 1-D CNN architecture that extracts local features along the spectral dimension. The aim is to investigate if CNNs improve the classification accuracy compared to other classic Machine Learning (ML) methods such as linear models. Furthermore, we explore Neural Networks (NNs) that employ the novel Sharpened Cosine Similarity (SCS) as a feature extraction method, opposed to convolution. We call those networks SCS-NN in correspondence to the Convolutional-NN (CNN). To evaluate these methods, we implement our pipeline for various IMS datasets, with different characteristics and classification tasks, using several performance metrics such as balanced accuracy and F1 score.

Moreover, we provide a detailed description of the methodology pipeline used for the CNN architecture design. The suggested methodology is the Tree-structured Parzen Estimator (TPE) algorithm, a Bayesian optimization technique for automated architecture selection. By implementing TPE, we manage to explore and exploit efficiently a complex and large hyperparameter configuration space and automatically select optimal hyperparameters (such as number of convolutional layers, kernel size, strides, learning rates etc.). This automated approach reduces time consumption, errors, and the need for specialized knowledge in biology and biochemistry that would be associated with manual design. In addition to developing a pipeline for designing, training and evaluating a CNN for IMS data classification, we also apply a model agnostic interpretation methodology based on SHapley Additive exPlanations (SHAP) and provide SHAP score maps that visualize the importance of features in the spatial dimension of the IMS datacube.

In this thesis, we present and analyse the automated selection of 1-D CNN architectures for IMS data classification based on the TPE algorithm. Furthermore, we investigate a novel alternative to convolution, SCS, and evaluate its strengths and weaknesses in IMS data classification. The experimental results show that the TPE-generated CNN architectures outperform all the other applied classifiers. Finally, our interpretation of the CNN models reveals that accuracy performance alone might not be a sufficient criterion to trust the model's output.

Ground based telescope imaging suffers from interference from the earth’s atmosphere. Fluctuations in the refractive index of the air delay incoming light randomly, resulting in blurred images. A deconvolution from wavefront sensing system is an adaptive optics system that measures the modes in which the light is corrupted (i.e. the wavefront) and corrects it using a process called deconvolution. The wavefront is measured using a wavefront sensor, which consists of an array of microlenses combined with an imaging sensor. Each microlens casts an image of the object unto the imaging sensor, resulting in a collection of images that are differently aberrated depending on their location on the sensor. Conventionally, the wavefront is calculated by measuring the shifts of each microlens image and integrating these shifts over the aperture. This method, however, discards information about the higher order deformations of the microlens images.

In this thesis, a novel method of wavefront reconstruction has been developed which makes use of artificial neural networks in order to extract this higher order information. In order to do this, the images produced by the microlenses are normalized, which is done using a modified version of the blind deconvolution algorithm called TIP. After the normalization, the microlens images are reduced to what they would look like if a point source was observed, instead of the object. With the influence of the object removed, an artificial neural network is used for the estimation of the wavefront.

By using this method, the wavefront can be reconstructed with twice the turbulence strength compared to what is possible with conventional methods. Combining this method with an image deconvolution step results in a real-time image correction system that works up to 10Hz on the tested system, consisting of a desktop PC with an Intel Xeon E5-2630 DUAL CPU and a NVIDIA GeForce GTX 970 GPU.

Correction techniques, such as tangential iterative projections (TIP), were developed to reconstruct the image for the whole field of view. However, due to the three dimensional nature of biological tissues, induced aberrations are different throughout the field of view. This Master of Science thesis shows the development of four algorithms. These algorithms apply TIP to deconvolve images locally. Thereafter, the local results are combined in order to restore images from anisoplanatic aberrations. The difference between the the algorithms developed during this research is the complexity in the spatial domain and in the Fourier domain. It was found that adaptive limited support in the spatial domain increases the im- age quality of the estimated object. A novel approach for multi-frame deconvolution in the Fourier domain has shown to be a promising modification. The use of weighted multi-frame deconvolution in the Fourier domain has lead to improved image quality.

Visual surveillance technologies are increasingly being used to monitor public spaces. These technologies process the recordings of surveillance cameras. Such recordings contain depictions of human actions such as "running", "waving", and "aggression". In the field of computer vision, automated detection of human actions in videos is known as action detection. Recently, deep learning models have been proposed for the task of action detection. Deep learning models for this task can be grouped into single-frame models and multi-frame models. Single-frame models detect actions using individual frames of videos whereas multi-frame models detect actions using sequences of frames.
This thesis proposes to use multi-frame models as compared to single-frame models for action detection in surveillance videos. To compare multi-frame and single-frame models, we implement the ACT-detector. The ACT-detector is a deep learning model that takes as input a sequence of K frames and outputs tubelets (labeled sequences of bounding boxes). We train and evaluate ACT for various values of K on the VIRAT dataset. In our comparison, K=1 serves as the single-frame model and K>1 as the multi-frame models. When compared qualitatively, we find that multi-frame models have less missed detections. When compared quantitatively, we find that multi-frame models outperform single-frame models in performance measures such as classification accuracy, MABO, frame-mAP, and video-mAP.
To assess whether the improvements of multi-frame models yield purely from the increased number of frames, or also from the temporal order encoded by those frames, we experiment with training multi-frame models on unordered sequences of frames, i.e., sequences for which the frames are shuffled in time. When compared qualitatively, we find that multi-frame models have less precise localization when trained on unordered sequences. When compared quantitatively, we find that multi-frame models perform worse when trained on unordered sequences, indicating that multi-frame models learn temporal dynamics of actions. Nevertheless, even when trained on unordered sequences, multi-frame models outperform single-frame models for action detection in surveillance videos.

This project aims to improve the accuracy of the eye tracking system, which consists of two cameras and two infrared LED light sources. This highly non-invasive technology, which is the feature-based video-oculographic eye tracking system, determines the position of the eye by monitoring the eye features such as the pupil center and glints. The accuracy of estimating the eye position and orientation is critical in the proton clinic environment, and is to be required higher than those in commercially available eye-tracking system.

Accurate and safe proton beam delivery is one of the most crucial tasks during Proton Therapy (PT) of ocular melanoma. The eye movement and gaze angle tracking system should be able to monitor in real time the eye position and orientation (6 degree of freedom) to exactly localize the tumor location inside the eye with respect to the proton beam. The system should also immediately switch the beam off if the tumor goes out of the irradiated area to protect vital organs and keep the non affected cells healthy. The non-invasive eye-tracking system will replace the painful surgical procedure of implantation of radio-opaque tantalum clips on the eye. In order to estimate accurately enough the eye gaze angle and torsion, a stereo imaging system consisting of two high-resolution imaging cameras and two infra-red beacons can be used. The six coordinates of the eye are extracted by image analysis of the acquired stereo camera images using the beacons reflections (glints) and location of the eye pupil. The accuracy of the method can be affected by motion artifacts and difficulty of pupil segmentation in some eyes. Further, the method is inclusive to cyclo-torsion (rotation of the eye about its optical axis). The eye-tracking system can thus benefit from motion artifacts suppression and implementation of analysis of the eye surface features, e.g. iris pattern. Another addition is to use steerable mounts and mirrors to improve measurements and, thus, eye tracking accuracy. The goal of this thesis is to build a prototype stereo eye-tracking system, write a tracking code and investigate the viability of this method for further improving eye tracking accuracy.