Reconstructing seen images from functional magnetic resonance imaging (fMRI) brain scans has been a growing topic of interest in the field of neuroscience, fostered by innovation in machine learning and AI. This paper investigates the possible presence of personal features allowing the identification of subjects from their reconstructed images. Identifying the extent to which personal information is present is necessary to prevent privacy and data protection breaches. Additionally, personal features may reveal information about how people see the world, furthering work in computer-brain interfacing or helping people with neurological conditions that affect sight. In this paper, a CNN model is presented that allows to identify subjects from their reconstructed image with an average accuracy of 90.4%. An encoder-decoder model was used to produce the reconstructed images from the Generic Object Data set. The accuracy shows that personal features are indeed present in the reconstructed images, raising important ethical and legal considerations when using image reconstruction technology.

The efficacy of endovascular therapy in large vessel occlusion (LVO) of the anterior circulation is dependent to a high degree on the selection of patients who are likely to benefit from this procedure. To this end, functional outcome prediction based on clinical parameters is an active area of research. In the preoperative screening of LVO patients, CT-Angiography (CTA) imaging is commonly acquired. We compare the functional outcome prediction performance of multiple deep learning based classifiers with multiple conventional methods, including the clinically validated MR PREDICTS decision tool. Using a dataset composed of 1929 preprocedural CTA images combined with clinical data, we compare a clinical baseline model with an imaging based pipeline and a combined pipeline. For the imaging model backbone we train various state-of-the-art architectures (Med3D, Vision Transformer, Voxel Transformer). These models are used to predict dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier outcomes are quantified using Area-Under the receiver operating characteristic Curve (AUC). The activation maps of the best performing image based model are further investigated using the GradCAM++ posthoc visualization method. Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. The information extracted from the images does not seem to be complementary to the clinical features. This multimodal technique can however replace radiologically derived biomarkers, as its performance is non-inferior.

This study aims to investigate the impact of various denoising algorithms on the quality of visual stimulus reconstructions based on functional magnetic resonance imaging (fMRI) data. While fMRI provides a valuable, noninvasive method for assessing brain activity, the reliability of this data can be impaired by multiple noise types, including thermal, physiological, and scanner-related noise. Numerous denoising methods have been proposed, such as independent component analysis, confound regression and filtering, and GLM denoise. However, their efficiency, especially in the context of limited task fMRI data, remains largely unexplored. Using the Generic Objects Dataset (GOD), our study explores three primary research subquestions: the effectiveness of different denoising algorithms in improving reconstruction quality; the impact of artificially induced noise on these algorithms and whether combining different denoising algorithms can further enhance image reconstruction quality. The primary contributions of this research include the evaluation of various denoising methods with limited task fMRI data, determining the most effective denoising algorithms given a small dataset size, and analyzing how these algorithms perform in the presence of artificially introduced noise. The results of this investigation showed an improvement in performance of reconstruction models given multiple denoising algorithm, the best performer being kurtosis-based PCA used together with nuisanse regression with constant and linear terms bringing a 6.2% increase in score. The noise ceiling is the worst performer, bringing the score down by 4.4%. Denoising algorithms fail to improve reconstructions poisoned with gaussian noise, however, ICA manages to achieve a minor improvement of the quality of reconstructed images given noise sampled from the uniform distribution.

Gaze estimation is an important area of research used in a wide range of applications. However, existing models trained for gaze estimation often suffer from high computational costs. In this study, frequency domain channel selection techniques were explored to decrease these costs by reducing the size of the input data. The main research objective was to investigate the impact of channel selection on the latency and accuracy of frequency domain gaze estimation. Channel selection methods used in related research were adapted and applied to the domain of gaze estimation. The evaluation was conducted on two popular network architectures used in this field, namely the AlexNet and ResNet-18. Multiple channel selection models were designed for each architecture and compared to a traditional RGB approach with the same network structure. Experimental results showed significant speedups during training, calibration, and inference with marginal accuracy loss. The specific speedups that the top-performing models of both the architectures achieves were 3.3, 4.0, and 1.35 for the AlexNet, and 1.5, 1.7, and 1.35 for the ResNet-18. Accompanying these speedups the AlexNet model error only increased by 0.08 degrees compared to a traditional RGB approach, while the ResNet-18 model lost around 0.44 degrees. All the code used in this research is publicly available on GitHub (https://github.com/tpenning/DLFDFaceGazeEstimation).

Automated negotiation agents can highly benefit from learning their opponent’s preferences. Multiple algorithms have been developed with the two main categories being: heuristic techniques and machine learning techniques. Historically, heuristic techniques have dominated the field, but with the recent development in the field of machine learning, this is no longer true. The main goal of the paper is to compare these two techniques quantitatively using the Pearson correlation of bids. The models that were chosen as the heuristic and machine learning baseline are the Smith and the Perceptron models, respectively. Our results show that the two baselines have similar performance. This leads us to conclude that machine learning algorithms have caught up with their heuristic counterparts. Furthermore, we have also found a statistically significant correlation between the Perceptron model’s accuracy and the seen bid space.

In the field of computer vision, the difficulty of processing the illumination in an arbitrary scene creates complications for a machine's understanding of the scene. Computer vision algorithms commonly use input images acquired by a camera, where the only measured quantity is the amount of light the camera's sensor receives without any additional information about the shadows, occlusion, highlights, light directionality and interreflections, to help with scene processing and understanding. This thesis applies polarization state measurements of the camera's incoming light to implicitly extract information on these illumination phenomena through a polarization-based intrinsic image decomposition. By defining and measuring the polarization state parameters of an image in the form of polarization cosines, the diffuse and specular intrinsic images can be estimated through an iterative local optimization algorithm to give additional information on the scene illumination. The decomposition algorithm is qualitatively evaluated on three image sample sets, which show that the algorithm performs well as long as the measurements are aligned. The diffuse and specular intrinsic images can then be used to improve existing scene post-processing operations without requiring additional illumination information. One of the three image sample sets is used to evaluate the effects of the decomposition on texture modification, glossiness remapping and tonemapping. The results indicate that the three operations can be either simplified or can have their quality improved with the additional information supplied by the diffuse and specular intrinsic images.

3D Gaussian Splatting (3DGS) is a promising 3D reconstruction and novel-view synthesis technique. However, the field of semantic 3D segmentation of 3D Gaussian Splats scenes remains largely unexplored. This paper discusses the challenges of performing 3D segmentation directly on 3D Gaussian Splats, introduces a new dataset facilitating evaluation of 3DGS semantic segmentation and proposes use of PointNet++, initially developed for point cloud segmentation, as a 3DGS segmentation model. As the results show, PointNet++ is also capable of performing 3DGS segmentation with performance close to the performance achieved in point cloud segmentation tasks. When taking into account only the positions, 3D Gaussian splats appear to be more difficult for PointNet++ to process than point clouds sampled from mesh faces, possibly due to their irregularity. However, as shown in the paper, inclusion of size, rotation and opacity of each splat allows PointNet++ to achieve nearly 87% of accuracy, outperforming PointNet++ on point clouds sampled from meshes.

The demand for high-quality 3D visualizations has surged across various professional fields, prompting significant advancements in computer graphics. One such advancement is 3D Gaussian Splatting, a technique evolving from Lee Westover’s splatting concept introduced in 1990. This study investigates the potential of 3D Gaussian Splatting to enhance PointNet classification techniques for 3D objects. Utilizing the Princeton ModelNet10 dataset, I convert 3D models into point clouds, applying 3D Gaussian Splatting to generate optimized Gaussian representations. These representations are used to train PointNet++ models with various feature configurations, including position, scale, rotation, opacity, and spherical harmonics. Our findings reveal that while 3D Gaussian Splatting enables effective 3D object classification, it does not outperform traditional methods that utilize ground truth data directly sampled from object surfaces. Nonetheless, the method demonstrates comparable accuracy, suggesting its viability in scenarios where initial meshes are unavailable. This research highlights the potential and limitations of 3D Gaussian Splatting in advancing 3D object classification.

The severity of hip osteoarthritis is measured a.o. by the minimal distance between the femoral head and the acetabular roof in an X-ray image. However, the whole joint space profile might be a more accurate estimator, since it would include irregularities in the bone surface. These irregular bulges (osteophytes) on the bone surface are one of the signals that a person might have OA. Thus the stage of OA might be better estimated automatically by having this data in the joint space profile instead of just using the minimal joint space. For this joint space profile, the distance between the femoral head and the acetabular roof needs to be calculated. Therefore, the positions of these parts in the hip joint are required to be know. These can be retrieved from e.g. a segmentation mask. One way of calculating the distance in a joint is to use a radial projection. A radial projection is a way of projecting points from a curved space to a plane by projecting lines from a central point along increasing angles. In this paper, we investigate how the joint space profile can be segmented most accurately from a radial projection originating from the center of the femoral head by several comparing noise filtering and edge-finding algorithms. After which is shown that a custom algorithm based on the theory behind edge detection in noisy images works most reliably and accurately. There are still multiple points of improvement for this algorithm. The femoral head can be segmented more accurately than the acetabular roof, the segmentation of the latter could be optimized by detecting the brightest line (peaks) instead of the most sudden change (steepest gradient) in the X-ray image as the edge for the femoral head. The algorithm could be further improved by taking care of local outliers off those edges. In conclusion, this paper compares multiple ways of segmenting the joint space of the hip joint. The best-performing algorithm could in the future be used in an assisting tool for doctors to highlight important irregularities and measurements in the hip joint space.

The severity of hip osteoarthritis is measured a.o. by the minimal distance between the femoral head and the acetabular roof in an X-ray image. However, the whole joint space profile might be a more accurate estimator, since it would include irregularities in the bone surface. These irregular bulges (osteophytes) on the bone surface are one of the signals that a person might have OA. Thus the stage of OA might be better estimated automatically by having this data in the joint space profile instead of just using the minimal joint space. For this joint space profile, the distance between the femoral head and the acetabular roof needs to be calculated. Therefore, the positions of these parts in the hip joint are required to be know. These can be retrieved from e.g. a segmentation mask. One way of calculating the distance in a joint is to use a radial projection. A radial projection is a way of projecting points from a curved space to a plane by projecting lines from a central point along increasing angles. In this paper, we investigate how the joint space profile can be segmented most accurately from a radial projection originating from the center of the femoral head by several comparing noise filtering and edge-finding algorithms. After which is shown that a custom algorithm based on the theory behind edge detection in noisy images works most reliably and accurately. There are still multiple points of improvement for this algorithm. The femoral head can be segmented more accurately than the acetabular roof, the segmentation of the latter could be optimized by detecting the brightest line (peaks) instead of the most sudden change (steepest gradient) in the X-ray image as the edge for the femoral head. The algorithm could be further improved by taking care of local outliers off those edges. In conclusion, this paper compares multiple ways of segmenting the joint space of the hip joint. The best-performing algorithm could in the future be used in an assisting tool for doctors to highlight important irregularities and measurements in the hip joint space.

Hip osteoarthritis is a widespread disease, with medical experts facing difficulties in this illness, due to a lack of standard grading score. Nevertheless, the minimum joint space width remains the most important score for osteoarthritis severity. Manual estimation of this metric is a tedious task, which can greatly benefit from employing an automated tool. While some research has been done in developing such a tool using deep learning methods, a novel and promising approach, most lack annotated data to use for training, which can be hard to obtain. Thus, thus research aims at developing a deep learning approach towards estimating the minimum joint space, while using automatically labels produced with an existing algorithm.

Hip osteoarthritis is a widespread disease, with medical experts facing difficulties in this illness, due to a lack of standard grading score. Nevertheless, the minimum joint space width remains the most important score for osteoarthritis severity. Manual estimation of this metric is a tedious task, which can greatly benefit from employing an automated tool. While some research has been done in developing such a tool using deep learning methods, a novel and promising approach, most lack annotated data to use for training, which can be hard to obtain. Thus, thus research aims at developing a deep learning approach towards estimating the minimum joint space, while using automatically labels produced with an existing algorithm.

An accurate segmentation model for hip compo- nents could improve the diagnosis of Osteoarthritis, a prevalent age-related condition affecting joints. A significant challenge in developing effective and robust segmentation models are the domain differ- ences across various datasets. In this study, we in- vestigate the impact of different data augmentation and preprocessing techniques on the generalizabil- ity of femur segmentation models across datasets. Using two labeled datasets, we evaluate the perfor- mance of a U-Net segmentation model, focusing on the effectiveness of augmentations like image flip- ping, random rotations, blur, contrast, and bright- ness adjustments. Our findings reveal that certain augmentations, particularly random rotations of up to 15 degrees, vertical image flipping and light blurring, significantly improve the model’s gener- alization to another data set, reducing boundary er- rors and enhancing segmentation accuracy. These results underscore the potential of targeted data augmentations in developing robust, generalizable models for hip joint component segmentation.

An accurate segmentation model for hip compo- nents could improve the diagnosis of Osteoarthritis, a prevalent age-related condition affecting joints. A significant challenge in developing effective and robust segmentation models are the domain differ- ences across various datasets. In this study, we in- vestigate the impact of different data augmentation and preprocessing techniques on the generalizabil- ity of femur segmentation models across datasets. Using two labeled datasets, we evaluate the perfor- mance of a U-Net segmentation model, focusing on the effectiveness of augmentations like image flip- ping, random rotations, blur, contrast, and bright- ness adjustments. Our findings reveal that certain augmentations, particularly random rotations of up to 15 degrees, vertical image flipping and light blurring, significantly improve the model’s gener- alization to another data set, reducing boundary er- rors and enhancing segmentation accuracy. These results underscore the potential of targeted data augmentations in developing robust, generalizable models for hip joint component segmentation.

Multi-view image recognition is crucial for numerous applications such as autonomous vehicles and robotics, where accurate 3D reconstructions from 2D images are essential. However, the presence of various noise factors like motion blur, variable lighting, and changes in the field of view can significantly degrade the quality of these reconstructions. Through a series of experiments using simulated data created in Blender, I explore the effectiveness of different de-noising methods, including AI-based and traditional image processing techniques. My findings indicate that specific adjustments in the Gaussian Splatting data preprocessing can significantly mitigate the adverse effects of noise, leading to more accurate and reliable 3D reconstructions. The study contributes to the field by providing a detailed analysis of noise impact and proposing viable solutions to enhance the fidelity of multi-view image reconstructions in noisy environments. The code and resources developed for this project are available in the GS-preprocessor repository on GitHub. https://github.com/surftijmen/GS-preprocessor.

Current robotic perception systems utilize a variety of sensors to estimate and understand a robot's surroundings. This paper focuses on a novel data representation technique that makes use of a recent scene reconstruction algorithm, known as 3D Gaussian Splatting, to explicitly represent and reason about an environment using only a sparse set of camera views as input. To achieve this, I generate and analyze the first cross-modal dataset consisting of 3D Gaussians and views taken around ten household objects. I introduce the resulting 3D Gaussians and images to a self-supervised feature learning network, that learns robust 2D and 3D embedding representations, by optimizing for the cross-view and cross-modality correspondence pretext tasks. I experiment with several 3D Gaussian features as input to the model and two point sub-network backbones, and report results on the two pretext tasks. The learned features are subsequently fine-tuned for the 2D and 3D shape recognition tasks. Moreover, by leveraging the fast scene reconstruction capabilities of the algorithm, I propose the use of rendered views as a visual memory aid to support downstream robotic tasks. The proposed networks achieve comparable results to state-of-the-art methods for point and image processing. The code associated to this paper is available at https://github.com/SimiOy/Self-Supervised-Learning-for-3DGS.

Current robotic perception systems utilize a variety of sensors to estimate and understand a robot's surroundings. This paper focuses on a novel data representation technique that makes use of a recent scene reconstruction algorithm, known as 3D Gaussian Splatting, to explicitly represent and reason about an environment using only a sparse set of camera views as input. To achieve this, I generate and analyze the first cross-modal dataset consisting of 3D Gaussians and views taken around ten household objects. I introduce the resulting 3D Gaussians and images to a self-supervised feature learning network, that learns robust 2D and 3D embedding representations, by optimizing for the cross-view and cross-modality correspondence pretext tasks. I experiment with several 3D Gaussian features as input to the model and two point sub-network backbones, and report results on the two pretext tasks. The learned features are subsequently fine-tuned for the 2D and 3D shape recognition tasks. Moreover, by leveraging the fast scene reconstruction capabilities of the algorithm, I propose the use of rendered views as a visual memory aid to support downstream robotic tasks. The proposed networks achieve comparable results to state-of-the-art methods for point and image processing. The code associated to this paper is available at https://github.com/SimiOy/Self-Supervised-Learning-for-3DGS.

Current robotic perception systems utilize a variety of sensors to estimate and understand a robot's surroundings. This paper focuses on a novel data representation technique that makes use of a recent scene reconstruction algorithm, known as 3D Gaussian Splatting, to explicitly represent and reason about an environment using only a sparse set of camera views as input. To achieve this, I generate and analyze the first cross-modal dataset consisting of 3D Gaussians and views taken around ten household objects. I introduce the resulting 3D Gaussians and images to a self-supervised feature learning network, that learns robust 2D and 3D embedding representations, by optimizing for the cross-view and cross-modality correspondence pretext tasks. I experiment with several 3D Gaussian features as input to the model and two point sub-network backbones, and report results on the two pretext tasks. The learned features are subsequently fine-tuned for the 2D and 3D shape recognition tasks. Moreover, by leveraging the fast scene reconstruction capabilities of the algorithm, I propose the use of rendered views as a visual memory aid to support downstream robotic tasks. The proposed networks achieve comparable results to state-of-the-art methods for point and image processing. The code associated to this paper is available at https://github.com/SimiOy/Self-Supervised-Learning-for-3DGS.

Osteoarthritis is a degenerative disease that affects the aging population by degrading the cartilage in the joints. The early and accurate diagnosis of this disease is key to effective treatment. For an early and accurate diagnosis of this disease, clinicians often use X-ray imaging. This allows medical professionals to manually measure the joint space width (JSW) in X-rays images to determine the progression of the disease. This method however proves to be both time-consuming and variable based on the professional. This research addresses the automation of the measurement of the JSW for the hip, using deep learning techniques, to improve precision and efficiency. The automated measurement of the JSW is challenged by variations in the imaging conditions across different clinical settings. To address these discrepancies and keep a good performance, domain adaptation techniques are used to counter these domain shifts to ensure a consistent JSW segmentation across different imaging domains. The study investigates whether a specific domain adaptation technique can enhance the accuracy and robustness of deep learning models specifically for femur segmentation in X-ray images across different datasets. A base deep learning model is developed for femur segmentation, and supervised domain adaptation is applied. The study compares the performance of the adapted model with the base model across two different datasets. Results indicate that supervised domain adaptation does not significantly improve the model’s robustness and accuracy in femur segmentation among two different datasets. These unexpected findings suggest that incorporating domain adaptation techniques may not always lead to a more reliable and efficient diagnosis of osteoarthritis, reducing the manual workload for clinicians.

Osteoarthritis is a degenerative disease that affects the aging population by degrading the cartilage in the joints. The early and accurate diagnosis of this disease is key to effective treatment. For an early and accurate diagnosis of this disease, clinicians often use X-ray imaging. This allows medical professionals to manually measure the joint space width (JSW) in X-rays images to determine the progression of the disease. This method however proves to be both time-consuming and variable based on the professional. This research addresses the automation of the measurement of the JSW for the hip, using deep learning techniques, to improve precision and efficiency. The automated measurement of the JSW is challenged by variations in the imaging conditions across different clinical settings. To address these discrepancies and keep a good performance, domain adaptation techniques are used to counter these domain shifts to ensure a consistent JSW segmentation across different imaging domains. The study investigates whether a specific domain adaptation technique can enhance the accuracy and robustness of deep learning models specifically for femur segmentation in X-ray images across different datasets. A base deep learning model is developed for femur segmentation, and supervised domain adaptation is applied. The study compares the performance of the adapted model with the base model across two different datasets. Results indicate that supervised domain adaptation does not significantly improve the model’s robustness and accuracy in femur segmentation among two different datasets. These unexpected findings suggest that incorporating domain adaptation techniques may not always lead to a more reliable and efficient diagnosis of osteoarthritis, reducing the manual workload for clinicians.