A. Vilanova Bartroli
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
15 records found
1
Analyzing high-dimensional images is a complex task. Unlike regular color images, they are not straightforward to visualize. Their additional information content increases the complexity of interpretation, both in terms of computational processing and human comprehension. Visual analytics - the combination of visualization, interaction, and automated analysis methods - has proven useful to gain insights into such large, difficult-to-handle data. For example, non-linear dimensionality reduction is a commonly employed technique in visual analytics for exposing interesting patterns through lower-dimensional representations of high-dimensional data.
In this thesis, we investigate non-linear dimensionality-reduction methods for the exploration of high-dimensional images. Specifically, we address the problem that current dimensionality-reduction methods are image-agnostic: they treat spatially resolved data without considering their spatial layout. We present algorithmic solutions that yield image-informed embeddings and interactions techniques that connect images and embedding representations. To a large extend, we utilize hierarchical approaches to handle the image data. We show how these techniques enable more insightful exploration of high-dimensional images.
Further, we present an open-source visual analytics software framework for rapid prototyping and extensible workflow development for high-dimensional data analysis. All algorithms and techniques described in this thesis are made available in or fully implemented as plugins for this framework. ...
In this thesis, we investigate non-linear dimensionality-reduction methods for the exploration of high-dimensional images. Specifically, we address the problem that current dimensionality-reduction methods are image-agnostic: they treat spatially resolved data without considering their spatial layout. We present algorithmic solutions that yield image-informed embeddings and interactions techniques that connect images and embedding representations. To a large extend, we utilize hierarchical approaches to handle the image data. We show how these techniques enable more insightful exploration of high-dimensional images.
Further, we present an open-source visual analytics software framework for rapid prototyping and extensible workflow development for high-dimensional data analysis. All algorithms and techniques described in this thesis are made available in or fully implemented as plugins for this framework. ...
Analyzing high-dimensional images is a complex task. Unlike regular color images, they are not straightforward to visualize. Their additional information content increases the complexity of interpretation, both in terms of computational processing and human comprehension. Visual analytics - the combination of visualization, interaction, and automated analysis methods - has proven useful to gain insights into such large, difficult-to-handle data. For example, non-linear dimensionality reduction is a commonly employed technique in visual analytics for exposing interesting patterns through lower-dimensional representations of high-dimensional data.
In this thesis, we investigate non-linear dimensionality-reduction methods for the exploration of high-dimensional images. Specifically, we address the problem that current dimensionality-reduction methods are image-agnostic: they treat spatially resolved data without considering their spatial layout. We present algorithmic solutions that yield image-informed embeddings and interactions techniques that connect images and embedding representations. To a large extend, we utilize hierarchical approaches to handle the image data. We show how these techniques enable more insightful exploration of high-dimensional images.
Further, we present an open-source visual analytics software framework for rapid prototyping and extensible workflow development for high-dimensional data analysis. All algorithms and techniques described in this thesis are made available in or fully implemented as plugins for this framework.
In this thesis, we investigate non-linear dimensionality-reduction methods for the exploration of high-dimensional images. Specifically, we address the problem that current dimensionality-reduction methods are image-agnostic: they treat spatially resolved data without considering their spatial layout. We present algorithmic solutions that yield image-informed embeddings and interactions techniques that connect images and embedding representations. To a large extend, we utilize hierarchical approaches to handle the image data. We show how these techniques enable more insightful exploration of high-dimensional images.
Further, we present an open-source visual analytics software framework for rapid prototyping and extensible workflow development for high-dimensional data analysis. All algorithms and techniques described in this thesis are made available in or fully implemented as plugins for this framework.
Doctoral thesis
(2025)
-
Nicolas F. Chaves-de-Plaza, K.A. Hildebrandt, R. van Egmond, A. Vilanova Bartroli
Radiotherapy (RT) is a widespread and effective technique to treat cancers by killing cancerous cells with rays of radiation. Building upon advances in image guidance and dose delivery technology like Proton Therapy, Adaptive RT promises more effective tumor decimation and a reduction of the incidence and severity of side effects. Unfortunately, the clinical implementation of adaptive workflows is challenging due to their resource-intensive nature. Therefore, their successful adoption lingers on overcoming several bottlenecks in the treatment planning process.
In this dissertation, we focus on methods used for the image segmentation or contouring step, which allows the localization of the anatomical structures required for dose optimization and evaluation. Until recently, clinicians had to manually delineate dozens of organs-at-risk and target volumes across hundreds of slices of the patient’s three-dimensional images. A process that is extremely time-consuming. The advent of deep learning-based artificial intelligence (AI) has changed the landscape: a modern auto-segmentation AI can produce segmentations for most of a patient’s anatomy in minutes.
Despite increasing automation in the segmentation process, it remains time and resource-intensive. Due to the segmentations’ criticality for the patient’s outcome and the errors the AI will commit, clinicians must perform a quality assessment of the AI’s outputs. Depending on the case’s complexity, the duration of the quality assessment process can negate the time gains auto-segmentation tools bring.
Deep ensemble AIs represent an advancement in medical image segmentation. Instead of providing a deterministic output, deep ensemble AIs produce a set of plausible candidates that aim to model inter-clinician annotation variability. Consensus segmentations obtained from ensembles tend to be more accurate and robust than the single-prediction deterministic counterpart. Nevertheless, by only using the consensus, a lot of potentially useful information is being discarded.
In this dissertation, we contribute to different phases of the segmentation quality assessment process. We characterize this process and introduce methods that leverage the raw outputs of deep ensemble AIs to support and speed up quality assessment tasks. The methods presented show new ways of analyzing and using ensembles in RT. Nevertheless, since these are relevant outside RT, we keep the presentation of the methods general and evaluate them in other application scenarios, such as the analysis of simulation ensembles or meteorological data.
Before fixing segmentation failures, clinicians must find them. This process can be time-consuming and fatiguing when failures are sparse and spread through the patient’s three-dimensional images. We present and evaluate a delineation error detection system, which guides clinicians to slices of three-dimensional images that contain potentially clinically relevant segmentation failures. We co-designed the DEDS with clinicians and refined it based on an observational study, which allowed us to characterize clinicians’ navigation patterns and the use of information sources like AI uncertainty and patients’ dose distributions. We evaluated the DEDS’ potential to speed up the QA process through a simulation study with a retrospective cohort of patients. Results indicate that speed-ups are the most significant when equipping the DEDS with information sources indicative of clinical priority, which prevents unnecessary edits.
Visual inspection of the segmentation ensemble permits understanding the main trends and detecting anomalies that might indicate segmentation failures. Using a spaghetti plot to visualize all ensemble members is straightforward but prone to clutter. Contour boxplots prevent clutter and extra complexity by distilling essential ensemble information, which permits more efficient ensemble inspection. Nevertheless, they are time-consuming to compute, reducing their practical value. We present Inclusion Depth for contour ensembles. Inclusion Depth yields per ensemble member centrality scores that allow characterizing the distribution of segmentation ensembles in terms of properties like the median, trimmed mean, confidence bands, and outliers. Compared to previous contour depth notions, Inclusion Depth is significantly faster, making it more applicable in practice for time-critical contexts like QA in adaptive RT. We show how Inclusion Depth permits creating contour boxplots for ensembles with hundreds of segmentations in seconds.
It is not uncommon for distinct representative shapes to co-occur within a contour ensemble. With ensembles created by clinicians, for instance, different institutions, training sessions, or experience levels can lead to distinct shapes (i.e., modes of variation) for the same structure. When trained on these data, deep ensemble AIs would yield similarly multimodal ensembles. In quality assessment, being able to extract these representatives would pave the way for new ensemble-based interactive segmentation workflows. Applying traditional contour depth notions to these multi-modal ensembles collapses the existing variation modes and can lead to uninformative centrality scores. To address this issue, we present the first framework for multi-modal contour depth, which also includes notable runtime improvements for depth computation. When used with Inclusion Depth, multi-modal contour depth permits clustering the different modes of variation and determining cluster-dependent scores that appropriately characterize the data. Variation modes can be then independently analyzed using uni-modal depth machinery like contour boxplots. xiii
The global perspective of contour depth methods, which consider the entire volume, may be insufficient when parts of the contours are noisy or when the resolution of the ensemble is too large to process within a reasonable time. Correlation clustering methods provide a solution by partitioning the spatial domain of the ensemble into highly correlated regions that can be used to localize analyses. Existing correlation clustering algorithms do not scale well as the resolution of the ensemble increases. We introduce the Local-to-Global Correlation Clustering (LoGCC) method, which partitions the ensemble’s spatial domain into coarser primitives, representing areas of consistent ensemble member behavior. Unlike previous correlation clustering methods, the proposed LoGCC achieves significantly faster runtimes by leveraging the ensemble’s spatial structure and decoupling computations into local and global steps. Like with Inclusion Depth, these speed gains enable LoGCC to analyze large datasets in time-critical fields such as adaptive radiotherapy (RT).
Throughout this dissertation, our approach focused on designing modular, flexible analysis methods applicable across different tasks and domains. We demonstrate how the delineation error detection system, multi-modal Inclusion Depth, and Local-to-Global Correlation Clustering support quality assessment in RT and extend to fields like meteorology. We also speculate on their potential as foundational elements for more complex workflows. For example, extracted modes of variation, which indicate representative shapes in the ensemble, could be repurposed as an interactive segmentation tool. Alternatively, consistent regions detected by correlation clustering could be used as building blocks to enable localized contour analysis and editing.
We hope the proposed contour ensemble visual analysis methods inspire the development of more efficient analysis workflows that harness ensembles’ power in RT and beyond. ...
In this dissertation, we focus on methods used for the image segmentation or contouring step, which allows the localization of the anatomical structures required for dose optimization and evaluation. Until recently, clinicians had to manually delineate dozens of organs-at-risk and target volumes across hundreds of slices of the patient’s three-dimensional images. A process that is extremely time-consuming. The advent of deep learning-based artificial intelligence (AI) has changed the landscape: a modern auto-segmentation AI can produce segmentations for most of a patient’s anatomy in minutes.
Despite increasing automation in the segmentation process, it remains time and resource-intensive. Due to the segmentations’ criticality for the patient’s outcome and the errors the AI will commit, clinicians must perform a quality assessment of the AI’s outputs. Depending on the case’s complexity, the duration of the quality assessment process can negate the time gains auto-segmentation tools bring.
Deep ensemble AIs represent an advancement in medical image segmentation. Instead of providing a deterministic output, deep ensemble AIs produce a set of plausible candidates that aim to model inter-clinician annotation variability. Consensus segmentations obtained from ensembles tend to be more accurate and robust than the single-prediction deterministic counterpart. Nevertheless, by only using the consensus, a lot of potentially useful information is being discarded.
In this dissertation, we contribute to different phases of the segmentation quality assessment process. We characterize this process and introduce methods that leverage the raw outputs of deep ensemble AIs to support and speed up quality assessment tasks. The methods presented show new ways of analyzing and using ensembles in RT. Nevertheless, since these are relevant outside RT, we keep the presentation of the methods general and evaluate them in other application scenarios, such as the analysis of simulation ensembles or meteorological data.
Before fixing segmentation failures, clinicians must find them. This process can be time-consuming and fatiguing when failures are sparse and spread through the patient’s three-dimensional images. We present and evaluate a delineation error detection system, which guides clinicians to slices of three-dimensional images that contain potentially clinically relevant segmentation failures. We co-designed the DEDS with clinicians and refined it based on an observational study, which allowed us to characterize clinicians’ navigation patterns and the use of information sources like AI uncertainty and patients’ dose distributions. We evaluated the DEDS’ potential to speed up the QA process through a simulation study with a retrospective cohort of patients. Results indicate that speed-ups are the most significant when equipping the DEDS with information sources indicative of clinical priority, which prevents unnecessary edits.
Visual inspection of the segmentation ensemble permits understanding the main trends and detecting anomalies that might indicate segmentation failures. Using a spaghetti plot to visualize all ensemble members is straightforward but prone to clutter. Contour boxplots prevent clutter and extra complexity by distilling essential ensemble information, which permits more efficient ensemble inspection. Nevertheless, they are time-consuming to compute, reducing their practical value. We present Inclusion Depth for contour ensembles. Inclusion Depth yields per ensemble member centrality scores that allow characterizing the distribution of segmentation ensembles in terms of properties like the median, trimmed mean, confidence bands, and outliers. Compared to previous contour depth notions, Inclusion Depth is significantly faster, making it more applicable in practice for time-critical contexts like QA in adaptive RT. We show how Inclusion Depth permits creating contour boxplots for ensembles with hundreds of segmentations in seconds.
It is not uncommon for distinct representative shapes to co-occur within a contour ensemble. With ensembles created by clinicians, for instance, different institutions, training sessions, or experience levels can lead to distinct shapes (i.e., modes of variation) for the same structure. When trained on these data, deep ensemble AIs would yield similarly multimodal ensembles. In quality assessment, being able to extract these representatives would pave the way for new ensemble-based interactive segmentation workflows. Applying traditional contour depth notions to these multi-modal ensembles collapses the existing variation modes and can lead to uninformative centrality scores. To address this issue, we present the first framework for multi-modal contour depth, which also includes notable runtime improvements for depth computation. When used with Inclusion Depth, multi-modal contour depth permits clustering the different modes of variation and determining cluster-dependent scores that appropriately characterize the data. Variation modes can be then independently analyzed using uni-modal depth machinery like contour boxplots. xiii
The global perspective of contour depth methods, which consider the entire volume, may be insufficient when parts of the contours are noisy or when the resolution of the ensemble is too large to process within a reasonable time. Correlation clustering methods provide a solution by partitioning the spatial domain of the ensemble into highly correlated regions that can be used to localize analyses. Existing correlation clustering algorithms do not scale well as the resolution of the ensemble increases. We introduce the Local-to-Global Correlation Clustering (LoGCC) method, which partitions the ensemble’s spatial domain into coarser primitives, representing areas of consistent ensemble member behavior. Unlike previous correlation clustering methods, the proposed LoGCC achieves significantly faster runtimes by leveraging the ensemble’s spatial structure and decoupling computations into local and global steps. Like with Inclusion Depth, these speed gains enable LoGCC to analyze large datasets in time-critical fields such as adaptive radiotherapy (RT).
Throughout this dissertation, our approach focused on designing modular, flexible analysis methods applicable across different tasks and domains. We demonstrate how the delineation error detection system, multi-modal Inclusion Depth, and Local-to-Global Correlation Clustering support quality assessment in RT and extend to fields like meteorology. We also speculate on their potential as foundational elements for more complex workflows. For example, extracted modes of variation, which indicate representative shapes in the ensemble, could be repurposed as an interactive segmentation tool. Alternatively, consistent regions detected by correlation clustering could be used as building blocks to enable localized contour analysis and editing.
We hope the proposed contour ensemble visual analysis methods inspire the development of more efficient analysis workflows that harness ensembles’ power in RT and beyond. ...
Radiotherapy (RT) is a widespread and effective technique to treat cancers by killing cancerous cells with rays of radiation. Building upon advances in image guidance and dose delivery technology like Proton Therapy, Adaptive RT promises more effective tumor decimation and a reduction of the incidence and severity of side effects. Unfortunately, the clinical implementation of adaptive workflows is challenging due to their resource-intensive nature. Therefore, their successful adoption lingers on overcoming several bottlenecks in the treatment planning process.
In this dissertation, we focus on methods used for the image segmentation or contouring step, which allows the localization of the anatomical structures required for dose optimization and evaluation. Until recently, clinicians had to manually delineate dozens of organs-at-risk and target volumes across hundreds of slices of the patient’s three-dimensional images. A process that is extremely time-consuming. The advent of deep learning-based artificial intelligence (AI) has changed the landscape: a modern auto-segmentation AI can produce segmentations for most of a patient’s anatomy in minutes.
Despite increasing automation in the segmentation process, it remains time and resource-intensive. Due to the segmentations’ criticality for the patient’s outcome and the errors the AI will commit, clinicians must perform a quality assessment of the AI’s outputs. Depending on the case’s complexity, the duration of the quality assessment process can negate the time gains auto-segmentation tools bring.
Deep ensemble AIs represent an advancement in medical image segmentation. Instead of providing a deterministic output, deep ensemble AIs produce a set of plausible candidates that aim to model inter-clinician annotation variability. Consensus segmentations obtained from ensembles tend to be more accurate and robust than the single-prediction deterministic counterpart. Nevertheless, by only using the consensus, a lot of potentially useful information is being discarded.
In this dissertation, we contribute to different phases of the segmentation quality assessment process. We characterize this process and introduce methods that leverage the raw outputs of deep ensemble AIs to support and speed up quality assessment tasks. The methods presented show new ways of analyzing and using ensembles in RT. Nevertheless, since these are relevant outside RT, we keep the presentation of the methods general and evaluate them in other application scenarios, such as the analysis of simulation ensembles or meteorological data.
Before fixing segmentation failures, clinicians must find them. This process can be time-consuming and fatiguing when failures are sparse and spread through the patient’s three-dimensional images. We present and evaluate a delineation error detection system, which guides clinicians to slices of three-dimensional images that contain potentially clinically relevant segmentation failures. We co-designed the DEDS with clinicians and refined it based on an observational study, which allowed us to characterize clinicians’ navigation patterns and the use of information sources like AI uncertainty and patients’ dose distributions. We evaluated the DEDS’ potential to speed up the QA process through a simulation study with a retrospective cohort of patients. Results indicate that speed-ups are the most significant when equipping the DEDS with information sources indicative of clinical priority, which prevents unnecessary edits.
Visual inspection of the segmentation ensemble permits understanding the main trends and detecting anomalies that might indicate segmentation failures. Using a spaghetti plot to visualize all ensemble members is straightforward but prone to clutter. Contour boxplots prevent clutter and extra complexity by distilling essential ensemble information, which permits more efficient ensemble inspection. Nevertheless, they are time-consuming to compute, reducing their practical value. We present Inclusion Depth for contour ensembles. Inclusion Depth yields per ensemble member centrality scores that allow characterizing the distribution of segmentation ensembles in terms of properties like the median, trimmed mean, confidence bands, and outliers. Compared to previous contour depth notions, Inclusion Depth is significantly faster, making it more applicable in practice for time-critical contexts like QA in adaptive RT. We show how Inclusion Depth permits creating contour boxplots for ensembles with hundreds of segmentations in seconds.
It is not uncommon for distinct representative shapes to co-occur within a contour ensemble. With ensembles created by clinicians, for instance, different institutions, training sessions, or experience levels can lead to distinct shapes (i.e., modes of variation) for the same structure. When trained on these data, deep ensemble AIs would yield similarly multimodal ensembles. In quality assessment, being able to extract these representatives would pave the way for new ensemble-based interactive segmentation workflows. Applying traditional contour depth notions to these multi-modal ensembles collapses the existing variation modes and can lead to uninformative centrality scores. To address this issue, we present the first framework for multi-modal contour depth, which also includes notable runtime improvements for depth computation. When used with Inclusion Depth, multi-modal contour depth permits clustering the different modes of variation and determining cluster-dependent scores that appropriately characterize the data. Variation modes can be then independently analyzed using uni-modal depth machinery like contour boxplots. xiii
The global perspective of contour depth methods, which consider the entire volume, may be insufficient when parts of the contours are noisy or when the resolution of the ensemble is too large to process within a reasonable time. Correlation clustering methods provide a solution by partitioning the spatial domain of the ensemble into highly correlated regions that can be used to localize analyses. Existing correlation clustering algorithms do not scale well as the resolution of the ensemble increases. We introduce the Local-to-Global Correlation Clustering (LoGCC) method, which partitions the ensemble’s spatial domain into coarser primitives, representing areas of consistent ensemble member behavior. Unlike previous correlation clustering methods, the proposed LoGCC achieves significantly faster runtimes by leveraging the ensemble’s spatial structure and decoupling computations into local and global steps. Like with Inclusion Depth, these speed gains enable LoGCC to analyze large datasets in time-critical fields such as adaptive radiotherapy (RT).
Throughout this dissertation, our approach focused on designing modular, flexible analysis methods applicable across different tasks and domains. We demonstrate how the delineation error detection system, multi-modal Inclusion Depth, and Local-to-Global Correlation Clustering support quality assessment in RT and extend to fields like meteorology. We also speculate on their potential as foundational elements for more complex workflows. For example, extracted modes of variation, which indicate representative shapes in the ensemble, could be repurposed as an interactive segmentation tool. Alternatively, consistent regions detected by correlation clustering could be used as building blocks to enable localized contour analysis and editing.
We hope the proposed contour ensemble visual analysis methods inspire the development of more efficient analysis workflows that harness ensembles’ power in RT and beyond.
In this dissertation, we focus on methods used for the image segmentation or contouring step, which allows the localization of the anatomical structures required for dose optimization and evaluation. Until recently, clinicians had to manually delineate dozens of organs-at-risk and target volumes across hundreds of slices of the patient’s three-dimensional images. A process that is extremely time-consuming. The advent of deep learning-based artificial intelligence (AI) has changed the landscape: a modern auto-segmentation AI can produce segmentations for most of a patient’s anatomy in minutes.
Despite increasing automation in the segmentation process, it remains time and resource-intensive. Due to the segmentations’ criticality for the patient’s outcome and the errors the AI will commit, clinicians must perform a quality assessment of the AI’s outputs. Depending on the case’s complexity, the duration of the quality assessment process can negate the time gains auto-segmentation tools bring.
Deep ensemble AIs represent an advancement in medical image segmentation. Instead of providing a deterministic output, deep ensemble AIs produce a set of plausible candidates that aim to model inter-clinician annotation variability. Consensus segmentations obtained from ensembles tend to be more accurate and robust than the single-prediction deterministic counterpart. Nevertheless, by only using the consensus, a lot of potentially useful information is being discarded.
In this dissertation, we contribute to different phases of the segmentation quality assessment process. We characterize this process and introduce methods that leverage the raw outputs of deep ensemble AIs to support and speed up quality assessment tasks. The methods presented show new ways of analyzing and using ensembles in RT. Nevertheless, since these are relevant outside RT, we keep the presentation of the methods general and evaluate them in other application scenarios, such as the analysis of simulation ensembles or meteorological data.
Before fixing segmentation failures, clinicians must find them. This process can be time-consuming and fatiguing when failures are sparse and spread through the patient’s three-dimensional images. We present and evaluate a delineation error detection system, which guides clinicians to slices of three-dimensional images that contain potentially clinically relevant segmentation failures. We co-designed the DEDS with clinicians and refined it based on an observational study, which allowed us to characterize clinicians’ navigation patterns and the use of information sources like AI uncertainty and patients’ dose distributions. We evaluated the DEDS’ potential to speed up the QA process through a simulation study with a retrospective cohort of patients. Results indicate that speed-ups are the most significant when equipping the DEDS with information sources indicative of clinical priority, which prevents unnecessary edits.
Visual inspection of the segmentation ensemble permits understanding the main trends and detecting anomalies that might indicate segmentation failures. Using a spaghetti plot to visualize all ensemble members is straightforward but prone to clutter. Contour boxplots prevent clutter and extra complexity by distilling essential ensemble information, which permits more efficient ensemble inspection. Nevertheless, they are time-consuming to compute, reducing their practical value. We present Inclusion Depth for contour ensembles. Inclusion Depth yields per ensemble member centrality scores that allow characterizing the distribution of segmentation ensembles in terms of properties like the median, trimmed mean, confidence bands, and outliers. Compared to previous contour depth notions, Inclusion Depth is significantly faster, making it more applicable in practice for time-critical contexts like QA in adaptive RT. We show how Inclusion Depth permits creating contour boxplots for ensembles with hundreds of segmentations in seconds.
It is not uncommon for distinct representative shapes to co-occur within a contour ensemble. With ensembles created by clinicians, for instance, different institutions, training sessions, or experience levels can lead to distinct shapes (i.e., modes of variation) for the same structure. When trained on these data, deep ensemble AIs would yield similarly multimodal ensembles. In quality assessment, being able to extract these representatives would pave the way for new ensemble-based interactive segmentation workflows. Applying traditional contour depth notions to these multi-modal ensembles collapses the existing variation modes and can lead to uninformative centrality scores. To address this issue, we present the first framework for multi-modal contour depth, which also includes notable runtime improvements for depth computation. When used with Inclusion Depth, multi-modal contour depth permits clustering the different modes of variation and determining cluster-dependent scores that appropriately characterize the data. Variation modes can be then independently analyzed using uni-modal depth machinery like contour boxplots. xiii
The global perspective of contour depth methods, which consider the entire volume, may be insufficient when parts of the contours are noisy or when the resolution of the ensemble is too large to process within a reasonable time. Correlation clustering methods provide a solution by partitioning the spatial domain of the ensemble into highly correlated regions that can be used to localize analyses. Existing correlation clustering algorithms do not scale well as the resolution of the ensemble increases. We introduce the Local-to-Global Correlation Clustering (LoGCC) method, which partitions the ensemble’s spatial domain into coarser primitives, representing areas of consistent ensemble member behavior. Unlike previous correlation clustering methods, the proposed LoGCC achieves significantly faster runtimes by leveraging the ensemble’s spatial structure and decoupling computations into local and global steps. Like with Inclusion Depth, these speed gains enable LoGCC to analyze large datasets in time-critical fields such as adaptive radiotherapy (RT).
Throughout this dissertation, our approach focused on designing modular, flexible analysis methods applicable across different tasks and domains. We demonstrate how the delineation error detection system, multi-modal Inclusion Depth, and Local-to-Global Correlation Clustering support quality assessment in RT and extend to fields like meteorology. We also speculate on their potential as foundational elements for more complex workflows. For example, extracted modes of variation, which indicate representative shapes in the ensemble, could be repurposed as an interactive segmentation tool. Alternatively, consistent regions detected by correlation clustering could be used as building blocks to enable localized contour analysis and editing.
We hope the proposed contour ensemble visual analysis methods inspire the development of more efficient analysis workflows that harness ensembles’ power in RT and beyond.
Fiber tracking enables the in-vivo reconstruction of white matter pathways in the brain and has significant potential in clinical workflows such as neurosurgical planning. However, its broader clinical adoption remains limited due to the high degree of uncertainty that arises throughout the processing pipeline, from diffusionMRI acquisition to modeling, tracking, and visualization. These uncertainties are rarely communicated in current clinical visualizations, which often present results as definitive, potentially misleading clinicians and affecting critical decisions.
This thesis explores the integration of fiber tracking uncertainty visualization into neurosurgical workflow, aiming to enhance the interpretability and transparency of the results in a clinical context. A key challenge lies in balancing computational complexity with clear representation, while ensuring the solutions remain aligned with clinical workflows.
To address this, we introduce interactive and computationally efficient visualization approaches that represent uncertainties and support clinicians in understanding how fiber tracking results may vary with inherent uncertainties. These techniques are evaluated through collaborations with medical experts and incorporated into decisionmaking studies to assess their practical relevance.
The contributions presented in this thesis advance the integration of uncertainty into clinical fiber tracking visualization and highlight how embracing uncertainty, rather than ignoring it, can lead to safer and more informed clinical decisions. ...
This thesis explores the integration of fiber tracking uncertainty visualization into neurosurgical workflow, aiming to enhance the interpretability and transparency of the results in a clinical context. A key challenge lies in balancing computational complexity with clear representation, while ensuring the solutions remain aligned with clinical workflows.
To address this, we introduce interactive and computationally efficient visualization approaches that represent uncertainties and support clinicians in understanding how fiber tracking results may vary with inherent uncertainties. These techniques are evaluated through collaborations with medical experts and incorporated into decisionmaking studies to assess their practical relevance.
The contributions presented in this thesis advance the integration of uncertainty into clinical fiber tracking visualization and highlight how embracing uncertainty, rather than ignoring it, can lead to safer and more informed clinical decisions. ...
Fiber tracking enables the in-vivo reconstruction of white matter pathways in the brain and has significant potential in clinical workflows such as neurosurgical planning. However, its broader clinical adoption remains limited due to the high degree of uncertainty that arises throughout the processing pipeline, from diffusionMRI acquisition to modeling, tracking, and visualization. These uncertainties are rarely communicated in current clinical visualizations, which often present results as definitive, potentially misleading clinicians and affecting critical decisions.
This thesis explores the integration of fiber tracking uncertainty visualization into neurosurgical workflow, aiming to enhance the interpretability and transparency of the results in a clinical context. A key challenge lies in balancing computational complexity with clear representation, while ensuring the solutions remain aligned with clinical workflows.
To address this, we introduce interactive and computationally efficient visualization approaches that represent uncertainties and support clinicians in understanding how fiber tracking results may vary with inherent uncertainties. These techniques are evaluated through collaborations with medical experts and incorporated into decisionmaking studies to assess their practical relevance.
The contributions presented in this thesis advance the integration of uncertainty into clinical fiber tracking visualization and highlight how embracing uncertainty, rather than ignoring it, can lead to safer and more informed clinical decisions.
This thesis explores the integration of fiber tracking uncertainty visualization into neurosurgical workflow, aiming to enhance the interpretability and transparency of the results in a clinical context. A key challenge lies in balancing computational complexity with clear representation, while ensuring the solutions remain aligned with clinical workflows.
To address this, we introduce interactive and computationally efficient visualization approaches that represent uncertainties and support clinicians in understanding how fiber tracking results may vary with inherent uncertainties. These techniques are evaluated through collaborations with medical experts and incorporated into decisionmaking studies to assess their practical relevance.
The contributions presented in this thesis advance the integration of uncertainty into clinical fiber tracking visualization and highlight how embracing uncertainty, rather than ignoring it, can lead to safer and more informed clinical decisions.
TSNE is a popular technique for visualizing high-dimensional data. It finds a low-dimensional representation of the data, also known as embedding, by optimizing a highly non-linear cost function. The optimization process is done iteratively, often with first-order methods such as gradient descent (GD). The performance of iterative optimization procedures depends on the time that it takes to execute each iteration and the number of iterations that are required to converge to an optimum. In the context of tSNE, different methods that tackle these issues have appeared, but, despite their success, they are still limited by their separate development. For instance, BH-SNE, one of the most popular variants of tSNE, uses the Barnes-Hut algorithm to accelerate each iteration, but still uses GD, which needs a large number of them to converge. On the other hand, quasi-Newton optimization methods are effective at reducing the number of iterations, but available implementations are constrained by the additional per-iteration cost. In our work, we attempted to reconcile these two research branches. For this, we investigated the usage of Line Search (LS) and Trust Region (TR) based quasi-Newton methods for tSNE and improved their performance by identifying and addressing several of their bottlenecks. Among the different aspects of quasi-Newton optimization for tSNE that we covered were the selection of a suitable Hessian approximation, the usage of approximate gradients and cost function evaluations, and the choice of linear solver, which the algorithm uses to find the next iterate. The resulting techniques, which we denote SD-LS-CG and SD-TR-CG, have a lower per-iteration cost than alternative quasi-Newton methods and converge in fewer iterations than the traditional implementation of tSNE, which uses a highly tuned version of GD. The results suggest that these quasi-Newton methods could be a better alternative to fast tSNE implementations such as BH-SNE, given their overall faster convergence. Furthermore, we observed that SD-LS-CG was able to consistently avoid local optima if initialized with PCA, suggesting that the choice of Hessian approximation can have a similar role to the early exaggeration phase in tSNE.
...
TSNE is a popular technique for visualizing high-dimensional data. It finds a low-dimensional representation of the data, also known as embedding, by optimizing a highly non-linear cost function. The optimization process is done iteratively, often with first-order methods such as gradient descent (GD). The performance of iterative optimization procedures depends on the time that it takes to execute each iteration and the number of iterations that are required to converge to an optimum. In the context of tSNE, different methods that tackle these issues have appeared, but, despite their success, they are still limited by their separate development. For instance, BH-SNE, one of the most popular variants of tSNE, uses the Barnes-Hut algorithm to accelerate each iteration, but still uses GD, which needs a large number of them to converge. On the other hand, quasi-Newton optimization methods are effective at reducing the number of iterations, but available implementations are constrained by the additional per-iteration cost. In our work, we attempted to reconcile these two research branches. For this, we investigated the usage of Line Search (LS) and Trust Region (TR) based quasi-Newton methods for tSNE and improved their performance by identifying and addressing several of their bottlenecks. Among the different aspects of quasi-Newton optimization for tSNE that we covered were the selection of a suitable Hessian approximation, the usage of approximate gradients and cost function evaluations, and the choice of linear solver, which the algorithm uses to find the next iterate. The resulting techniques, which we denote SD-LS-CG and SD-TR-CG, have a lower per-iteration cost than alternative quasi-Newton methods and converge in fewer iterations than the traditional implementation of tSNE, which uses a highly tuned version of GD. The results suggest that these quasi-Newton methods could be a better alternative to fast tSNE implementations such as BH-SNE, given their overall faster convergence. Furthermore, we observed that SD-LS-CG was able to consistently avoid local optima if initialized with PCA, suggesting that the choice of Hessian approximation can have a similar role to the early exaggeration phase in tSNE.
Narcolepsy is a chronic neurological condition that results from the dysregulation of the sleep-wake cycle occurring in an early stage, specifically in adolescence. Patients with narcolepsy experience excessive daytime sleepiness, cataplexy, hypnagogic hallucinations, sleep paralysis and disturbed nocturnal sleep and these symptoms together form the narcolepsy symptom pentad. However, the symptoms related to narcolepsy are not limited to the pentad and cover a broad range of other symptoms, some of which are not directly related to sleep, like, increase in weight, binge eating, anxiety, agitation. Therefore, researchers from sleep medicine center, Kempenhaeghe wanted to understand how a selected set of 20 symptoms are related to narcolepsy.
In this thesis we present a visual analytics framework to help the researchers understand these symptoms in relation to narcolepsy and identify patterns among them. Using relevant attributes interesting population subsets are formed and the results are compared. The thesis comprises of three main tasks, the first being visualizing the distribution of individual attributes of patients in which a symptom is present and not present and for severity level. Then, pairs of symptoms are visualized based on their agreement and association to identify groups of related symptoms and trends present. Lastly, multiple symptoms are visualized to identify patterns amongst them.
The visual analytics framework was implemented and evaluated through a user study. The study showed that the visualization developed was useful in gaining new insights to form interesting hypothesis along with possible extensions for future development. This is the first step towards an extensive visual analytics framework in the study of narcolepsy symptom spectrum. ...
In this thesis we present a visual analytics framework to help the researchers understand these symptoms in relation to narcolepsy and identify patterns among them. Using relevant attributes interesting population subsets are formed and the results are compared. The thesis comprises of three main tasks, the first being visualizing the distribution of individual attributes of patients in which a symptom is present and not present and for severity level. Then, pairs of symptoms are visualized based on their agreement and association to identify groups of related symptoms and trends present. Lastly, multiple symptoms are visualized to identify patterns amongst them.
The visual analytics framework was implemented and evaluated through a user study. The study showed that the visualization developed was useful in gaining new insights to form interesting hypothesis along with possible extensions for future development. This is the first step towards an extensive visual analytics framework in the study of narcolepsy symptom spectrum. ...
Narcolepsy is a chronic neurological condition that results from the dysregulation of the sleep-wake cycle occurring in an early stage, specifically in adolescence. Patients with narcolepsy experience excessive daytime sleepiness, cataplexy, hypnagogic hallucinations, sleep paralysis and disturbed nocturnal sleep and these symptoms together form the narcolepsy symptom pentad. However, the symptoms related to narcolepsy are not limited to the pentad and cover a broad range of other symptoms, some of which are not directly related to sleep, like, increase in weight, binge eating, anxiety, agitation. Therefore, researchers from sleep medicine center, Kempenhaeghe wanted to understand how a selected set of 20 symptoms are related to narcolepsy.
In this thesis we present a visual analytics framework to help the researchers understand these symptoms in relation to narcolepsy and identify patterns among them. Using relevant attributes interesting population subsets are formed and the results are compared. The thesis comprises of three main tasks, the first being visualizing the distribution of individual attributes of patients in which a symptom is present and not present and for severity level. Then, pairs of symptoms are visualized based on their agreement and association to identify groups of related symptoms and trends present. Lastly, multiple symptoms are visualized to identify patterns amongst them.
The visual analytics framework was implemented and evaluated through a user study. The study showed that the visualization developed was useful in gaining new insights to form interesting hypothesis along with possible extensions for future development. This is the first step towards an extensive visual analytics framework in the study of narcolepsy symptom spectrum.
In this thesis we present a visual analytics framework to help the researchers understand these symptoms in relation to narcolepsy and identify patterns among them. Using relevant attributes interesting population subsets are formed and the results are compared. The thesis comprises of three main tasks, the first being visualizing the distribution of individual attributes of patients in which a symptom is present and not present and for severity level. Then, pairs of symptoms are visualized based on their agreement and association to identify groups of related symptoms and trends present. Lastly, multiple symptoms are visualized to identify patterns amongst them.
The visual analytics framework was implemented and evaluated through a user study. The study showed that the visualization developed was useful in gaining new insights to form interesting hypothesis along with possible extensions for future development. This is the first step towards an extensive visual analytics framework in the study of narcolepsy symptom spectrum.
Master thesis
(2020)
-
Gijs Reichert, Anna Vilanova Bartroli, Ricardo Marroquim, Jan van Gemert, Gert van der Heijden, Douwe Broekens
Nowadays video plays an important role in the coaching of athletes across many different sports. To make more use of the advantages videos can provide for coaching, the Dutch Sailing team is shifting from manually recording short videos towards continuously recording training sessions. This new recording approach provides opportunities and creates challenges at the same time. In this thesis we present a pipeline to address the problems with the stability of the recording and the first steps towards a Visual Analysis Framework, which leverages the available video data. New information is extracted from the video recordings by detecting and tracking the boat and sailors. Moreover, we semi-automatically highlight interesting intervals in time of a recorded training session. These are the first steps towards an extensive Visual Analysis Framework which has the potential to make the analysis of the videos easier and provide the coaches with tools to improve the analysis of the performance of the sailors.
...
Nowadays video plays an important role in the coaching of athletes across many different sports. To make more use of the advantages videos can provide for coaching, the Dutch Sailing team is shifting from manually recording short videos towards continuously recording training sessions. This new recording approach provides opportunities and creates challenges at the same time. In this thesis we present a pipeline to address the problems with the stability of the recording and the first steps towards a Visual Analysis Framework, which leverages the available video data. New information is extracted from the video recordings by detecting and tracking the boat and sailors. Moreover, we semi-automatically highlight interesting intervals in time of a recorded training session. These are the first steps towards an extensive Visual Analysis Framework which has the potential to make the analysis of the videos easier and provide the coaches with tools to improve the analysis of the performance of the sailors.
PC-MRI Blood-Flow measurements
Visualization and Data Assimilation
Heart and vessel diseases, or cardiovascular diseases (CVDs), are globally the main cause of mortality and morbidity. The blood flow plays an important role in their occurrence and progression. Therefore, knowledge of the blood flow is of key importance to reduce and threat these diseases. This knowledge requires both high-quality data and an insightful visual representation. Using advanced imaging techniques, for example phase-contrast magnetic resonance imaging (PC-MRI), the blood flow in a patient can be measured. This technique provides patient-specific 3D data over time, that is, for every measured position, a so-called voxel, the velocity of the blood is measured in all three directions. From these three values per voxel, a vector can then be reconstructed that tells us how fast and in which direction the blood was flowing at the measured location. By doing this for multiple moments one can obtain this data throughout a heart cycle. Since this data is three dimensional and changing over time generating a visual representation of this data is challenging for multiple reasons. One such reason is occlusion where part of the visualization is hidden by the rest of the visualization. Another is visual clutter where the visualization is ``too busy'' and therefore unclear. Moreover, the measured data is subject to noise and artefacts which further hinder the visualization. In this dissertation, novel visualization approaches are presented that address these and other visualization challenges of PC-MRI data. Besides PC-MRI data, blood flow data can be created using computer simulation models, for example using computational fluid dynamics (CFD) models, that are based on physical models. For this usually the shape of the blood vessel is measured using imaging techniques which is in turn used to simulate the blood flow inside the vessel. However, both measuring and modelling the blood flow have their own advantages and disadvantages. For example, PC-MRI measurements suffer from the inevitable effects of measurement noise which causes the data to deviate from the actual blood flow in the patient. Simulations, on the other hand, require detailed input information and are based on model assumptions, and hence result in data that is not always representative for the patient, however, the resulting data does correspond to the physical model. In addition to visualization approaches, this work also presents novel methods that combine PC-MRI measurements and simulations such that the resulting data is both physically-plausible and patient-specific.
...
Heart and vessel diseases, or cardiovascular diseases (CVDs), are globally the main cause of mortality and morbidity. The blood flow plays an important role in their occurrence and progression. Therefore, knowledge of the blood flow is of key importance to reduce and threat these diseases. This knowledge requires both high-quality data and an insightful visual representation. Using advanced imaging techniques, for example phase-contrast magnetic resonance imaging (PC-MRI), the blood flow in a patient can be measured. This technique provides patient-specific 3D data over time, that is, for every measured position, a so-called voxel, the velocity of the blood is measured in all three directions. From these three values per voxel, a vector can then be reconstructed that tells us how fast and in which direction the blood was flowing at the measured location. By doing this for multiple moments one can obtain this data throughout a heart cycle. Since this data is three dimensional and changing over time generating a visual representation of this data is challenging for multiple reasons. One such reason is occlusion where part of the visualization is hidden by the rest of the visualization. Another is visual clutter where the visualization is ``too busy'' and therefore unclear. Moreover, the measured data is subject to noise and artefacts which further hinder the visualization. In this dissertation, novel visualization approaches are presented that address these and other visualization challenges of PC-MRI data. Besides PC-MRI data, blood flow data can be created using computer simulation models, for example using computational fluid dynamics (CFD) models, that are based on physical models. For this usually the shape of the blood vessel is measured using imaging techniques which is in turn used to simulate the blood flow inside the vessel. However, both measuring and modelling the blood flow have their own advantages and disadvantages. For example, PC-MRI measurements suffer from the inevitable effects of measurement noise which causes the data to deviate from the actual blood flow in the patient. Simulations, on the other hand, require detailed input information and are based on model assumptions, and hence result in data that is not always representative for the patient, however, the resulting data does correspond to the physical model. In addition to visualization approaches, this work also presents novel methods that combine PC-MRI measurements and simulations such that the resulting data is both physically-plausible and patient-specific.
Gamification and game-based learning have received wide attention in the past few decades. By blending game design and mechanics into traditional learning environment, they enhance students’ participation, motivation and engagement. Squla is such a gamified learning platform where we can find game components like coins, virtual shop, competitive activities and collectables. Besides the above gamified elements, Squla has also transformed standard questions into fun games such as shooting catapults and clicking the popping bubbles. These games are designed to further engage the students and improve their learning.
In this project, we analysed player game type preference based on their game log data, and measure the impact of customised game type delivery. We targeted education group 4 and 5 students users, and focus on catapult games and bubble popper games as they are the most played. A set of features that could reflect students’ preference and emotion states are selected and analysed, including correct ratio, playtime, quitting possibility, etc. Using data clustering, we group students who have similar behaviour and predict their preferred game types. We identified three group of students, one shows high completion rate on all forms of questions, another shows rather low overall completion rate, and the last group has rather high completion rate on bubble popper games and lower completion rate on the catapult shooting games. Based on such findings, we conducted experiment on them to look into different gaming contents’ impact on their learning and engagement. A final experiment consists of a short math quiz and a follow-up questionnaire. The two-week online experiment receives 91 valid responses. Post-play questionnaire, as well as the game log suggest different contents could affect students’ engagement. In particular, preferred contents can elevate a sense of happiness and enhance perceived learning. ...
In this project, we analysed player game type preference based on their game log data, and measure the impact of customised game type delivery. We targeted education group 4 and 5 students users, and focus on catapult games and bubble popper games as they are the most played. A set of features that could reflect students’ preference and emotion states are selected and analysed, including correct ratio, playtime, quitting possibility, etc. Using data clustering, we group students who have similar behaviour and predict their preferred game types. We identified three group of students, one shows high completion rate on all forms of questions, another shows rather low overall completion rate, and the last group has rather high completion rate on bubble popper games and lower completion rate on the catapult shooting games. Based on such findings, we conducted experiment on them to look into different gaming contents’ impact on their learning and engagement. A final experiment consists of a short math quiz and a follow-up questionnaire. The two-week online experiment receives 91 valid responses. Post-play questionnaire, as well as the game log suggest different contents could affect students’ engagement. In particular, preferred contents can elevate a sense of happiness and enhance perceived learning. ...
Gamification and game-based learning have received wide attention in the past few decades. By blending game design and mechanics into traditional learning environment, they enhance students’ participation, motivation and engagement. Squla is such a gamified learning platform where we can find game components like coins, virtual shop, competitive activities and collectables. Besides the above gamified elements, Squla has also transformed standard questions into fun games such as shooting catapults and clicking the popping bubbles. These games are designed to further engage the students and improve their learning.
In this project, we analysed player game type preference based on their game log data, and measure the impact of customised game type delivery. We targeted education group 4 and 5 students users, and focus on catapult games and bubble popper games as they are the most played. A set of features that could reflect students’ preference and emotion states are selected and analysed, including correct ratio, playtime, quitting possibility, etc. Using data clustering, we group students who have similar behaviour and predict their preferred game types. We identified three group of students, one shows high completion rate on all forms of questions, another shows rather low overall completion rate, and the last group has rather high completion rate on bubble popper games and lower completion rate on the catapult shooting games. Based on such findings, we conducted experiment on them to look into different gaming contents’ impact on their learning and engagement. A final experiment consists of a short math quiz and a follow-up questionnaire. The two-week online experiment receives 91 valid responses. Post-play questionnaire, as well as the game log suggest different contents could affect students’ engagement. In particular, preferred contents can elevate a sense of happiness and enhance perceived learning.
In this project, we analysed player game type preference based on their game log data, and measure the impact of customised game type delivery. We targeted education group 4 and 5 students users, and focus on catapult games and bubble popper games as they are the most played. A set of features that could reflect students’ preference and emotion states are selected and analysed, including correct ratio, playtime, quitting possibility, etc. Using data clustering, we group students who have similar behaviour and predict their preferred game types. We identified three group of students, one shows high completion rate on all forms of questions, another shows rather low overall completion rate, and the last group has rather high completion rate on bubble popper games and lower completion rate on the catapult shooting games. Based on such findings, we conducted experiment on them to look into different gaming contents’ impact on their learning and engagement. A final experiment consists of a short math quiz and a follow-up questionnaire. The two-week online experiment receives 91 valid responses. Post-play questionnaire, as well as the game log suggest different contents could affect students’ engagement. In particular, preferred contents can elevate a sense of happiness and enhance perceived learning.
Doctoral thesis
(2019)
-
Nicola Pezzotti, Anna Vilanova Bartroli, Boudewijn Lelieveldt, Elmar Eisemann
Visual analysis of high dimensional data is a challenging process. Direct visualizations work well for a few dimensions but do not scale to the hundreds or thousands of dimensions that have become increasingly common in current data analytics problems. Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces, and it has been proven as an effective tool for high dimensional data analysis. In visual analytics systems, several visualizations are jointly analyzed in order to discover patterns in the data. One of the fundamental tools that has been integrated in visual analytics, is nonlinear dimensionality-reduction; a tool for the indirect visualization aimed at the discovery and analysis of non-linear patterns in the high-dimensional data. However, the computational complexity of non-linear dimensionality-reduction techniques does not allow direct employment in interactive systems. This limitation makes the analytic process a time-consuming task that can take hours, days or even weeks to be performed. In this thesis, we present novel algorithmic solutions that enable integration of non-linear dimensionality-reduction techniques in visual analytics systems. Our proposed algorithms are, not only much faster than existing solutions, but provide richer insights into the data at hand. This result, is achieved by introducing new data processing and optimization techniques and by embracing the recently introduced concept of Progressive Visual Analytics; a computational paradigm that enables the interactivity of complex analytics techniques by means of visualization as well as interaction with intermediate results. Moreover, we present several applications that are designed to provide unprecedented analytical capabilities in several domains. These applications are powered by the algorithms introduced in this dissertation and led to several discoveries in areas ranging from the biomedical research field, to social-network data analysis and machine-learning models interpretability.
...
Visual analysis of high dimensional data is a challenging process. Direct visualizations work well for a few dimensions but do not scale to the hundreds or thousands of dimensions that have become increasingly common in current data analytics problems. Visual analytics is the science of analytical reasoning facilitated by interactive visual interfaces, and it has been proven as an effective tool for high dimensional data analysis. In visual analytics systems, several visualizations are jointly analyzed in order to discover patterns in the data. One of the fundamental tools that has been integrated in visual analytics, is nonlinear dimensionality-reduction; a tool for the indirect visualization aimed at the discovery and analysis of non-linear patterns in the high-dimensional data. However, the computational complexity of non-linear dimensionality-reduction techniques does not allow direct employment in interactive systems. This limitation makes the analytic process a time-consuming task that can take hours, days or even weeks to be performed. In this thesis, we present novel algorithmic solutions that enable integration of non-linear dimensionality-reduction techniques in visual analytics systems. Our proposed algorithms are, not only much faster than existing solutions, but provide richer insights into the data at hand. This result, is achieved by introducing new data processing and optimization techniques and by embracing the recently introduced concept of Progressive Visual Analytics; a computational paradigm that enables the interactivity of complex analytics techniques by means of visualization as well as interaction with intermediate results. Moreover, we present several applications that are designed to provide unprecedented analytical capabilities in several domains. These applications are powered by the algorithms introduced in this dissertation and led to several discoveries in areas ranging from the biomedical research field, to social-network data analysis and machine-learning models interpretability.
Coronary artery disease is the most common type of heart disease, which influences 110 million people's health and causes 8.9 million deaths in 2015. Physicians can visualize the lesion in coronary arteries by cardiac angiography (X-ray image) during diagnosis and treatment of coronary artery disease. The pathological findings in cardiac angiography are reported per segment or per artery of the coronary artery tree, therefore, it requires to annotate the name of each segment or artery in the coronary artery tree.
This thesis proposes a data-driven method as a first attempt at annotating cardiac angiography based on deep learning. The method aims at automatically regressing segment points between different segments on the coronary artery tree as the annotation of the cardiac angiography. The proposed data-driven cardiac angiography annotation methods can learn and generalize from manually annotated cardiac angiography examples, but its performance is limited by the number and quality of examples for learning. ...
This thesis proposes a data-driven method as a first attempt at annotating cardiac angiography based on deep learning. The method aims at automatically regressing segment points between different segments on the coronary artery tree as the annotation of the cardiac angiography. The proposed data-driven cardiac angiography annotation methods can learn and generalize from manually annotated cardiac angiography examples, but its performance is limited by the number and quality of examples for learning. ...
Coronary artery disease is the most common type of heart disease, which influences 110 million people's health and causes 8.9 million deaths in 2015. Physicians can visualize the lesion in coronary arteries by cardiac angiography (X-ray image) during diagnosis and treatment of coronary artery disease. The pathological findings in cardiac angiography are reported per segment or per artery of the coronary artery tree, therefore, it requires to annotate the name of each segment or artery in the coronary artery tree.
This thesis proposes a data-driven method as a first attempt at annotating cardiac angiography based on deep learning. The method aims at automatically regressing segment points between different segments on the coronary artery tree as the annotation of the cardiac angiography. The proposed data-driven cardiac angiography annotation methods can learn and generalize from manually annotated cardiac angiography examples, but its performance is limited by the number and quality of examples for learning.
This thesis proposes a data-driven method as a first attempt at annotating cardiac angiography based on deep learning. The method aims at automatically regressing segment points between different segments on the coronary artery tree as the annotation of the cardiac angiography. The proposed data-driven cardiac angiography annotation methods can learn and generalize from manually annotated cardiac angiography examples, but its performance is limited by the number and quality of examples for learning.
Master thesis
(2018)
-
Dirk Schut, Theo van Walsum, Anna Vilanova Bartroli, Rob Remis, Marius Staring
Ablation is a medical procedure to treat liver cancer where a needle-like catheter has to be inserted into a tumor, which will then be heated or frozen to destroy the tumor tissue. To guide the catheter, Ultrasound(US) imaging is used which shows the catheter position in real time. However, some tumors are not visible on US images. To make these tumors visible, image fusion can be used between the inter-operative US image and a pre-operative contrast enhanced CT(CECT) scan, on which the tumors are visible. Several methods exist for tracking the motions of the US transducer relative to the CECT scan, but they all require a manual initialization or external tracking hardware to align the coordinate systems of both scans. In this thesis we present a technique for finding an initialization using only the image data. To achieve this, deep learning is used to segment liver vessels and the boundary of the liver in 3D US images. To find the rigid transformation parameters, the SaDE evolutionary algorithm was used to optimize the alignment between the blood vessels and the liver boundary between both scans.
...
Ablation is a medical procedure to treat liver cancer where a needle-like catheter has to be inserted into a tumor, which will then be heated or frozen to destroy the tumor tissue. To guide the catheter, Ultrasound(US) imaging is used which shows the catheter position in real time. However, some tumors are not visible on US images. To make these tumors visible, image fusion can be used between the inter-operative US image and a pre-operative contrast enhanced CT(CECT) scan, on which the tumors are visible. Several methods exist for tracking the motions of the US transducer relative to the CECT scan, but they all require a manual initialization or external tracking hardware to align the coordinate systems of both scans. In this thesis we present a technique for finding an initialization using only the image data. To achieve this, deep learning is used to segment liver vessels and the boundary of the liver in 3D US images. To find the rigid transformation parameters, the SaDE evolutionary algorithm was used to optimize the alignment between the blood vessels and the liver boundary between both scans.
2D/3D registration is the process of aligning 3D volume data to a 2D image. Typically, during orthopedic trauma surgeries, 3D volume data is acquired prior to the surgery to visualize the extent of the injury and also plan the operation. During the surgery, 2D fluoroscopic images are acquired to visualize the position of surgical instruments and understand the progress of the operation. However, the depth perception from these 2D images becomes difficult. By employing 2D/3D registration to overlay pre-operative imaging modalities, doctors may be presented a better way to visualize the progress of the surgery. Although several approaches have been proposed in the literature to deal with 2D/3D registration, they are limited by the need for a good initial alignment. This requirement for accurate initial alignment is the main reason hindering the adoption of 2D/3D registration in clinical practice. In this thesis, we propose automated initialization strategies, the output of which can be used by any common 2D/3D registration algorithm for
further fine tuning.
A library based approach is to used to study the initialization problem. Four parameters of the six dimensional parameter space are sampled and their projections are stored prior to the surgery. We proposed frequency and feature based methods to retrieve the correct library match which is used for the initialization during the surgery. The feasibility of the approach is demonstrated by initializing an intensity based 2D/3D registration method with the automatically obtained estimation of the initialization transformation parameters. For fixed C-arm cases with available ground truth the initialization was successful for 87:41% for various anatomies such as the head, pelvis, spine, knees and feet. We further tested the approach in a completely un-calibrated scenario using a mobile C-arm. Visual evaluation of the results revealed a success rate of 88:4% for the initialization. Successful results were also obtained even in the presence of additional surgical instruments indicating the robustness of the proposed approach. ...
further fine tuning.
A library based approach is to used to study the initialization problem. Four parameters of the six dimensional parameter space are sampled and their projections are stored prior to the surgery. We proposed frequency and feature based methods to retrieve the correct library match which is used for the initialization during the surgery. The feasibility of the approach is demonstrated by initializing an intensity based 2D/3D registration method with the automatically obtained estimation of the initialization transformation parameters. For fixed C-arm cases with available ground truth the initialization was successful for 87:41% for various anatomies such as the head, pelvis, spine, knees and feet. We further tested the approach in a completely un-calibrated scenario using a mobile C-arm. Visual evaluation of the results revealed a success rate of 88:4% for the initialization. Successful results were also obtained even in the presence of additional surgical instruments indicating the robustness of the proposed approach. ...
2D/3D registration is the process of aligning 3D volume data to a 2D image. Typically, during orthopedic trauma surgeries, 3D volume data is acquired prior to the surgery to visualize the extent of the injury and also plan the operation. During the surgery, 2D fluoroscopic images are acquired to visualize the position of surgical instruments and understand the progress of the operation. However, the depth perception from these 2D images becomes difficult. By employing 2D/3D registration to overlay pre-operative imaging modalities, doctors may be presented a better way to visualize the progress of the surgery. Although several approaches have been proposed in the literature to deal with 2D/3D registration, they are limited by the need for a good initial alignment. This requirement for accurate initial alignment is the main reason hindering the adoption of 2D/3D registration in clinical practice. In this thesis, we propose automated initialization strategies, the output of which can be used by any common 2D/3D registration algorithm for
further fine tuning.
A library based approach is to used to study the initialization problem. Four parameters of the six dimensional parameter space are sampled and their projections are stored prior to the surgery. We proposed frequency and feature based methods to retrieve the correct library match which is used for the initialization during the surgery. The feasibility of the approach is demonstrated by initializing an intensity based 2D/3D registration method with the automatically obtained estimation of the initialization transformation parameters. For fixed C-arm cases with available ground truth the initialization was successful for 87:41% for various anatomies such as the head, pelvis, spine, knees and feet. We further tested the approach in a completely un-calibrated scenario using a mobile C-arm. Visual evaluation of the results revealed a success rate of 88:4% for the initialization. Successful results were also obtained even in the presence of additional surgical instruments indicating the robustness of the proposed approach.
further fine tuning.
A library based approach is to used to study the initialization problem. Four parameters of the six dimensional parameter space are sampled and their projections are stored prior to the surgery. We proposed frequency and feature based methods to retrieve the correct library match which is used for the initialization during the surgery. The feasibility of the approach is demonstrated by initializing an intensity based 2D/3D registration method with the automatically obtained estimation of the initialization transformation parameters. For fixed C-arm cases with available ground truth the initialization was successful for 87:41% for various anatomies such as the head, pelvis, spine, knees and feet. We further tested the approach in a completely un-calibrated scenario using a mobile C-arm. Visual evaluation of the results revealed a success rate of 88:4% for the initialization. Successful results were also obtained even in the presence of additional surgical instruments indicating the robustness of the proposed approach.
Master thesis
(2017)
-
Laura Jacquemod, Anna Vilanova Bartroli, Binyam Gebre, Elmar Eisemann, David Tax
VolCam
Context-Aware Intuitive Touchless Interaction For Medical Volume Data
Master thesis
(2017)
-
Rustam Alashrafov, Elmar Eisemann, Anna Vilanova Bartroli, Ioannis Katramados
Touchless interaction has recently gained considerable attention by researchers as well as industry. Different domains are interested in implementing this technology in their solutions. Medical visualization has a special interest in this technology due to the sterile conditions in operating rooms. Exploration and detailed inspection of the scanned objects are among the most common interactions performed by professionals. These operations become more challenging when combined with touchless input. Context-aware methods exist, which facilitate navigation, but these methods are made for meshes and not for volume renderings. Hence the research question: Can these methods be extended to volume renderings and how well will they perform with touchless interaction metaphors? Metaphor and underlying VolCam algorithm are presented in this work. The metaphor allows users to perform exploration and inspection tasks on medical volume data using touchless input device - LeapMotion. The VolCam - an extension of the ShellCam algorithm, automatically maps the user input to distinct camera movements based on the current scene view by sampling the visible part of the volume. Interactive frame rates are achieved by performing computations on GPU. No pre-processing or specialized data structures are required which makes the technique directly applicable to wide-range of volume datasets.
...
Touchless interaction has recently gained considerable attention by researchers as well as industry. Different domains are interested in implementing this technology in their solutions. Medical visualization has a special interest in this technology due to the sterile conditions in operating rooms. Exploration and detailed inspection of the scanned objects are among the most common interactions performed by professionals. These operations become more challenging when combined with touchless input. Context-aware methods exist, which facilitate navigation, but these methods are made for meshes and not for volume renderings. Hence the research question: Can these methods be extended to volume renderings and how well will they perform with touchless interaction metaphors? Metaphor and underlying VolCam algorithm are presented in this work. The metaphor allows users to perform exploration and inspection tasks on medical volume data using touchless input device - LeapMotion. The VolCam - an extension of the ShellCam algorithm, automatically maps the user input to distinct camera movements based on the current scene view by sampling the visible part of the volume. Interactive frame rates are achieved by performing computations on GPU. No pre-processing or specialized data structures are required which makes the technique directly applicable to wide-range of volume datasets.
The explosion of video data in surveillance calls for large amount of annotated datasets that could be used for information retrieval, learning-based network training and algorithms evaluation phase. A number of annotated video
datasets have been shared to public, however, these open annotated datasets lack spatio-temporal information which can contribute to motion analysis researches. Moreover, manual annotation is tedious in lengthy frame sequences
where the numbers are overwhelming. In this work we propose a visualization solution to facilitate video data retrieval, mining and annotation. It works as an integrated visual analytics system which supports collecting moving object samples and studying motion patterns in video, with the facilitation of video data visualization, using various spatial and/or temporal features, filtering parameters and similarity measuring models. The annotation output can be
applied to multiple video analysis researches. In this paper, we present the system and propose a workflow for costeffective video annotation of spatio-temporal data as well as facilitating comprehension of video data. ...
datasets have been shared to public, however, these open annotated datasets lack spatio-temporal information which can contribute to motion analysis researches. Moreover, manual annotation is tedious in lengthy frame sequences
where the numbers are overwhelming. In this work we propose a visualization solution to facilitate video data retrieval, mining and annotation. It works as an integrated visual analytics system which supports collecting moving object samples and studying motion patterns in video, with the facilitation of video data visualization, using various spatial and/or temporal features, filtering parameters and similarity measuring models. The annotation output can be
applied to multiple video analysis researches. In this paper, we present the system and propose a workflow for costeffective video annotation of spatio-temporal data as well as facilitating comprehension of video data. ...
The explosion of video data in surveillance calls for large amount of annotated datasets that could be used for information retrieval, learning-based network training and algorithms evaluation phase. A number of annotated video
datasets have been shared to public, however, these open annotated datasets lack spatio-temporal information which can contribute to motion analysis researches. Moreover, manual annotation is tedious in lengthy frame sequences
where the numbers are overwhelming. In this work we propose a visualization solution to facilitate video data retrieval, mining and annotation. It works as an integrated visual analytics system which supports collecting moving object samples and studying motion patterns in video, with the facilitation of video data visualization, using various spatial and/or temporal features, filtering parameters and similarity measuring models. The annotation output can be
applied to multiple video analysis researches. In this paper, we present the system and propose a workflow for costeffective video annotation of spatio-temporal data as well as facilitating comprehension of video data.
datasets have been shared to public, however, these open annotated datasets lack spatio-temporal information which can contribute to motion analysis researches. Moreover, manual annotation is tedious in lengthy frame sequences
where the numbers are overwhelming. In this work we propose a visualization solution to facilitate video data retrieval, mining and annotation. It works as an integrated visual analytics system which supports collecting moving object samples and studying motion patterns in video, with the facilitation of video data visualization, using various spatial and/or temporal features, filtering parameters and similarity measuring models. The annotation output can be
applied to multiple video analysis researches. In this paper, we present the system and propose a workflow for costeffective video annotation of spatio-temporal data as well as facilitating comprehension of video data.