W.J. Niessen
Please Note
190 records found
1
Non-linear interactions among single nucleotide polymorphisms (SNPs), genes, and pathways play an important role in human diseases, but identifying these interactions is a challenging task. Neural networks are state-of-the-art predictors in many domains due to their ability to analyze big data and model complex patterns, including non-linear interactions. In genetics, visible neural networks are popular as they provide insight into the most important SNPs, genes, and pathways for prediction. Visible neural networks use prior knowledge (e.g., gene and pathway annotations) to define node connections in the network, making them sparse and interpretable. Currently, most of these networks provide measures for the importance of SNPs, genes, and pathways but do not provide information about interactions. In this paper, we explore different methods to detect non-linear interactions with visible neural networks. We adapt and speed up existing methods, create a comprehensive benchmark with simulated data from GAMETES and EpiGEN, and demonstrate that these methods can extract multiple types of interactions from trained neural networks. Finally, we apply these methods to a genome-wide case-control study of inflammatory bowel disease and find high consistency of the epistasis pairs candidates between interpretation methods. The follow-up association test on these candidates identifies seven significant epistasis pairs.
CAVE
Cerebral artery–vein segmentation in digital subtraction angiography
Cerebral X-ray digital subtraction angiography (DSA) is a widely used imaging technique in patients with neurovascular disease, allowing for vessel and flow visualization with high spatio-temporal resolution. Automatic artery–vein segmentation in DSA plays a fundamental role in vascular analysis with quantitative biomarker extraction, facilitating a wide range of clinical applications. The widely adopted U-Net applied on static DSA frames often struggles with disentangling vessels from subtraction artifacts. Further, it falls short in effectively separating arteries and veins as it disregards the temporal perspectives inherent in DSA. To address these limitations, we propose to simultaneously leverage spatial vasculature and temporal cerebral flow characteristics to segment arteries and veins in DSA. The proposed network, coined CAVE, encodes a 2D+time DSA series using spatial modules, aggregates all the features using temporal modules, and decodes it into 2D segmentation maps. On a large multi-center clinical dataset, CAVE achieves a vessel segmentation Dice of 0.84 (±0.04) and an artery–vein segmentation Dice of 0.79 (±0.06). CAVE surpasses traditional Frangi-based k-means clustering (P < 0.001) and U-Net (P < 0.001) by a significant margin, demonstrating the advantages of harvesting spatio-temporal features. This study represents the first investigation into automatic artery–vein segmentation in DSA using deep learning. The code is publicly available at https://github.com/RuishengSu/CAVE_DSA.
Rationale and Objectives: Distinguishing malignant from benign liver lesions based on magnetic resonance imaging (MRI) is an important but often challenging task, especially in noncirrhotic livers. We developed and externally validated a radiomics model to quantitatively assess T2-weighted MRI to distinguish the most common malignant and benign primary solid liver lesions in noncirrhotic livers. Materials and Methods: Data sets were retrospectively collected from three tertiary referral centers (A, B, and C) between 2002 and 2018. Patients with malignant (hepatocellular carcinoma and intrahepatic cholangiocarcinoma) and benign (hepatocellular adenoma and focal nodular hyperplasia) lesions were included. A radiomics model based on T2-weighted MRI was developed in data set A using a combination of machine learning approaches. The model was internally evaluated on data set A through cross-validation, externally validated on data sets B and C, and compared to visual scoring of two experienced abdominal radiologists on data set C. Results: The overall data set included 486 patients (A: 187, B: 98, and C: 201). The radiomics model had a mean area under the curve (AUC) of 0.78 upon internal validation on data set A and a similar AUC in external validation (B: 0.74 and C: 0.76). In data set C, the two radiologists showed moderate agreement (Cohen's κ: 0.61) and achieved AUCs of 0.86 and 0.82. Conclusion: Our T2-weighted MRI radiomics model shows potential for distinguishing malignant from benign primary solid liver lesions. External validation indicated that the model is generalizable despite substantial MRI acquisition protocol differences. Pending further optimization and generalization, this model may aid radiologists in improving the diagnostic workup of patients with liver lesions.
Changes in Cerebral Hemodynamics and Progression of Subclinical Vascular Brain Disease
A Population-Based Cohort Study
Cerebral hypoperfusion is associated with vascular brain injury and neurodegeneration, but their longitudinal relationship is largely unknown, especially in healthy older adults.
METHODS:
We investigated the longitudinal relationship between cerebral hemodynamics and subclinical vascular brain disease in community-dwelling older adults without stroke or dementia at baseline. Participants underwent brain magnetic resonance imaging scans every 3 to 4 years between 2005 and 2016. Cerebral blood flow (CBF) was measured through 2-dimensional phase-contrast magnetic resonance imaging; the cerebrovascular resistance index (CVRi) was defined as the ratio of mean arterial blood pressure to total CBF. Simultaneous progression in subclinical brain disease was evaluated through repeated magnetic resonance imaging assessment of white matter hyperintensities (WMH), cerebral microbleeds, lacune, and brain atrophy. The longitudinal relationship was estimated using generalized estimating equations, with adjustment for age, sex, smoking habits, body mass index, systolic blood pressure (for CBF measures), lipid level, history of diabetes and cardiovascular disease, and the baseline burden of magnetic resonance imaging markers.
RESULTS:
Among 3623 older adults (mean age, 61.4±9.3 years; 54.6% women), large decreases and increases in CBF and increases in CVRi over time were associated with white matter hyperintensity progression. The risk ratios for white matter hyperintensity progression were 1.36 (95% CI, 1.19–1.55) for large decreases in total CBF (lowest quartile), 1.02 (95% CI, 0.91–1.14) for moderate decreases (second quartile), and 1.28 (95% CI, 1.14–1.45) for large increases (highest quartile), compared with stable CBF (third quartile). The corresponding risk ratios for changes in CVRi were 1.13 (95% CI, 1.00–1.30), 1.25 (95% CI, 1.09–1.43), and 1.33 (95% CI, 1.16–1.52) for the second to fourth (versus lowest) quartiles, respectively, showing a dose-response relationship. The changes in CBF also demonstrate a similar U-shaped association with the progression of brain atrophy and incident microbleeds, whereas increases in CVRi were associated with lower microbleed risk.
CONCLUSIONS:
Longitudinal changes in CBF and CVRi may capture distinct pathophysiologies linking cerebral hemodynamics to subclinical brain disease, extending beyond single–time point measurements. ...
Cerebral hypoperfusion is associated with vascular brain injury and neurodegeneration, but their longitudinal relationship is largely unknown, especially in healthy older adults.
METHODS:
We investigated the longitudinal relationship between cerebral hemodynamics and subclinical vascular brain disease in community-dwelling older adults without stroke or dementia at baseline. Participants underwent brain magnetic resonance imaging scans every 3 to 4 years between 2005 and 2016. Cerebral blood flow (CBF) was measured through 2-dimensional phase-contrast magnetic resonance imaging; the cerebrovascular resistance index (CVRi) was defined as the ratio of mean arterial blood pressure to total CBF. Simultaneous progression in subclinical brain disease was evaluated through repeated magnetic resonance imaging assessment of white matter hyperintensities (WMH), cerebral microbleeds, lacune, and brain atrophy. The longitudinal relationship was estimated using generalized estimating equations, with adjustment for age, sex, smoking habits, body mass index, systolic blood pressure (for CBF measures), lipid level, history of diabetes and cardiovascular disease, and the baseline burden of magnetic resonance imaging markers.
RESULTS:
Among 3623 older adults (mean age, 61.4±9.3 years; 54.6% women), large decreases and increases in CBF and increases in CVRi over time were associated with white matter hyperintensity progression. The risk ratios for white matter hyperintensity progression were 1.36 (95% CI, 1.19–1.55) for large decreases in total CBF (lowest quartile), 1.02 (95% CI, 0.91–1.14) for moderate decreases (second quartile), and 1.28 (95% CI, 1.14–1.45) for large increases (highest quartile), compared with stable CBF (third quartile). The corresponding risk ratios for changes in CVRi were 1.13 (95% CI, 1.00–1.30), 1.25 (95% CI, 1.09–1.43), and 1.33 (95% CI, 1.16–1.52) for the second to fourth (versus lowest) quartiles, respectively, showing a dose-response relationship. The changes in CBF also demonstrate a similar U-shaped association with the progression of brain atrophy and incident microbleeds, whereas increases in CVRi were associated with lower microbleed risk.
CONCLUSIONS:
Longitudinal changes in CBF and CVRi may capture distinct pathophysiologies linking cerebral hemodynamics to subclinical brain disease, extending beyond single–time point measurements.
Integrating multi-omics data into predictive models has the potential to enhance accuracy, which is essential for precision medicine. In this study, we developed interpretable predictive models for multi-omics data by employing neural networks informed by prior biological knowledge, referred to as visible networks. These neural networks offer insights into the decision-making process and can unveil novel perspectives on the underlying biological mechanisms associated with traits and complex diseases. We tested the performance, interpretability and generalizability for inferring smoking status, subject age and LDL levels using genome-wide RNA expression and CpG methylation data from the blood of the BIOS consortium (four population cohorts, Ntotal = 2940). In a cohort-wise cross-validation setting, the consistency of the diagnostic performance and interpretation was assessed. Performance was consistently high for predicting smoking status with an overall mean AUC of 0.95 (95% CI: 0.90-1.00) and interpretation revealed the involvement of well-replicated genes such as AHRR, GPR15 and LRRN3. LDL-level predictions were only generalized in a single cohort with an R2 of 0.07 (95% CI: 0.05-0.08). Age was inferred with a mean error of 5.16 (95% CI: 3.97-6.35) years with the genes COL11A2, AFAP1, OTUD7A, PTPRN2, ADARB2 and CD34 consistently predictive. For both regression tasks, we found that using multi-omics networks improved performance, stability and generalizability compared to interpretable single omic networks. We believe that visible neural networks have great potential for multi-omics analysis; they combine multi-omic data elegantly, are interpretable, and generalize well to data from different cohorts.
Background & Aims: Impaired liver function affects brain health and therefore understanding potential mechanisms for subclinical liver disease is essential. We assessed the liver–brain associations using liver measures with brain imaging markers, and cognitive measures in the general population. Methods: Within the population-based Rotterdam Study, liver serum and imaging measures (ultrasound and transient elastography), metabolic dysfunction-associated fatty liver disease (MAFLD), non-alcoholic fatty liver disease (NAFLD) and fibrosis phenotypes, and brain structure were determined in 3493 non-demented and stroke-free participants in 2009–2014. This resulted in subgroups of n = 3493 for MAFLD (mean age 69 ± 9 years, 56% ♀), n = 2938 for NAFLD (mean age 70 ± 9 years, 56% ♀) and n = 2252 for fibrosis (mean age 65 ± 7 years, 54% ♀). Imaging markers of small vessel disease and neurodegeneration, cerebral blood flow (CBF) and brain perfusion (BP) were acquired from brain MRI (1.5-tesla). General cognitive function was assessed by Mini-Mental State Examination and the g-factor. Multiple linear and logistic regression models were used for liver-brain associations and adjusted for age, sex, intracranial volume, cardiovascular risk factors and alcohol use. Results: Higher gamma-glutamyltransferase (GGT) levels were significantly associated with smaller total brain volume (TBV, standardized mean difference (SMD), −0.02, 95% confidence interval (CI) (−0.03 to −0.01); p = 8.4·10−4), grey matter volumes, and lower CBF and BP. Liver serum measures were not related to small vessel disease markers, nor to white matter microstructural integrity or general cognition. Participants with ultrasound-based liver steatosis had a higher fractional anisotropy (FA, SMD 0.11, 95% CI (0.04 to 0.17), p = 1.5·10−3) and lower CBF and BP. MAFLD and NAFLD phenotypes were associated with alterations in white matter microstructural integrity (NAFLD ~ FA, SMD 0.14, 95% CI (0.07 to 0.22), p = 1.6·10−4; NAFLD ~ mean diffusivity, SMD −0.12, 95% CI (−0.18 to −0.05), p = 4.7·10−4) and also with lower CBF and BP (MAFLD ~ CBF, SMD −0.13, 95% CI (−0.20 to −0.06), p = 3.1·10−4; MAFLD ~ BP, SMD −0.12, 95% CI (−0.20 to −0.05), p = 1.6·10−3). Furthermore, fibrosis phenotypes were related to TBV, grey and white matter volumes. Conclusions: Presence of liver steatosis, fibrosis and elevated serum GGT are associated with structural and hemodynamic brain markers in a population-based cross-sectional setting. Understanding the hepatic role in brain changes can target modifiable factors and prevent brain dysfunction.
Sagittal Craniosynostosis
Comparing Surgical Techniques Using 3D Photogrammetry
Background: The aim of this study was to compare three surgical interventions for correction of sagittal synostosis-frontobiparietal remodeling (FBR), extended strip craniotomy (ESC), and spring-Assisted correction (SAC)-based on three-dimensional (3D) photogrammetry and operation characteristics. Methods: Patients who were born between 1991 and 2019 and diagnosed with nonsyndromic sagittal synostosis who underwent FBR, ESC, or SAC and had at least one postoperative 3D photogrammetry image taken during one of six follow-up appointments until age 6 were considered for this study. Operative characteristics, postoperative complications, reinterventions, and presence of intracranial hypertension were collected. To assess cranial growth, orthogonal cranial slices and 3D photocephalometric measurements were extracted automatically and evaluated from 3D photogrammetry images. Results: A total of 322 postoperative 3D images from 218 patients were included. After correcting for age and sex, no significant differences were observed in 3D photocephalometric measurements. Mean cranial shapes suggested that postoperative growth and shape gradually normalized with higher occipitofrontal head circumference and intracranial volume values compared with normal values, regardless of type of surgery. Flattening of the vertex seems to persist after surgical correction. The authors' cranial 3D mesh processing tool has been made publicly available as a part of this study. Conclusions: The findings suggest that until age 6, there are no significant differences among the FBR, ESC, and SAC in their ability to correct sagittal synostosis with regard to 3D photocephalometric measurements. Therefore, efforts should be made to ensure early diagnosis so that minimally invasive surgery is a viable treatment option. CLINICAL QUESTION/LEVEL OF EVIDENCE: Therapeutic, III.
BACKGROUND: Accurate characterization of glioma is crucial for clinical decision making. A delineation of the tumor is also desirable in the initial decision stages but is time-consuming. Previously, deep learning methods have been developed that can either non-invasively predict the genetic or histological features of glioma, or that can automatically delineate the tumor, but not both tasks at the same time. Here, we present our method that can predict the molecular subtype and grade, while simultaneously providing a delineation of the tumor. METHODS: We developed a single multi-task convolutional neural network that uses the full 3D, structural, preoperative MRI scans to predict the IDH mutation status, the 1p/19q co-deletion status, and the grade of a tumor, while simultaneously segmenting the tumor. We trained our method using a patient cohort containing 1508 glioma patients from 16 institutes. We tested our method on an independent dataset of 240 patients from 13 different institutes. RESULTS: In the independent test set, we achieved an IDH-AUC of 0.90, an 1p/19q co-deletion AUC of 0.85, and a grade AUC of 0.81 (grade II/III/IV). For the tumor delineation, we achieved a mean whole tumor Dice score of 0.84. CONCLUSIONS: We developed a method that non-invasively predicts multiple, clinically relevant features of glioma. Evaluation in an independent dataset shows that the method achieves a high performance and that it generalizes well to the broader clinical population. This first-of-its-kind method opens the door to more generalizable, instead of hyper-specialized, AI methods.
Computer-aided methods have shown added value for diagnosing and predicting brain disorders and can thus support decision making in clinical care and treatment planning. This chapter will provide insight into the type of methods, their working, their input data –such as cognitive tests, imaging, and genetic data– and the types of output they provide. We will focus on specific use cases for diagnosis, i.e., estimating the current “condition” of the patient, such as early detection and diagnosis of dementia, differential diagnosis of brain tumors, and decision making in stroke. Regarding prediction, i.e., estimation of the future “condition” of the patient, we will zoom in on use cases such as predicting the disease course in multiple sclerosis and predicting patient outcomes after treatment in brain cancer. Furthermore, based on these use cases, we will assess the current state-of-the-art methodology and highlight current efforts on benchmarking of these methods and the importance of open science therein. Finally, we assess the current clinical impact of computer-aided methods and discuss the required next steps to increase clinical impact.
Association between prenatal alcohol exposure and children's facial shape:
a prospective population-based cohort study
STUDY QUESTION: Is there an association between low-to-moderate levels of prenatal alcohol exposure (PAE) and children's facial shape? SUMMARY ANSWER: PAE before and during pregnancy, even at low level (<12 g of alcohol per week), was found associated with the facial shape of children, and these associations were found attenuated as children grow older. WHAT IS KNOWN ALREADY: High levels of PAE during pregnancy can have significant adverse associations with a child's health development resulting in recognizably abnormal facial development. STUDY DESIGN, SIZE, DURATION: This study was based on the Generation R Study, a prospective cohort from fetal life onwards with maternal and offspring data. We analyzed children 3-dimensional (3D) facial images taken at ages 9 (n = 3149) and 13 years (n = 2477) together with the data of maternal alcohol consumption. PARTICIPANTS/MATERIALS, SETTING, METHODS: We defined six levels of PAE based on the frequency and dose of alcohol consumption and defined three tiers based on the timing of alcohol exposure of the unborn child. For the image analysis, we used 3D graph convolutional networks for non-linear dimensionality reduction, which compressed the high-dimensional images into 200 traits representing facial morphology. These 200 traits were used for statistical analysis to search for associations with PAE. Finally, we generated heatmaps to display the facial phenotypes associated with PAE. MAIN RESULTS AND THE ROLE OF CHANCE: The results of the linear regression in the 9-year-old children survived correction for multiple testing with false discovery rate (FDR). In Tier 1 where we examined PAE only before pregnancy (exposed N = 278, unexposed N = 760), we found three traits survived FDR correction. The lowest FDR-P is 1.7e-05 (beta = 0.021, SE = 0.0040) in Trait #29; In Tier 2b where we examine any PAE during first trimester (exposed N = 756; unexposed N = 760), we found eight traits survived FDR correction. The lowest FDR-P is 9.0e-03 (beta = -0.013, SE = 0.0033) in Trait #139. Moreover, more statistically significant facial traits were found in higher levels of PAE. No FDR-significant results were found in the 13-year-old children. We map these significant traits back to the face, and found the most common detected facial phenotypes included turned-up nose tip, shortened nose, turned-out chin, and turned-in lower-eyelid-related regions. LIMITATIONS, REASONS FOR CAUTION: We had no data for alcohol consumption more than three months prior to pregnancy and thus do not know if maternal drinking had chronic effects. The self-reported questionnaire might not reflect accurate alcohol measurements because mothers may have denied their alcohol consumption. WIDER IMPLICATIONS OF THE FINDINGS: Our results imply that facial morphology, such as quantified by the approach we proposed here, can be used as a biomarker in further investigations. Furthermore, our study suggests that for women who are pregnant or want to become pregnant soon, should quit alcohol consumption several months before conception and completely during pregnancy to avoid adverse health outcomes in the offspring. STUDY FUNDING/COMPETING INTEREST(S): This work was supported by Erasmus Medical Centre, Rotterdam, the Erasmus University Rotterdam, and the Netherlands Organization for Health Research. V.W.V.J. reports receipt of funding from the Netherlands Organization for Health Research (ZonMw 90700303). W.J.N. is a founder, a scientific lead, and a shareholder of Quantib BV. TRIAL REGISTRATION NUMBER: N/A.
Identification and detection of thin-cap fibroatheroma (TCFA) from intravascular optical coherence tomography (IVOCT) images is critical for treatment of coronary heart diseases. Recently, deep learning methods have shown promising successes in TCFA identification. However, most methods usually do not effectively utilize multi-view information or incorporate prior domain knowledge. In this paper, we propose a multi-view contour-constrained transformer network (MVCTN) for TCFA identification in IVOCT images. Inspired by the diagnosis process of cardiologists, we use contour constrained self-attention modules (CCSM) to emphasize features corresponding to salient regions (i.e., vessel walls) in an unsupervised manner and enhance the visual interpretability based on class activation mapping (CAM). Moreover, we exploit transformer modules (TM) to build global-range relations between two views (i.e., polar and Cartesian views) to effectively fuse features at multiple feature scales. Experimental results on a semi-public dataset and an in-house dataset demonstrate that the proposed MVCTN outperforms other single-view and multi-view methods. Lastly, the proposed MVCTN can also provide meaningful visualization for cardiologists via CAM.
Augmented reality (AR) has shown potential in computer-aided surgery. It allows for the visualization of hidden anatomical structures as well as assists in navigating and locating surgical instruments at the surgical site. Various modalities (devices and/or visualizations) have been used in the literature, but few studies investigated the adequacy/superiority of one modality over the other. For instance, the use of optical see-through (OST) HMDs has not always been scientifically justified. Our goal is to compare various visualization modalities for catheter insertion in external ventricular drain and ventricular shunt procedures. We investigate two AR approaches: (1) 2D approaches consisting of a smartphone and a 2D window visualized through an OST (Microsoft HoloLens 2), and (2) 3D approaches consisting of a fully aligned patient model and a model that is adjacent to the patient and is rotationally aligned using an OST. 32 participants joined this study. For each visualization approach, participants were asked to perform five insertions after which they filled NASA-TLX and SUS forms. Moreover, the position and orientation of the needle with respect to the planning during the insertion task were collected. The results show that participants achieved a better insertion performance significantly under 3D visualizations, and the NASA-TLX and SUS forms reflected the preference of participants for these approaches compared to 2D approaches.
Towards quantitative digital subtraction perfusion angiography
An animal study
Background: X-ray digital subtraction angiography (DSA) is the imaging modality for peri-procedural guidance and treatment evaluation in (neuro-) vascular interventions. Perfusion image construction from DSA, as a means of quantitatively depicting cerebral hemodynamics, has been shown feasible. However, the quantitative property of perfusion DSA has not been well studied. Purpose: To comparatively study the independence of deconvolution-based perfusion DSA with respect to varying injection protocols, as well as its sensitivity to alterations in brain conditions. Methods: We developed a deconvolution-based algorithm to compute perfusion parametric images from DSA, including cerebral blood volume (CBV (Figure presented.)), cerebral blood flow (CBF (Figure presented.)), time to maximum (Tmax), and mean transit time (MTT (Figure presented.)) and applied it to DSA sequences obtained from two swine models. We also extracted the time intensity curve (TIC)-derived parameters, that is, area under the curve (AUC), peak concentration of the curve, and the time to peak (TTP) from these sequences. Deconvolution-based parameters were quantitatively compared to TIC-derived parameters in terms of consistency upon variations in injection profile and time resolution of DSA, as well as sensitivity to alterations of cerebral condition. Results: Comparing to TIC-derived parameters, the standard deviation (SD) of deconvolution-based parameters (normalized with respect to the mean) are two to five times smaller, indicating that they are more consistent across different injection protocols and time resolutions. Upon ischemic stroke induced in a swine model, the sensitivities of deconvolution-based parameters are equal to, if not higher than, those of TIC-derived parameters. Conclusions: In comparison to TIC-derived parameters, deconvolution-based perfusion imaging in DSA shows significantly higher quantitative reliability against variations in injection protocols across different time resolutions, and is sensitive to alterations in cerebral hemodynamics. The quantitative nature of perfusion angiography may allow for objective treatment assessment in neurovascular interventions.
Extracting the cerebral anterior vessel tree of patients with an intracranial large vessel occlusion (LVO) is relevant to investigate potential biomarkers that can contribute to treatment decision making. The purpose of our work is to develop a method that can achieve this from routinely acquired computed tomography angiography (CTA) and computed tomography perfusion (CTP) images. To this end, we regard the anterior vessel tree as a set of bifurcations and connected centerlines. The method consists of a proximal policy optimization (PPO) based deep reinforcement learning (DRL) approach for tracking centerlines, a convolutional neural network based bifurcation detector, and a breadth-first vessel tree construction approach taking the tracking and bifurcation detection results as input. We experimentally determine the added values of various components of the tracker. Both DRL vessel tracking and CNN bifurcation detection were assessed in a cross validation experiment using 115 subjects. The anterior vessel tree formation was evaluated on an independent test set of 25 subjects, and compared to interobserver variation on a small subset of images. The DRL tracking result achieves a median overlapping rate until the first error (1.8 mm off the reference standard) of 100, [46, 100] % on 8032 vessels over 115 subjects. The bifurcation detector reaches an average recall and precision of 76% and 87% respectively during the vessel tree formation process. The final vessel tree formation achieves a median recall of 68% and precision of 70%, which is in line with the interobserver agreement.
Intracranial vessel perforation is a peri-procedural complication during endovascular therapy (EVT). Prompt recognition is important as its occurrence is strongly associated with unfavorable treatment outcomes. However, perforations can be hard to detect because they are rare, can be subtle, and the interventionalist is working under time pressure and focused on treatment of vessel occlusions. Automatic detection holds potential to improve rapid identification of intracranial vessel perforation. In this work, we present the first study on automated perforation detection and localization on X-ray digital subtraction angiography (DSA) image series. We adapt several state-of-the-art single-frame detectors and further propose temporal modules to learn the progressive dynamics of contrast extravasation. Application-tailored loss function and post-processing techniques are designed. We train and validate various automated methods using two national multi-center datasets (i.e., MR CLEAN Registry and MR CLEAN-NoIV Trial), and one international multi-trial dataset (i.e., the HERMES collaboration). With ten-fold cross-validation, the proposed methods achieve an area under the curve (AUC) of the receiver operating characteristic of 0.93 in terms of series level perforation classification. Perforation localization precision and recall reach 0.83 and 0.70 respectively. Furthermore, we demonstrate that the proposed automatic solutions perform at similar level as an expert radiologist.
Optical-based navigation systems are widely used in surgical interventions. However, despite their great utility and accuracy, they are expensive and require time and effort to setup for surgeries. Moreover, traditional navigation systems use 2D screens to display instrument positions causing the surgeons to look away from the operative field. Head mounted displays such as the Microsoft HoloLens may provide an attractive alternative for surgical navigation that also permits augmented reality visualization. The HoloLens is equipped with multiple sensors for tracking and scene understanding. Mono and stereo-vision in the HoloLens have been both reported to be used for marker tracking, but no extensive evaluation on accuracy has been done to compare the two approaches. The objective of our work is to investigate the tracking performance of various camera setups in the HoloLens, and to study the effect of the marker size, marker distance from camera, and camera resolution on marker locating accuracy. We also investigate the speed and stability of marker pose for each camera setup. The tracking approaches are evaluated using ArUco markers. Our results show that mono-vision is more accurate in marker locating than stereo-vision when high resolution is used. However, this comes at the expense of higher frame processing time. Alternatively, we propose a combined low-resolution mono-stereo tracking setup that outperforms each tracking approach individually and is comparable to high resolution mono tracking, with a mean translational error of 1.8 ± 0.6mm for 10cm marker size at 50cm distance. We further discuss our findings and their implications for navigation in surgical interventions.
Treatment planning of gastrointestinal stromal tumors (GISTs) includes distinguishing GISTs from other intra-abdominal tumors and GISTs’ molecular analysis. The aim of this study was to evaluate radiomics for distinguishing GISTs from other intra-abdominal tumors, and in GISTs, predict the c-KIT, PDGFRA, BRAF mutational status, and mitotic index (MI). Patients diagnosed at the Erasmus MC between 2004 and 2017, with GIST or non-GIST intra-abdominal tumors and a contrast-enhanced venous-phase CT, were retrospectively included. Tumors were segmented, from which 564 image features were extracted. Prediction models were constructed using a combination of machine learning approaches. The evaluation was performed in a 100 × random-split cross-validation. Model performance was compared to that of three radiologists. One hundred twenty-five GISTs and 122 non-GISTs were included. The GIST vs. non-GIST radiomics model had a mean area under the curve (AUC) of 0.77. Three radiologists had an AUC of 0.69, 0.76, and 0.84, respectively. The radiomics model had an AUC of 0.52 for c-KIT, 0.56 for c-KIT exon 11, and 0.52 for the MI. The numbers of PDGFRA, BRAF, and other c-KIT mutations were too low for analysis. Our radiomics model was able to distinguish GISTs from non-GISTs with a performance similar to three radiologists, but less observer dependent. Therefore, it may aid in the early diagnosis of GIST, facilitating rapid referral to specialized treatment centers. As the model was not able to predict any genetic or molecular features, it cannot aid in treatment planning yet.
Augmented reality (AR) permits the visualization of pre-operative data in the surgical field of view of the surgeon. This requires the alignment of the AR device’s coordinate system with the used navigation/tracking system. We propose a multimodal marker approach to align an AR device with a tracking system: in our implementation, an electromagnetic tracking system (EMTS). The solution makes use of a calibration method which determines the relationship between a 2D pattern detected by an RGB camera and an electromagnetic sensor of the EMTS. This allowed the projection of a 3D skull model on its physical counterpart. This projection was evaluated using a monocular camera and an optical see-through device (HoloLens 2) (https://www.microsoft.com/en-us/hololens/) achieving an accuracy of less than 2.5 mm in the image plane of the HoloLens 2 (HL2). Additionally, 10 volunteers participated in a user study consisting of an alignment task of a pointer with 25 projections viewed through the HL2. The participants achieved a mean error of 2.7 1.3 mm and 2.9 2.9∘ in positional and orientation error. This study showcases the feasibility of the approach, provides an evaluation of the alignment, and finally, discusses its advantages and limitations.
Purpose: The assessment of collateral status may depend on the timing of image acquisition. The purpose of this study is to investigate whether there are optimal time points in CT Perfusion (CTP) for collateral status assessment, and compare collaterals scores at these time points with collateral scores from multiphase CT angiography (mCTA). Methods: Patients with an acute intracranial occlusion who underwent baseline non-contrast CT, mCTA and CT perfusion were selected. Collateral status was assessed using an automatically computed Collateral Ratio (CR) score in mCTA, and predefined time points in CTP acquisition. CRs extracted from CTP were correlated with CRs from mCTA. In addition, all CRs were related to baseline National Institutes of Health Stroke Scale (NIHSS) and Alberta Stoke Program Early CT Score (ASPECTS) with linear regression analysis to find the optimal CR. Results: In total 58 subjects (median age 74 years; interquartile range 61–83 years; 33 male) were included. When comparing the CRs from the CTP vs. mCTA acquisition, the strongest correlations were found between CR from baseline mCTA and the CR at the maximal intensity projection of time-resolved CTP (r = 0.81) and the CR at the peak of arterial enhancement point (r = 0.78). Baseline mCTA-derived CR had the highest correlation with ASPECTS (β = 0.36 (95%CI 0.11, 0.61)) and NIHSS (β = − 0.48 (95%CI − 0.72, − 0.16)). Conclusion: Collateral status assessment strongly depends on the timing of acquisition. Collateral scores obtained from mCTA imaging is close to the optimal collateral score obtained from CTP imaging.