The complexity and interdependency of hospital activities make it challenging to seamlessly integrate new technologies into clinical routines, especially with various stakeholders involved. Considering contextual factors when implementing innovations has been recognized as critical to success. This highlights the significance of an approach that focuses on integrating technology with the existing healthcare culture and motivating the context to change, rather than focusing on technology alone. Therefore, the aim of this thesis is to provide a method to ensure the clinical context is considered when developing a new technology. This approach is put in practice by utilizing it to introduce augmented reality (AR) at the oral and maxillofacial surgery (OMFS) department of the University Medical Center (UMC) Utrecht, to provide new insights on how to better align the technology with clinical needs and to familiarize and motivate the medical team with the technology. This is done in close collaboration with the UMC Groningen. Currently, the UMC Groningen, UMC Utrecht, and the Center for Information Technology (CIT) of the University of Groningen are developing an AR surgical navigation system. The aim of the introduction is to provide valuable information for the further development of the system and to prepare and involve the clinical environment. The approach used in this thesis is based on human-centered design and design-thinking. Furthermore, key factors for successful adoption found from previous research are considered. The approach consists of four main components. The first is to assign a med-tech innovation coordinator; someone with both a medical and technical background who will be responsible for driving adoption. Furthermore, it is an iterative three-step process: 1) Gain understanding; 2) Share understanding; 3) Actively engage stakeholders. The application of this approach to introduce AR at the OMFS department of the UMC Utrecht has resulted in multiple positive effects. In clinical context it resulted in an understanding of the technology and its value, motivations to invest time in the system, and the establishment of a shared future perspective on the implementation of AR. Furthermore, valuable input from a clinical perspective is obtained, including confirmation of valuable system elements and new ideas/insights for the further development of the AR navigation system. In conclusion, this thesis provides guidance for departments seeking to adopt new technologies based on a practical example. The methodology provides activities that facilitate the early introduction of technologies, ensuring the active involvement of the clinical context.

Objective: The two-fold aim of this phantom study was first to evaluate the accuracy of pulmonary nodule detection for lung cancer screening at progressively lower dose levels of photon-counting computed tomography (PC-CT) and, second, to objectively compare the image quality across different acquisition and reconstruction settings, through a comparison of PC-CT and conventional energy-integrating detector (EID) CT. Methods: Thirty-six artificial lung nodules with 6 diameters (2.5, 3, 4, 5, 6 and 10mm), 3 shapes (spherical, lobulated, spiculated) and two densities (-300 HU and +100 HU) were placed in an anthropomorphic chest phantom. The phantom was scanned using a standard lung cancer screening protocol (Sn 100 kV) with dose-matched EID-CT and PC-CT (CTDIvol of 0.8 mGy), in addition to 75%, 50%, 25% and 10% doses on the PC-CT (CTDIvol 0.6-0.07mGy). Nodule detection was performed by one experienced reader, and denoted as the sensitivity, specificity, precision and false positive (FP) nodules. The acquisitions for assessing image quality were performed on a Quality Assurance (QA) phantom at full dose, with varying levels of iterative reconstruction (IR), virtual monoenergetic image (VMI) keV levels, slice thicknesses and increments, kernel strengths and scan modes. The noise power spectrum (NPS) and the task-based transfer function (TTF) were computed. The detectability index (d’) was computed to model the detection of a 4 mm solid and 5 mm subsolid pulmonary nodule. Results: Sensitivity of 51-62% and specificity of 50-95% were obtained at 100%, 75%, 50%, 25% and 10% doses on the PC-CT, respectively, compared to the 59% sensitivity and 100% specificity on the EID-CT. The precision was 100% on the EID-CT and 94-99% on the PC-CT down to 25% dose, whereas at 10% dose the precision dropped to 88% and the FP nodules tripled from 5 to 15. Increasing the IR level, VMI keV and slice thickness decreased noise magnitude, with only minimal changes in noise texture. In contrast, higher kernel strength increased the noise magnitude, but created a finer noise texture. All settings showed a minimal impact on the spatial resolution. Noise magnitude was 40-60% higher on the EID-CT compared to PC-CT for all scan modes, although no difference in spatial resolution was found. Furthermore, the PC-CT attained 30-50% higher d’ values independent of scanmode or tin filter, in comparison to the EID-CT at a similar dose. Conclusion: PC-CT with similar acquisition and reconstruction settings demonstrated comparable sensitivity for lung nodule detection despite lower radiation dose, when compared with EID-CT for low-dose lung cancer CT screening. However, attention has to be paid to FP findings at ultra-low dose levels, since the specificity decreased with every dose reduction. Furthermore, PC-CT achieved higher d’ values compared to EID-CT at an equivalent dose, through enhanced image quality and less noise. Additionally, the study shows that optimizing the image acquisition and reconstruction parameters can further enhance the image quality of PC-CT. These findings show that PC-CT holds significant promise as an alternative method for low-dose lung cancer screening. Consequently, future research should focus on evaluating the performance of PC-CT further on a more representative anthropomorphic phantom and in a clinical setting.

The efficacy of endovascular therapy in large vessel occlusion (LVO) of the anterior circulation is dependent to a high degree on the selection of patients who are likely to benefit from this procedure. To this end, functional outcome prediction based on clinical parameters is an active area of research. In the preoperative screening of LVO patients, CT-Angiography (CTA) imaging is commonly acquired. We compare the functional outcome prediction performance of multiple deep learning based classifiers with multiple conventional methods, including the clinically validated MR PREDICTS decision tool. Using a dataset composed of 1929 preprocedural CTA images combined with clinical data, we compare a clinical baseline model with an imaging based pipeline and a combined pipeline. For the imaging model backbone we train various state-of-the-art architectures (Med3D, Vision Transformer, Voxel Transformer). These models are used to predict dichotomized modified Rankin Scale score 90 days after mechanical thrombectomy. Binary classifier outcomes are quantified using Area-Under the receiver operating characteristic Curve (AUC). The activation maps of the best performing image based model are further investigated using the GradCAM++ posthoc visualization method. Combining clinical features with information extracted from CTA images does not significantly improve the performance of functional outcome prediction methods compared to the baseline model. The information extracted from the images does not seem to be complementary to the clinical features. This multimodal technique can however replace radiologically derived biomarkers, as its performance is non-inferior.

Introduction The current gold standard for preoperative planning of deep inferior epigastric artery perforator (DIEAP) flap breast reconstruction is assessment of perforator vessels on computed tomography angiography (CTA) or magnetic resonance angiography (MRA) images in two-dimensional (2D) planes on a 2D screen. This research investigates the potential benefit of a 3D model of blood vessels on a 2D screen or Virtual Reality (VR) environment in preoperative planning for DIEAP flap breast reconstruction, compared to conventional CTA on a 2D screen. Materials & Methods A qualitative study was conducted among six reconstructive plastic surgeons and four residents. Participants were asked to select the ideal perforator for unilateral DIEAP flap reconstruction in three viewing environments: conventional CTA in a 2D view, a 3D model in a 2D view and a 3D model in an immersive virtual reality (VR) view. Subsequently, a questionnaire on anatomical structure visibility assessing six perforator characteristics (perforator location in the flap, calibre, intramuscular course through the rectus abdominis, perforator origin, subcutaneous branching and connection to other vessels) was completed. Second, an adjusted Usefulness, Satisfaction and Ease of Use (USE) questionnaire on system usability was filled out.Results Ten participants completed the experiments. The overall score distribution for perforator visibility characteristics was equal or higher except for perforator calibre in both 3D model views compared to conventional CTA. In the USE questionnaire, the 3D model in 2D view reported the highest scores for satisfaction and ease of learning. 90% of participants considered a 3D model in VR view to be of added value to preoperative planning, 60% considered a 3D model in 2D to be of added value. Overall, 40% would currently opt for the 3D model in a 2D view and 10% for the 3D model in a VR view. Conclusion This study presents a perspective on the application of 3D models for preoperative planning in DIEAP flap breast reconstructions. There is an interest in use of a 3D model in a 2D view or an immersive VR environment to optimize preoperative planning. The visibility of important perforator characteristics received overall higher scores for the viewing environments with a 3D model, highlighting the potential benefit for 3D models and VR application for selection of ideal perforators in preoperative planning for DIEAP flap breast reconstructions. 

Introduction: Breast reconstruction after a mastectomy is crucial to improve a patient’s quality of life. The Deep Inferior Epigastric artery Perforator (DIEP) flap procedure is considered the golden standard for breast reconstruction. Three-dimensional (3D) visualization methods have shown promise in providing a better understanding of Computed Tomography Angiography (CTA) information and potentially reducing operative time. However, for these methods manual segmentation of the perforators is needed, which is time-consuming and prone to variability. Automated segmentation using Deep learning (DL) can potentially overcome these limitations and provide accurate and efficient segmentation of the DIEP flap perforators. This study aimed to evaluate the application of DL for automated perforator segmentation in DIEP flap breast reconstruction, improving the efficiency and objectivity of DIEP flap perforator segmentation. Materials and methods: The dataset comprised 25 CTA scans for training and validation, and 5 CTA scans for testing. DL was employed for automated segmentation, and quantitative evaluation included metrics such as Dice coefficient score, recall, precision, surface distance, and centerline overlap. The qualitative evaluation involved grading the clinical acceptability of segmentations by four experienced plastic reconstructive surgeons. Results: On the training set, a Dice score of 0.58 (± 0.08) and a true positive centerline overlap of 0.66 (± 0.10) were achieved for perforator segmentation. The DL model successfully segmented the intramuscular main branch, but some perforators were missed in the subcutaneous fat tissue. Combined grading by all surgeons showed no statistical difference between manual and automated segmentations and both segmentations were evaluated as clinically acceptable. Conclusion: Automated DL segmentation holds promise for enhancing the efficiency and objectivity of identifying DIEP flap perforators in CTA images, providing an alternative to manual segmentation. Nonetheless, further research is needed to refine the automated segmentation results and to validate the generalizability and clinical applicability of the DL segmentation approach in larger patient cohorts and different clinical settings.

Purpose. Distal radius fractures are common fractures of the wrist. These fractures are often displaced and need reduction, after adequate reduction, the patients will have follow-up X-rays to check if the fracture stays stable. This is important because surgery might be required if the fracture becomes unstable. This can lead to delayed surgery which can worsen the treatment outcome. Therefore, it would be valuable to predict which distal radius fractures are likely to become unstable. Machine learning could help predict the stability of distal radius fractures based on CT. In addition, because there is only a small dataset available, this research also studies the effect of different machine learning methods. Method. Two different methods were evaluated for the stability prediction of distal radius fractures: traditional machine learning (radiomics method) and a residual network with and without transfer learning (deep learning method). For the radiomics method, a python package called WORC was used, which automatically extracts radiomic features and optimizes machine learning models. For the deep learning method, the backbone of a residual network called Med3D with added layers for classification was used. Results. The radiomics method combined with augmentation, gave the best results (AUC: 0.64± 0.01). Also, it was found that using an augmented data set in the radiomics method resulted in improved performance. This gives a slight indication that the radiomic method can learn to predict DRF when there would be more data available. Conclusion. The radiomic method is the most promising method for predicting the stability of distal radius fractures, despite the small difference in performance compared to random guessing and the deep learning method. However, for further research, it is highly recommended to acquire a larger dataset.

The thesis is composed of two parts. The first part is the scientific article that describes the main findings for our objective to align cerebral DSA series. The second part of the thesis includes more extensive information regarding the clinical and technical background of the project. Additionally, it contains information on deep-learning

Ablation is a medical procedure to treat liver cancer where a needle-like catheter has to be inserted into a tumor, which will then be heated or frozen to destroy the tumor tissue. To guide the catheter, Ultrasound(US) imaging is used which shows the catheter position in real time. However, some tumors are not visible on US images. To make these tumors visible, image fusion can be used between the inter-operative US image and a pre-operative contrast enhanced CT(CECT) scan, on which the tumors are visible. Several methods exist for tracking the motions of the US transducer relative to the CECT scan, but they all require a manual initialization or external tracking hardware to align the coordinate systems of both scans. In this thesis we present a technique for finding an initialization using only the image data. To achieve this, deep learning is used to segment liver vessels and the boundary of the liver in 3D US images. To find the rigid transformation parameters, the SaDE evolutionary algorithm was used to optimize the alignment between the blood vessels and the liver boundary between both scans.

Percutaneous coronary intervention is a minimally-invasive procedure to treat coronary artery disease. In such procedures, X-ray angiography, a real-time imaging technique, is commonly used for image guidance to identify lesion sites and navigate catheters and guide-wires within coronary arteries. Due to the physical nature of X-ray imaging, photon energy undergoes absorption when penetrating tissues, rendering a 2D projection image of a 3D scene, in which semi-transparent structures overlap with each other. The overlapping structures make robust information processing of X-ray images challenging. To tackle this issue, layer separation techniques for X-ray images were proposed to separate those structures into different image layers based on structure appearance or motion information. These techniques have been proven effective for vessel enhancement in X-ray angiograms. However, layer separation approaches still suffer either from non-robust separation or long processing time, which prevent their application in clinics. The purposes of this work are to investigate whether vessel layer separation from X-ray angiography images is possible via deep learning methods and further to what extent vessel layer separation can be achieved with deep learning methods. To this end, several deep learning based methods were developed and evaluated to extract the vessel layer. In particular, all the proposed methods utilize a fully convolutional network (FCN) with two different architectures (Appendix A and Chapter 2), which was trained by two different strategies: conventional losses (Appendix A and L1 method in Chapter 2) and an adversarial loss (AN +L1 method in Chapter 2). The results of all the methods show good vessel layer separation on 42 clinical sequences. Compared to the previous state-of-the-art, the proposed methods have similar performance but runs much faster, which makes it a potential real-time clinical application. Both the L1 method and AN + L1 method in Chapter 2 achieve better background than the method proposed in Appendix A, which can remove catheter and other tubular structures well. On the other hand, the L1 method results in better contrast and clearer backgrounds than the AN+L1 method.

Wrong-level spine surgery (WLS) is a medical error with severe consequences for the patient and medical staff. The current golden standard to prevent WLS during spinal surgery is fluoroscopy images acquired by a C-arm. However, this approach uses ionising radiation and interrupts the standard surgical workflow. To overcome these disadvantages, ultrasound-based navigation may provide an alternative to prevent WLS during spinal surgery. Baka et al. developed the Lumbar Localiser for ultrasound-based level localisation; however, this has not yet been applied in clinical practice. This project aims to improve and evaluate this level localisation approach in clinical practice. Chapter 1 highlights the social relevance of ultrasound-based level localisation. Chapter 2 shows in a literature review that navigated ultrasound-based guidance is not yet implemented in clinical practice. Identified recommendations for further research were optimising ease of use, workflow integration, registration accuracy and computation time. Chapter 3 describes a prospective multi-centre study (n=34) which evaluated the accuracy of the level localisation approach and the added value of intraoperative ultrasound acquisition. The accuracy of the improved level localisation approach was too low (53%) to recommend implementation in clinical practice. We showed that pre- and intraoperative ultrasound acquisitions could be integrated efficiently into the operation room (OR) workflow, with clinically negligible time differences compared to the current approach for level localisation. Intraoperative ultrasound acquisition added valuable information, enabling a re-check of the level localisation without the attenuating subcutaneous fat layer between the SP and ultrasound transducer. Chapter 4 describes the development of an automated contour segmentation approach based on magnetic resonance imaging (MRI) using deep learning. A nnU-Net was trained to segment the lumbar spinous process automatically. Subsequently, the posterior contours were extracted based on these segmentations and imported into the Lumbar Localiser. The automatic contours showed a successful matching in all 16 test cases. Chapter 5 provides an overall conclusion and discussion. In this master’s thesis, the technical aspects and clinical workflow of the Lumbar Localiser were improved to get closer to the clinical application of ultrasound-based level localisation for lumbar surgery. Implemented improvements include adding intraoperative ultrasound acquisition, extension to MRI as a preoperative modality and automatisation of the contour annotation. In conclusion, this approach cannot be applied yet in clinical practice, as an accuracy of around 100% should be achieved to minimise the risk of WLS and thereby prevent irreversible damage to the patient.

A craniotomy is a procedure were a neurosurgeon has to open the skull to gain direct access to the brain. When a brain tumor has to be removed from a patient, the craniotomy position is of great importance. This mostly defines the access path from the skull surface to the tumor and thus also what healthy brain tissue will be removed to access to the tumor itself. To minimize the amount of important brain structures that are removed, the craniotomy has to be carefully planned. This is a complex procedure, where a neurosurgeon is required to mentally reconstruct spatial relations of important brain structures to avoid these as much as possible. We propose a visualization using augmented reality which may assist in the planning of a craniotomy. In this visualization the goal is to show important brain structures aligned with the physical position of the patient. This should allow better perception of the spatial relations of these structures and thus assist the neurosurgeon. Additionally to the visualization of the structures, we created a heat map that is projected on top of the skull. This should give a quick overview of in which areas there are many important structures between the tumor and the skull surface, and should therefore be avoided. User studies were conducted amongst neurosurgeons and surgeons from other fields to evaluate the proposed visualization. We found that many of the participants indeed thought that the visualization can assist in surgery. For the specific case of craniotomy planning, several improvements have to be made on the heat map before it can be useful. Nevertheless, the visualization of the structures in itself can assist neurosurgeons in the planning of a craniotomy. Although more work has be performed at practical aspects of the visualization to make it ready for clinical experiments.

Purpose: Image-guided surgery can decrease perioperative complication incidences. Augmented Reality (AR) with head-mounted displays (HMDs) provides an accessible method of visualizing 3D preoperative models intraoperatively. This could provide the surgeon with an easy to use intraoperative image-guided surgery system. Image-to-patient registration, the other key step of image-guided surgery, can still be intricate and time-consuming with traditional systems. This research explores the feasibility of using the depth sensors of the HoloLens 2, a state-of-the-art AR HMD, for depth-based image-to-patient registration. This research contributes to the advancement of less complex and more efficient image-guided surgical techniques. Methods: To achieve these objectives, three experiments were conducted using a pilot system based on the HoloLens 2's depth sensors. The first experiment evaluated the accuracy of the depth sensors quantitatively. The second experiment compared four registration initialization methods, including manual and automated approaches. The accuracy and success rate of alignment were assessed using a multi-modal ground truth. Finally, a qualitative assessment of the pilot system was performed on various objects and materials. This experiment aimed to evaluate the system's performance and usability in real-world scenarios. Results: The depth accuracy experiment showed that both the AHAT and LT sensors had mean overestimation errors of 5.7 and 9.0 mm, respectively. In the registration experiment, the two manual initialization methods consistently achieved successful registration (100%), while the two automatic methods had varying success rates (23.3% and 50%). Three out of four depth registration methods completed the registration within 5 seconds. The mean translation errors ranged from 12.6 to 14.7 mm, and rotation errors ranged from 1.5 to 1.8 degrees. The minimum observed translation and rotation errors were 6.9 mm and 0.5 degrees, respectively, while the maximum errors were 18.8 mm and 3.2 degrees, respectively. Conclusion: The study's results suggest the potential for achieving sub-10 mm registration accuracy within 5 seconds with depth-based image-to-patient registration. This offers a fast and convenient alternative to other tracking systems that require invasive fiducial markers and time-consuming calibration steps. However, the current accuracy level of the system poses some limitations. Nonetheless, the developed system holds promise for a wide range of surgical procedures that currently do not utilize image guidance due to its complexity. By enabling faster and more accessible image guidance, depth-based registration has the potential to enhance surgical outcomes, such as improving tumor resection margins and avoidance of vulnerable tissues, making it highly beneficial for various procedures.

Introduction: Trauma-induced rib fractures are a common injury, affecting millions of individuals globally each year. Although anteroposterior thoracic radiographs are part of the standard posttraumatic screening, the most sensitive modality, and therefore golden standard for diagnosing rib fractures, is computed tomography (CT). Still, between 19.2% and 26.8% of rib fractures are missed. Another problem encountered in rib fracture treatment management is the large interobserver variability on their taxonomy. This thesis aims to automate rib fracture detection and improve consistency in their classification by developing a Deep Learning (DL) model, using CT data. Methods: The rib fractures were classified according to the Chest Wall Injury Society (CWIS) taxonomy, evaluating rib fracture’s type, displacement and location. Furthermore, the ribs were numbered from 1 up to and including 12 from cranio-caudal direction. For the detection and three CWIS labels, three classification models of the nnDetection framework were trained. The rib numbering consisted of a trained nnU-Net segmentation model. The four models were combined to obtain the proposed DCRibFrac model. Experiments and results: The dataset is composed of retrospectively collected and anonymized CT scans of 100 randomly selected patients (1010 rib fractures) who were admitted to the Erasmus MC following blunt chest trauma. On the internal test set, DCRibFrac achieved a detection sensitivity of 77%, precision of 79%, and F1-score of 78%, with a mean false-positives per scan of 2.26. The type labels had the lowest scores, with sensitivities between 17% and 90%. The displacement labels had sensitivities between 43% and 91%. The location labels had the highest scores, with sensitivities between 88% and 96%. The rib number was correct in 72% of the rib fractures when wrong segmentations were excluded. Conclusion: The proposed DL model automates acute rib fracture detection and reaches a sensitivity that is on par with clinicians. This model is the first, to the authors’ knowledge, to incorporate the CWIS taxonomy and shows its potential for achieving a consistent classification.

Predicting functional outcome after intra-arterial treatment (IAT) in acute ischemic stroke (AIS) patients is an important aspect of treatment decision making and prognostics. Standard methods for functional outcome prediction after stroke combine baseline clinical (and radiological) parameters. In this study, we investigated to what extent baseline CTA images can be used for the prediction of functional outcome and how this relates to standard scoring methods. Furthermore, it was investigated whether combining baseline CTA images with clinical parameters improved the predictive accuracy compared to outcome prediction based on clinical parameters. We proposed two network architectures, a convolutional neural network (CNN) for the processing of image data and a multilayer perceptron for the processing of clinical (and radiological) parameters. Various training strategies were applied for the fusion of image and clinical data. The CNN processing CTA images achieved an average cross-validated area under the curve (AUC) score of 0.67, which was lower than for models processing clinical (and radiological) parameters. The best performing model combining CTA images and clinical parameters was trained end-to-end and applied weight initialization of the pre-trained CNN (AUC = 0.78). The DeLong test showed that the combined model performed significantly better than the model processing clinical parameters (AUC = 0.75). However, the difference is small and might not be clinically relevant. Compared to scoring methods processing clinical and radiological parameters the combined model achieved similar performance.