Objective: This study introduces a novel deep-learning-based orientation recognition approach for detecting intraoperative lung orientation during robot-assisted anatomical resections, including lobectomy and segmentectomy. This method can potentially aid in anatomical structure identification, facilitate training and education, improve procedural efficiency, and enhance intraoperative imaging navigation. Methods: We developed a unique dataset encompassing various pulmonary procedures, being the first to report on recognition of intraoperative orientation. The TeCNO model, initially developed for laparoscopic cholecystectomies, was adapted for this study. Model performance was evaluated using accuracy, precision, recall, and F1-score, and we explored the influence of dataset composition, intraoperative factors such as 3D model presence, and visual impairments. Results: The model achieved an overall accuracy of 70%, indicating potential in recognizing lung orientation. High performance was achieved in recognizing non-surgical sequences, ‘Fissure’, and ‘Inferior’ views. ‘Posterior’ and ‘Anterior’ views showed inferior performance. Variability in performance was attributed to the heterogeneity of orientation transitions and increased complexity compared to more standardized procedures. The limited dataset size and imbalances in label distribution potentially impacted model performance. Conclusion: This study demonstrates the feasibility of applying phase recognition to detect orientation of the lung and exploring how the unique characteristics of our dataset affect model performance opposed to surgical phase recognition. The results suggest promising applications for intraoperative imaging guidance and automated adjustment of 3D models, particularly for complex orientations like the interlobar ‘Fissure view’. Future research should focus on enhancing model performance and assessing its clinical implementation in diverse surgical settings.

Currently, morphological measurements on conventional radiographs are used to assess calcaneal fractures. The two most commonly used angles for determining the degree of depression and displacement of such a fracture are Böhler’s angle (BA) and the Critical angle of Gissane (CAG). However, conventional radiographs only provide limited amount of information and are increasingly being omitted, while computerized tomography (CT) scans are increasingly being used as they contain much more information regarding fracture configuration and displacement. It is therefore important that morphological measurements can be performed using CT-scans. The purpose of this thesis were to design a novel method to perform semi-automated morphological measurements based on 3D models of the calcaneus. The results of the novel method were compared with conventional 2D measurements of the same calcaneus. 119 lateral foot radiographs and CT-scan of the same foot were obtained. The data consisted out of three populations: 40 non-fractured, 39 fractured and 40 reconstructed calcanei. Based on the CT-scans, 3D models were reconstructed through interactive segmentation processes. Measurements on the radiographs and 3D models were performed in a specially designed Graphical User Interface. Three observers measured BA and the CAG, after which they repeated the measurements after a minimum of one week. The inter- and intra-observers reliability were calculated separately for all three populations, using the intra-class correlation coefficient (ICC). Additionally, the frequency of consensus given an allowed discrepancy was calculated. The ICC for inter- and intra-observer reliability of the 3D measurements showed an improvement for BA and the CAG for all populations compared with the ICC of the 2D measurements. The ICC for inter- and intra-observer reliability of the 3D measured BA ranged from 0.97 to 1.00, indicating an almost perfect agreement between the observers. For the CAG, the inter- and intra-observer ICC ranged from 0.89 to 0.92 and from 0.87 to 0.96, respectively. To achieve 80% consensus for the BA among the observers, an allowable discrepancy of 20 degrees was needed for the 2D measurements, while only 5 degrees was needed for the 3D measurements. To achieve 80% of consensus for the 2D CAG measurements, an allowable discrepancy of at least 20 degrees was needed. For the fractured and reconstructed population, even an allowed discrepancy of 30 degrees was needed. For the 3D CAG measurements, 80% of consensus was achieved with an allowable discrepancy of 5 degrees, 20 degrees and 10 degrees for the non-fractured, fractured and reconstructed calcanei populations, respectively. 3D morphological measurements based on 3D models of the calcaneus showed better reliability and repeatability compared to the 2D measurements. However, the novel method of performing the measurements is not yet safe for use in clinical practice, as the measurements first should be validated. Nevertheless, the method is well suited to be employed as a reference for the development of further (semi-)automated methods to perform morphological measurements.

In today’s medicine, 3D planning and PSI are not commonly used in revision TKA surgery and little is known about the accuracy and implementation of these techniques in revision TKA. Therefore, the question arises, what are the potential benefits of 3D planning and PSI in revision surgery? The goal of this thesis is to explore the potential benefits of 3D planning and PSI in revision surgery. To achieve this, a retrospective study and a prospective study are performed, and a first design of a 3D printed PSI is created. For both retrospective and prospective study, 3D plannings are made of patients undergoing revision TKA, comparing the pre-operative planning with the post-operative results based on size prediction, augmentation prediction and component placement analysis. The results of the studies conducted in this research align closely with each other, providing similar results for size and augmentation prediction and component placement analysis. All in all, these studies indicated consistency and reliability of the 3D prediction in different scenarios, affirming the potential of 3D planning in revision TKA surgery. Additionally, the first design for a PSI, a 3D-printed guide, is constructed. This design incorporates several important landmarks for component placement during revision TKA, increasing the outcome of these surgeries, especially if landmarks are missing or unidentifiable during surgery. Together 3D planning and PSI have a bright future in revision surgery, however, future research needs to be conducted to implement 3D planning and PSI in modern-day healthcare.

Background Head-mounted augmented reality (AR) can be beneficial in orthopedic trauma surgery by projecting 3-dimensional (3D) holograms of preoperative imaging data, drilling or cutting planes, and osteosynthesis material, directly onto the surgeon’s field of view (FOV). For successful implementation of head-mounted AR in orthopedic trauma surgery, the surgeon’s experience is of utmost importance. This prospective clinical study aimed to evaluate the surgeon’s experience with head-mounted AR in orthopedic trauma surgery. Methods The Microsoft HoloLens 2 (Microsoft, Redmond, WA, USA) was used to project patient-specific 3D models of the patient’s fracture site as 3D holograms in the surgeon’s FOV during surgery of intra-articular fractures. The surgeon’s experience was recorded using the Simulator Sickness Questionnaire, Borg CR10 Scale, National Aeronautics and Space Administration Task Load Index, System Usability Scale, and a custom-made questionnaire. Results A total of 14 ORIF surgeries were performed while using a head-mounted AR device. Overall, surgeons were satisfied with AR HMDs (67.9 ± 22.3 (mean ± SD)) and wanted to use it again (70.0 ± 22.54 (mean ± SD)). Good results were reported regarding the usability of the AR HMD (65.7 ± 16.6 (mean ± SD)) and the interaction with 3D holograms 72.1 ± 15.8 (mean ± SD). However, the mental demand, size, and weight of the AR HMD were rated less favorable. Conclusion The surgeon’s experience with the intraoperative use of head-mounted AR in orthopedic trauma surgery is sufficient and head-mounted AR has great future potential in orthopedic trauma surgery. Future research has to focus on an ergonomic design of surgical AR HMDs, technological developments, a simple and fluent workflow, and proving clinical value of head-mounted AR.

Introduction: 3D imaging has become the standard for orthognathic surgery planning. Manual segmentation and landmark localization are the foundation for assessing deformities and planning of the corrective surgery. However, in 3D imaging, these manual procedures are complex and time consuming. Deep learning has emerged as a promising solution to automate this workflow. Objective: To develop an automated segmentation and landmark localization workflow to reduce time and efforted required for orthognathic surgery planning using CBCT. Methods: The dataset consisted of 57 presurgical Cone Beam CT (CBCT) scans. Manual segmentations were created for the mandible, maxilla, skin, mandibular canals (MC) left and right. Additionally, 43 landmarks (13 mandibular, 10 maxillary, 10 dental, and 10 surgical) were manually annotated. Automated landmark localization was approached as a segmentation task, with spherical segmentation of adjacent tissue with radii of 4mm, 5mm, or 6mm around landmarks. The dataset was split into 64% train, 16% validation 20% testset. Seven nnU-Net models were trained: one for segmentation and six (two models for each radius configuration) for landmark localization. Landmark positions were determined by calculating the center of mass of predicted segmentations. Performance was evaluated by comparing results to manual ground truth segmentations and landmark locations. Results: The nnU-Net was successfully trained to identify five segmentations and 43 landmarks. The overall median [Q1 – Q3] volumetric dice coefficient (vDSC) was 0.91 [0.79 - 0.96] for the segmentation model. The overall median radial error (MRE) for the landmarking models using 4mm, 5mm and 6mm spherical segmentations (each based on two models) were 0.98 [0.58 - 1.60] mm, 1.03 [0.63 - 1.67] mm, and 1.08 [0.66 - 1.82] mm, respectively. The successful detection rate below the clinical acceptability threshold of < 2mm ranged from 78.6% to 81.6%. The inference workflow required 17.7 minutes per patient on average. Conclusion: The nnU-Net achieved accurate craniomaxillofacial (CMF) anatomy segmentations and precise localization of landmarks, maintaining a clinical acceptability level with an error margin of less than 2mm. This fully automated workflow has the potential to enhance the efficiency of CMF planning.