Augmented Reality (AR) offers significant potential to enhance surgical procedures by overlaying critical digital information directly onto the surgeon's view, improving spatial awareness and precision. Accurate real-time tracking of surgical instruments is fundamental to the success of AR navigation systems. However, implementing established tracking modalities, such as infrared (IR) tracking using retro-reflective markers, on newer Head-Mounted Displays (HMDs) like the Magic Leap 2 presents unique challenges due to its specific sensor configuration, notably the lack of accessible IR intensity data from its Time-of-Flight (ToF) sensor and the presence of IR-cut filters on its World Cameras.

This thesis addresses these challenges through the development, implementation, and evaluation of ISITLeap, a novel system designed specifically for IR instrument tracking on the Magic Leap 2 platform. The system employs a stereo vision approach, utilizing the Magic Leap 2's built-in World Cameras configured with short exposure times, synergistically combined with a custom-designed external 760 nm IR illumination module to detect standard passive retro-reflective markers. A comprehensive software pipeline was developed, incorporating image undistortion adapted for the Magic Leap 2's equidistant projection model, marker detection involving binarization, connected components analysis, and area filtering, 3D marker reconstruction using stereo triangulation with empirically derived coordinate system corrections to reconcile Magic Leap 2 SDK data with geometric requirements, and finally, 6-DoF tool pose estimation via point-based registration. Crucially, the entire software pipeline, from image acquisition to pose estimation, operates directly on the Magic Leap 2 headset, requiring no external computing hardware.

Experimental validation successfully demonstrated the feasibility of the ISITLeap system for real-time tracking. Static evaluations characterized the system's performance, identifying an optimal working range of approximately 25-50 cm. Dynamic tracking experiments, evaluated against an NDI optical tracker using an ArUco marker as a common reference frame, yielded an overall Mean Absolute Error (MAE) of approximately 7 mm. Crucially, error analysis revealed that the measured accuracy was significantly influenced by limitations inherent in the ArUco marker-based evaluation methodology, likely masking the intrinsic performance capabilities of the ISITLeap core tracking system.

This work presents a complete, functional hardware and software solution tailored to overcome the specific sensor constraints of the Magic Leap 2, enabling IR marker tracking on this platform. It provides a crucial performance benchmark under the defined evaluation conditions and demonstrates a viable technical pathway for developing precise AR surgical navigation applications for the Magic Leap 2 headset. While the achieved referenced accuracy highlights the need for improved evaluation techniques and further system refinement (e.g., via dynamic filtering and optimized hardware), ISITLeap establishes a foundational system for future advancements in Magic Leap 2-based AR surgical guidance. ... Read more

The application of Augmented Reality (AR) technology in neurosurgery is becoming increas- ingly widespread, especially in External Ventricular Drainage (EVD) procedures, where achiev- ing efficient and precise registration remains a key challenge. Traditional marker-based registra- tion methods are often complex and time-consuming, making them unsuitable for emergency surgical situations. This paper presents a markerless automatic registration method based on HoloLens 2, using the Dlib library to automatically detect facial landmarks, and perform- ing point cloud registration based on these features. We designed four sets of experiments: first, to verify the feasibility of the triangle-based registration algorithm; second, to assess the accuracy and stability of Dlib’s feature point extraction on HoloLens; third, to validate the registration accuracy, including the impact of skin displacement; and finally, to evaluate the accuracy of the insertion path in a simulated EVD procedure. The experimental results show that this method simplifies the registration process and demonstrates advantages in terms of registration accuracy and speed, effectively meeting the real-time and precision requirements of emergency surgeries. This study provides an efficient and reliable AR navigation solution for EVD procedures, with promising prospects for further research. ... Read more

Surgical navigation is a tool that surgeons rely on everyday to perform accurate surgeries all over the world. However, this technology requires good hand-eye coordination and a high level of concentration. HoloNav is a project that inquires to see if using the HoloLens and augmented reality can replace the current surgical navigation methods. To do so, the HoloLens must be able to identify the patient and the location of the surgery instruments, which uses optical reflective spheres. This study focuses on using the grayscale cameras of the HoloLens and a deep learning algorithm YOLOv5 to test if it is possible to precisely detect optical reflective spheres. 3 models were trained with two different data sets, where the results show that the model trained on a data set would perform well on the validation set. However, they would perform far worse when exposed to a data set it was not trained on. ... Read more

Purpose. Distal radius fractures are common fractures of the wrist. These fractures are often displaced and need reduction, after adequate reduction, the patients will have follow-up X-rays to check if the fracture stays stable. This is important because surgery might be required if the fracture becomes unstable. This can lead to delayed surgery which can worsen the treatment outcome.
Therefore, it would be valuable to predict which distal radius fractures are likely to become unstable. Machine learning could help predict the stability of distal radius fractures based on CT. In addition, because there is only a small dataset available, this research also studies the effect of different machine learning methods.
Method. Two different methods were evaluated for the stability prediction of distal radius fractures: traditional machine learning (radiomics method) and a residual network with and without transfer learning (deep learning method). For the radiomics method, a python package called WORC was used, which automatically extracts radiomic features and optimizes machine learning models. For the deep learning method, the backbone of a residual network called Med3D with added layers for classification was used.
Results. The radiomics method combined with augmentation, gave the best results (AUC: 0.64± 0.01). Also, it was found that using an augmented data set in the radiomics method resulted in improved performance. This gives a slight indication that the radiomic method can learn to predict DRF when there would be more data available.
Conclusion. The radiomic method is the most promising method for predicting the stability of distal radius fractures, despite the small difference in performance compared to random guessing and the deep learning method. However, for further research, it is highly recommended to acquire a larger dataset. ... Read more