Virtual Reality-Based Training Simulator for Robot-Assisted Surgery Using Unity and 3D Slicer

Abstract

Robotic interventions have become a pivotal component in minimally invasive neurosurgical procedures. While the use of robotic devices in medical operations is advancing, there is still a pressing need to train and equip medical professionals with emerging biomedical developments. Though Virtual reality (VR) has been proven to be a great tool for embodied learning, existing studies demonstrate a lack of interactive open-source platforms for medical interventions. Addressing this need for robot-assisted surgery (RAS) skills acquisition task, we propose a heuristic learning scheme in a simulator encompassing cross-platform registration tasks to train individuals for a typical image-guided surgery (IGS) procedure. The proposed training simulator offers an interactive, risk-free, cost-effective, and efficient learning platform for acquiring RAS skills both in Desktop VR and Head Mounted Display (HMD) VR settings. This training module provides a workflow for patient registration between the simulated robotic coordinate space in Unity 3D, and the IGS coordinate space in 3D slicer. The simulator's interface provides a user-friendly training environment with realtime instructions as well as visual and tactile feedback. A data collection scheme is also presented for a comparative user study of the learning experiences between HMD and desktop VR-based learning settings. This work showcases the efficacy of RAS training platform in performing cross-platform fiducial registration with an RMSE of 1.34 mm (in Desktop VR) and 1.49 mm (in HMD settings).