Increased demand for less invasive procedures has accelerated the adoption of Intraluminal Procedures (IP) and Endovascular Interventions (EI) performed through body lumens and vessels. As navigation through lumens and vessels is quite complex, interest grows to establish autonomous navigation techniques for IP and EI for reaching the target area. Current research efforts are directed toward increasing the Level of Autonomy (LoA) during the navigation phase. One key ingredient for autonomous navigation is Motion Planning (MP) techniques. This paper provides an overview of MP techniques categorizing them based on LoA. Our analysis investigates advances for the different clinical scenarios. Through a systematic literature analysis using the PRISMA method, the study summarizes relevant works and investigates the clinical aim, LoA, adopted MP techniques, and validation types. We identify the limitations of the corresponding MP methods and provide directions to improve the robustness of the algorithms in dynamic intraluminal environments. MP for IP and EI can be classified into four subgroups: node, sampling, optimization, and learning-based techniques, with a notable rise in learning-based approaches in recent years. One of the review's contributions is the identification of the limiting factors in IP and EI robotic systems hindering higher levels of autonomous navigation. In the future, navigation is bound to become more autonomous, placing the clinician in a supervisory position to improve control precision and reduce workload.@en

Minimally Invasive Procedures (MIPs) emerged as an alternative to more invasive surgical approaches, offering patient benefits such as smaller incisions, less pain, and shorter hospital stay. In one class of MIPs, where natural body lumens or small incisions are used to access deeper anatomical locations, Flexible Surgical and Interventional Robots (FSIRs) such as catheters and endoscopes are widely used. Due to their flexible and compliant nature, FSIRs can be inserted via natural orifices or small incisions, then moved towards hard-to-reach targets to perform interventional tasks. However, existing FSIRs are confronted with challenges in sensing, control, and navigation. These issues stem from the robot's non-linear behavior and the intricate nature of the lumens, where accurately modeling the complex interactions and disturbances proves to be exceptionally difficult. The rapid advances in Machine Learning (ML) have facilitated the widespread adoption of ML techniques in FSIRs. This article provides an overview of these efforts by first introducing a classification of existing ML algorithms, including traditional ML methods and modern Deep Learning (DL) approaches, commonly used in FSIRs. Next, the use of ML algorithms is surveyed per sub-domain, namely for perception, modeling, control, and navigation. Trends, popularity, strengths, and/or limitations of different ML algorithms are analyzed. The different roles that ML plays among tasks are investigated and described. Finally, discussions are conducted on the limitations and the prospects of ML in MIPs.@en

Endovascular interventions, as emerging medical therapies, utilize blood vessels as conduits to access anatomically challenging regions deep within the body. Within endovascular interventions, one of the prominent challenges involves maneuvering the instrument tip by coordinating insertion, retraction, and torque actions at the proximal end of the instrument. This intricate task is hindered by the presence of a complex mapping between input actions and resulting motion, rendering precise control and accurate targeting of the desired area difficult. Thanks to the introduction of robotic assistance and the steerability of robotic catheters, the complexity of endovascular interventions has been mitigated. The integration of steerable catheters and navigation guidance has the potential to reduce the level of expertise required for endovascular interventions. By leveraging autonomous navigation, path-related complications, such as perforation, embolization, and dissection, arising from excessive interaction forces between interventional tools and the vessels, can be effectively addressed and potentially reduced. Within the context of robotic catheters navigating through narrow, delicate, and deformable vessels, path planning presents significant challenges, particularly under complex operating conditions, stringent safety constraints, and the inherent limitations on catheter steering capability. Furthermore, the intricate interplay between the steerable catheter and vessel walls, coupled with the deformable nature of the vessels, intensifies the complexity of achieving reliable and real-time path planning, rendering it a hard problem to solve. This dissertation aims to develop a safe, accurate, and efficient path planner for steerable robotic catheters. Firstly, this dissertation provides a systematic literature analysis of path planning techniques, collating the findings from the most significant research contributions in the field employing the PRISMA method. In the first part of this dissertation, a novel path planning approach named BFS-GA is proposed, which effectively adheres to the robot curvature constraint while keeping the catheter's path as close to the vasculature's centerline as possible. This path planner is capable of swiftly calculating obstacle-free trajectories that conform to the patient's vasculature, while incorporating the inherent limitations of the catheter such as maximum curvature. A major challenge during autonomous navigation in endovascular interventions is the complexity of operating in a deformable but constrained workspace with an instrument. To address this, two methods are proposed in the second part of this dissertation to provide a realistic and dynamic environment for path planning. Specifically, a realistic, auto-adaptive, and visually plausible simulator is developed. This simulator has the capability to accurately predict the interplay between catheters and vessel walls. Additionally, it accounts for the deformable nature of the vessels induced by the cyclic heartbeat motion. In addition, a novel deformable model-to-image registration framework is designed to reconstruct comprehensive intra-operative vessel structures from medical imaging data, while accurately accounting for deformations. Given the dynamic vascular environments generated as above, a robust path planner named C-GAIL for steerable catheters is proposed in the third part of this dissertation. This path planner ensures higher precision and robustness by accounting for both the deformable properties of vessels and the catheter's steering capabilities. The in-vitro experiments demonstrate that the path generated by the proposed C-GAIL path planner aligns better with the actual steering capability of robotic catheters. Thereafter, the dissertation presents an in-depth exploration of path planning assistance utilizing various interactive modalities based on augmented reality. Three interactive control modalities for steering robotic catheters are introduced, and their impact on human-in-the-loop robot-assisted cardiac catheterization is investigated. The path guidance is facilitated by the previously discussed C-GAIL path planning method. A user study is conducted, which demonstrates the feasibility of harnessing the capabilities of a gaming joystick for catheter teleoperation and the practicality of utilizing a head-mounted display to receive 3D visual feedback.@en

Purpose: Planning a safe path for flexible catheters is one of the major challenges of endovascular catheterization. State-of-the-art methods rarely consider the catheter curvature constraint and reduced computational time of path planning which guarantees the possibility to re-plan the path during the actual operation. Methods: In this manuscript, we propose a fast two-phase path planning approach under the robot curvature constraint. Firstly, the vascular structure is extracted and represented by vascular centerlines and corresponding vascular radii. Then, the path is searched along the vascular centerline using breadth first search (BFS) strategy and locally optimized via the genetic algorithm (GA) to satisfy the robot curvature constraint. This approach (BFS-GA) is able to respect the robot curvature constraint while keeping it close to the centerlines as much as possible. We can also reduce the optimization search space and perform parallel optimization to shorten the computational time. Results: We demonstrate the method’s high efficiency in two-dimensional and three-dimensional space scenarios. The results showed the planner’s ability to satisfy the robot curvature constraint while keeping low computational time cost compared with sampling-based methods. Path replanning in femoral arteries can reach an updating frequency at 6.4 ± 2.3 Hz. Conclusion: The presented work is suited for surgical procedures demanding satisfying curvature constraints while optimizing specified criteria. It is also applicable for curvature constrained robots in narrow passages.@en

A major challenge during autonomous navigation in endovascular interventions is the complexity of operating in a deformable but constrained workspace with an instrument. Simulation of deformations for it can provide a cost-effective training platform for path planning. Aim of this study is to develop a realistic, auto-adaptive, and visually plausible simulator to predict vessels’ global deformation induced by the robotic catheter's contact and cyclic heartbeat motion. Based on a Position-based Dynamics (PBD) approach for vessel modeling, Particle Swarm Optimization (PSO) algorithm is employed for an auto-adaptive calibration of PBD deformation parameters and of the vessels movement due to a heartbeat. In-vitro experiments were conducted and compared with in-silico results. The end-user evaluation results were reported through quantitative performance metrics and a 5-Point Likert Scale questionnaire. Compared with literature, this simulator has an error of 0.23±0.13% for deformation and 0.30±0.85mm for the aortic root displacement. In-vitro experiments show an error of 1.35±1.38mm for deformation prediction. The end-user evaluation results show that novices are more accustomed to using joystick controllers, and cardiologists are more satisfied with the visual authenticity. The real-time and accurate performance of the simulator make this framework suitable for creating a dynamic environment for autonomous navigation of robotic catheters.@en

Cardiologists highlight the need for an intra-operative 3D visualization to assist interventions. The intra-operative 2D X-ray/Digital Subtraction Angiography (DSA) images in the standard clinical workflow limit cardiologists’ views significantly. Compared with image-to-image registration, model-to-image registration is an essential approach taking advantage of the reuse of pre-operative 3D models reconstructed from Computed Tomography Angiography (CTA) images. Traditional optimized-based registration methods suffer severely from high computational complexity. Moreover, the consequence of lacking ground truth for learning-based registration approaches should not be neglected. To overcome these challenges, we introduce a model-to-image registration framework via deep learning for image-guided endovascular catheterization. This work performs autonomous vessel segmentation from intra-operative fluoroscopy images via a deep residual U-net and a model-to-image matching via a convolutional neural network. For this study, image data were collected from 10 patients who performed Transcatheter Aortic Valve Implantation (TAVI) procedures. It was found that vessel segmentation of test data results in median values of Dice Similarity Coefficient, Precision, and Recall of (0.75, 0.58, 0.67) for femoral artery, and (0.71, 0.56, 0.74) for aortic root. The segmentation network behaves better than manual annotation, and it recognizes part of vessels that were not labeled manually. Image matching between the transformed moving image and the fixed image results in a median value of Recall of 0.90. The proposed approach achieves a good accuracy of vessel segmentation and a good recall value of model-to-image matching.@en

Endovascular intervention is a minimally invasive method for treating cardiovascular diseases. Although fluoroscopy, known for real-time catheter visualization, is commonly used, it exposes patients and physicians to ionizing radiation and lacks depth perception due to its 2D nature. To address these limitations, a study was conducted using teleoperation and 3D visualization techniques. This <italic>in-vitro</italic> study involved the use of a robotic catheter system and aimed to evaluate user performance through both subjective and objective measures. The focus was on determining the most effective modes of interaction. Three interactive modes for guiding robotic catheters were compared in the study: 1) Mode GM, using a gamepad for control and a standard 2D monitor for visual feedback; 2) Mode GH, with a gamepad for control and HoloLens providing 3D visualization; and 3) Mode HH, where HoloLens serves as both control input and visualization device. Mode GH outperformed other modalities in subjective metrics, except for mental demand. It exhibited a median tracking error of 4.72 mm, a median targeting error of 1.01 mm, a median duration of 82.34 s, and a median natural logarithm of dimensionless squared jerk of 40.38 in the <italic>in-vitro</italic> study. Mode GH showed 8.5%, 4.7%, 6.5%, and 3.9% improvements over Mode GM and 1.5%, 33.6%, 34.9%, and 8.1% over Mode HH for tracking error, targeting error, duration, and dimensionless squared jerk, respectively. To sum up, the user study emphasizes the potential benefits of employing HoloLens for enhanced 3D visualization in catheterization. The user study also illustrates the advantages of using a gamepad for catheter teleoperation, including user-friendliness and passive haptic feedback, compared to HoloLens. To further gauge the potential of using a more traditional joystick as a control input device, an additional study utilizing the Haption VirtuoseTM robot was conducted. It reveals the potential for achieving smoother trajectories, with a 38.9% reduction in total path length compared to a gamepad, potentially due to its larger range of motion and single-handed control.@en

Endovascular intervention is a minimally invasive method for treating cardiovascular diseases. Although fluoroscopy, known for real-time catheter visualization, is commonly used, it exposes patients and physicians to ionizing radiation and lacks depth perception due to its 2D nature. To address these limitations, a study was conducted using teleoperation and 3D visualization techniques. This <italic>in-vitro</italic> study involved the use of a robotic catheter system and aimed to evaluate user performance through both subjective and objective measures. The focus was on determining the most effective modes of interaction. Three interactive modes for guiding robotic catheters were compared in the study: 1) Mode GM, using a gamepad for control and a standard 2D monitor for visual feedback; 2) Mode GH, with a gamepad for control and HoloLens providing 3D visualization; and 3) Mode HH, where HoloLens serves as both control input and visualization device. Mode GH outperformed other modalities in subjective metrics, except for mental demand. It exhibited a median tracking error of 4.72 mm, a median targeting error of 1.01 mm, a median duration of 82.34 s, and a median natural logarithm of dimensionless squared jerk of 40.38 in the <italic>in-vitro</italic> study. Mode GH showed 8.5%, 4.7%, 6.5%, and 3.9% improvements over Mode GM and 1.5%, 33.6%, 34.9%, and 8.1% over Mode HH for tracking error, targeting error, duration, and dimensionless squared jerk, respectively. To sum up, the user study emphasizes the potential benefits of employing HoloLens for enhanced 3D visualization in catheterization. The user study also illustrates the advantages of using a gamepad for catheter teleoperation, including user-friendliness and passive haptic feedback, compared to HoloLens. To further gauge the potential of using a more traditional joystick as a control input device, an additional study utilizing the Haption VirtuoseTM robot was conducted. It reveals the potential for achieving smoother trajectories, with a 38.9% reduction in total path length compared to a gamepad, potentially due to its larger range of motion and single-handed control.@en

Endovascular intervention is a minimally invasive method for treating cardiovascular diseases. Although fluoroscopy, known for real-time catheter visualization, is commonly used, it exposes patients and physicians to ionizing radiation and lacks depth perception due to its 2D nature. To address these limitations, a study was conducted using teleoperation and 3D visualization techniques. This <italic>in-vitro</italic> study involved the use of a robotic catheter system and aimed to evaluate user performance through both subjective and objective measures. The focus was on determining the most effective modes of interaction. Three interactive modes for guiding robotic catheters were compared in the study: 1) Mode GM, using a gamepad for control and a standard 2D monitor for visual feedback; 2) Mode GH, with a gamepad for control and HoloLens providing 3D visualization; and 3) Mode HH, where HoloLens serves as both control input and visualization device. Mode GH outperformed other modalities in subjective metrics, except for mental demand. It exhibited a median tracking error of 4.72 mm, a median targeting error of 1.01 mm, a median duration of 82.34 s, and a median natural logarithm of dimensionless squared jerk of 40.38 in the <italic>in-vitro</italic> study. Mode GH showed 8.5%, 4.7%, 6.5%, and 3.9% improvements over Mode GM and 1.5%, 33.6%, 34.9%, and 8.1% over Mode HH for tracking error, targeting error, duration, and dimensionless squared jerk, respectively. To sum up, the user study emphasizes the potential benefits of employing HoloLens for enhanced 3D visualization in catheterization. The user study also illustrates the advantages of using a gamepad for catheter teleoperation, including user-friendliness and passive haptic feedback, compared to HoloLens. To further gauge the potential of using a more traditional joystick as a control input device, an additional study utilizing the Haption VirtuoseTM robot was conducted. It reveals the potential for achieving smoother trajectories, with a 38.9% reduction in total path length compared to a gamepad, potentially due to its larger range of motion and single-handed control.@en

Endovascular interventions are minimally invasive procedures that utilize the vascular system to access anatomical regions deep within the body. Image-guided assistance provides valuable real-time information about the dynamic state of the vascular environment. However, the reliance on intra-operative 2D fluoroscopy images limits depth perception, prompting the demand for intra-operative 3D imaging. Existing image registration methods face difficulties in accurately incorporating tissue deformations compared to the pre-operative 3D model, particularly in a weakly-supervised manner. Additionally, reconstructing deformations from 2D to 3D space and presenting this intra-operative model visually to clinicians poses further complexities. To address these challenges, this study introduces a novel deformable model-to-image registration framework using deep learning. Furthermore, this research proposes a visualization method through augmented reality to provide guidance for endovascular interventions. This study utilized image data collected from nine patients who underwent Transcatheter Aortic Valve Implantation (TAVI) procedures. The registration results in 2D indicate that the proposed deformable model-to-image registration framework achieves a Modified Dice Similarity Coefficient (MDSC) value of 0.89 ? 0.02 and a Penalization of Deformations in Spare Space (PDSS) value of 0.04 ? 0.01, offering an improvement of 3.5% and 98.6% over the state-of-the-art image registration approach. Additionally, the accuracy of registration in 3D was evaluated using a dataset obtained from an intervention simulator, resulting in a Mean Absolute Error (MAE) of 1.51 ? 1.02 mm within the region of interest. Overall, the study validates the feasibility and accuracy of the proposed weakly-supervised deformable model-to-image registration framework, demonstrating its potential to provide intra-operative 3D imaging as intervention assistance in dynamic vascular environments.@en