TK
T.H. Knell
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
1 records found
1
Pre-operative anatomical data is essential for guidance in neurosurgery, but brain movement and tissue manipulation degrade its registration to the patient during surgery. Restoring the registration requires intra-operative updates of the surface geometry, which a calibrated stereo surgical microscope could triangulate from the rectified image pairs it delivers. However, low-texture surfaces, reflections, and blur make stereo matching in this domain difficult. Zero-shot stereo foundation models, deep networks trained on large synthetic datasets of natural images, generalize to unseen domains without fine-tuning and handle comparable conditions well on public benchmarks. Yet their performance for neurosurgery is unknown because no benchmark exists to evaluate them in this domain.
We close this gap by assessing whether zero-shot stereo foundation models are suitable for stereo reconstruction in neurosurgical microscopy by analyzing their accuracy, robustness, and failure predictability. To this end, the Microscopy Images for Neurosurgical Dense Stereo reconstruction (MINDS) benchmark is introduced, which provides realistic synthetic and ex-vivo porcine brain images with reference depth and segmentation masks. Four zero-shot models and two classical reference methods are evaluated on MINDS for reconstruction accuracy and robustness under reflections, blur, and weak texture. Additionally, a failure predictor is developed from inference-time confidence cues to investigate whether unreliable depth estimates could be caught in practice.
The best classical method places 23% and 76% of pixels within 0.1 mm and 1 mm of the reference, respectively. The best zero-shot model reaches 59% and 97% at the same thresholds. This accuracy gap persists across all texture levels. Under specular reflection, the best zero-shot model stays at 55% and 99%, while the reference methods drop to 4% and 42%. Defocus blur, by contrast, degrades every tested method to a similar extent and remains the dominant barrier to sub-millimeter accuracy. The failure predictor separates reliable from unreliable pixels at an average precision of 0.63 on synthetic data, nearly four times the chance level of 0.17, with recall rising from 0.75 for errors of 1 mm to 5 mm to 0.99 for errors above 10 mm. Together, these results establish the present capabilities and limitations of zero-shot foundation models for neurosurgical stereo reconstruction.
...
We close this gap by assessing whether zero-shot stereo foundation models are suitable for stereo reconstruction in neurosurgical microscopy by analyzing their accuracy, robustness, and failure predictability. To this end, the Microscopy Images for Neurosurgical Dense Stereo reconstruction (MINDS) benchmark is introduced, which provides realistic synthetic and ex-vivo porcine brain images with reference depth and segmentation masks. Four zero-shot models and two classical reference methods are evaluated on MINDS for reconstruction accuracy and robustness under reflections, blur, and weak texture. Additionally, a failure predictor is developed from inference-time confidence cues to investigate whether unreliable depth estimates could be caught in practice.
The best classical method places 23% and 76% of pixels within 0.1 mm and 1 mm of the reference, respectively. The best zero-shot model reaches 59% and 97% at the same thresholds. This accuracy gap persists across all texture levels. Under specular reflection, the best zero-shot model stays at 55% and 99%, while the reference methods drop to 4% and 42%. Defocus blur, by contrast, degrades every tested method to a similar extent and remains the dominant barrier to sub-millimeter accuracy. The failure predictor separates reliable from unreliable pixels at an average precision of 0.63 on synthetic data, nearly four times the chance level of 0.17, with recall rising from 0.75 for errors of 1 mm to 5 mm to 0.99 for errors above 10 mm. Together, these results establish the present capabilities and limitations of zero-shot foundation models for neurosurgical stereo reconstruction.
...
Pre-operative anatomical data is essential for guidance in neurosurgery, but brain movement and tissue manipulation degrade its registration to the patient during surgery. Restoring the registration requires intra-operative updates of the surface geometry, which a calibrated stereo surgical microscope could triangulate from the rectified image pairs it delivers. However, low-texture surfaces, reflections, and blur make stereo matching in this domain difficult. Zero-shot stereo foundation models, deep networks trained on large synthetic datasets of natural images, generalize to unseen domains without fine-tuning and handle comparable conditions well on public benchmarks. Yet their performance for neurosurgery is unknown because no benchmark exists to evaluate them in this domain.
We close this gap by assessing whether zero-shot stereo foundation models are suitable for stereo reconstruction in neurosurgical microscopy by analyzing their accuracy, robustness, and failure predictability. To this end, the Microscopy Images for Neurosurgical Dense Stereo reconstruction (MINDS) benchmark is introduced, which provides realistic synthetic and ex-vivo porcine brain images with reference depth and segmentation masks. Four zero-shot models and two classical reference methods are evaluated on MINDS for reconstruction accuracy and robustness under reflections, blur, and weak texture. Additionally, a failure predictor is developed from inference-time confidence cues to investigate whether unreliable depth estimates could be caught in practice.
The best classical method places 23% and 76% of pixels within 0.1 mm and 1 mm of the reference, respectively. The best zero-shot model reaches 59% and 97% at the same thresholds. This accuracy gap persists across all texture levels. Under specular reflection, the best zero-shot model stays at 55% and 99%, while the reference methods drop to 4% and 42%. Defocus blur, by contrast, degrades every tested method to a similar extent and remains the dominant barrier to sub-millimeter accuracy. The failure predictor separates reliable from unreliable pixels at an average precision of 0.63 on synthetic data, nearly four times the chance level of 0.17, with recall rising from 0.75 for errors of 1 mm to 5 mm to 0.99 for errors above 10 mm. Together, these results establish the present capabilities and limitations of zero-shot foundation models for neurosurgical stereo reconstruction.
We close this gap by assessing whether zero-shot stereo foundation models are suitable for stereo reconstruction in neurosurgical microscopy by analyzing their accuracy, robustness, and failure predictability. To this end, the Microscopy Images for Neurosurgical Dense Stereo reconstruction (MINDS) benchmark is introduced, which provides realistic synthetic and ex-vivo porcine brain images with reference depth and segmentation masks. Four zero-shot models and two classical reference methods are evaluated on MINDS for reconstruction accuracy and robustness under reflections, blur, and weak texture. Additionally, a failure predictor is developed from inference-time confidence cues to investigate whether unreliable depth estimates could be caught in practice.
The best classical method places 23% and 76% of pixels within 0.1 mm and 1 mm of the reference, respectively. The best zero-shot model reaches 59% and 97% at the same thresholds. This accuracy gap persists across all texture levels. Under specular reflection, the best zero-shot model stays at 55% and 99%, while the reference methods drop to 4% and 42%. Defocus blur, by contrast, degrades every tested method to a similar extent and remains the dominant barrier to sub-millimeter accuracy. The failure predictor separates reliable from unreliable pixels at an average precision of 0.63 on synthetic data, nearly four times the chance level of 0.17, with recall rising from 0.75 for errors of 1 mm to 5 mm to 0.99 for errors above 10 mm. Together, these results establish the present capabilities and limitations of zero-shot foundation models for neurosurgical stereo reconstruction.