Language-Conditioned Behavioral Cloning for Autonomous Endoscope Guidance

Automated endoscope guidance reduces the need for continuous manual camera control during minimally invasive surgery. However, the desired camera motion depends on the surgeon's immediate intentions and cannot always be inferred from the visual operative scene alone. We propose to automate the task using language-conditioned behavioral cloning; we define 6 clinically relevant guidance behaviors and collect 235 subtask demonstrations. In preliminary phantom experiments, our policy achieves 100% success on focus, overview, and stop, 80% on search-tip and follow-tip behaviors, and 50% on horizon leveling. These results demonstrate the feasibility of learning heterogeneous surgeon-requested camera behaviors with a single visuomotor policy, while the observed failure modes motivate future work on adding temporal context for more demanding guidance tasks.

Source: arXiv:2610.41978v1 - http://arxiv.org/abs/2610.41978v1 PDF: https://arxiv.org/pdf/2610.41978v1 Original Link: http://arxiv.org/abs/2610.41978v1