Track2Map Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery

Jul 16, 2026

Gaussian splatting enables dense, deformable 3D anatomy reconstruction for robot-assisted minimally invasive surgery, but typical pipelines are offline and rely on accurate camera trajectory priors, restricting use when priors are absent or unreliable. Track2Map introduces an online 3D Gaussian splatting method that jointly optimizes camera trajectory and a deformable 3D scene representation directly from stereo surgical video. Track-anchored deformation initialization and motion-aware stabilization reduce drift under tissue motion. Experiments on StereoMIS show improved reconstruction quality and trajectory accuracy versus SLAM and non-SLAM baselines using pose priors.

LanguageEnglish

CategoryHealthcare

FormatPDF

Pages11

Reading Time28 min

1 Introduction

Motivation and challenges in RAMIS

Related work in deformable reconstruction and SLAM

Key failure mode and contributions of Track2Map

arXiv :2607 .08408v 1 [ cs .CV] 9 Jul 2026 Track2Map: Online Deformable SLAM with Motion-Aware Pose Optimization in Robotic Surgery Tianyi Song 1⋆(B) , Sierra Bonilla 1⋆, Xinwei Ju 1 , Evangelos Mazomenos 1 , Danail Stoyanov 1 , Adam Schmidt2 , Omid Mohareri2 , Sophia Bano 1 , and Francisco Vasconcelos 1 1 Department of Computer Science and UCL Hawkes Institute, University College London, London WC1E 6BT, UK 2 Intuitive Surgical, Inc. , Sunnyvale, CA 94086, USA {[tianyi.song.24](tianyi.song.24) , [sierra.bonilla.21}@ucl.ac.uk](sierra.bonilla.21}@ucl.ac.uk) Abstract. Gaussian splatting is the current state-of-the-art for dense, deformable 3D anatomy reconstruction in robot-assisted minimally invasive surgery (RAMIS); however, most pipelines are offline and depend on accurate camera trajectory priors (often from robotic kinematics), limiting applicability when priors are missing or noisy. To address these limitations, we propose Track2Map, an online 3D Gaussian Splatting pipeline that jointly optimizes camera trajectory and 3D deformable scene representation directly from surgical video. Track2Map is therefore capable of robust 3D reconstructions when camera trajectory priors are either absent or noisy, and due to its online nature it effectively works as a Simultaneous Localisation and Mapping (SLAM) method. To stabilize optimization in the presence of tissue motion and ambiguous visual cues, we introduce a track-anchored deformation initialization using dense 2D point tracks. Track statistics are further utilized to disentangle camera motion from scene deformation by detecting static camera periods and reducing drift during incremental mapping. Experiments on StereoMIS show improved reconstruction quality and camera trajectory against competing SLAM methods, as well as compared to non-SLAM methods that utilize camera trajectory priors. The code is available at [https://track2map.github.io/](https://track2map.github.io/) . Keywords: Robot-assisted surgery · Stereo endoscopy · Online 3D reconstruction · Camera pose optimization 1 Introduction Online 3D reconstruction of deformable tissue is a key prerequisite for developing a range of computer guidance techniques in RAMIS, including characterisation of instrument-tissue interactions, autonomous manipulation, and stable ⋆ These authors contributed equally. 2 T. Song et al. Fig. 1. Given stereo RGB frames, depth maps, and optional (clean/noisy) pose extrinsics/intrinsics and tool masks, Track2Map jointly performs 2D tracking and online deformable 3D reconstruction. [12] provides 2D correspondences that are lifted to 3D and used to estimate/refine camera motion via motion-gated pose optimization and initialize sparse anchor-based deformation. A Gaussian representation like [8] is updated online. The system outputs refined poses and metric reconstruction; 2D/3D tracks are available as by-products of the correspondence module and 3D lifting. augmented reality overlays. However, achieving metric, stable reconstruction in RAMIS remains difficult because the scene is non-rigid, frequently occluded by instruments, and captured with limited viewpoint variation. 3D reconstruction methods have progressed from classical SfM [19] to neural radiance fields [15] and 3D Gaussian Splatting (3DGS) [14] representations tailored to endoscopy [4,25,21] . However, extending these representations to deformable surgical scenes introduces a fundamental coupling problem: camera pose, non-rigid deformation, and appearance/geometry estimation must be estimated jointly, but supervision and priors are often unreliable. Most surgical neural/3DGS methods either assume static or known camera motion, or depend on relatively clean pose inputs to prevent drift in deformable sequences [8,10] . In RAMIS, camera pose is often calculated from robot joint kinematics, which can be inaccurate due to calibration errors and measurement noise, and may also not always be available in a clinical

FAQs

What problem does Track2Map address in robot-assisted minimally invasive surgery (RAMIS)?

Track2Map targets the difficulty of metric, stable deformable 3D reconstruction when camera pose priors are missing, noisy, or unavailable in clinical settings.

How does Track2Map estimate camera trajectory and the deformable 3D scene?

It jointly optimizes camera trajectory and an online deformable 3D Gaussian splatting representation directly from surgical stereo video.

What techniques does Track2Map use to reduce drift caused by tissue motion and ambiguous visual cues?

It uses dense 2D point tracks for track-anchored deformation initialization and uses track statistics to detect static camera periods and disentangle camera motion from scene deformation.

Jul 16, 2026

Upload by River Wang