Point-Guided Latent Diffusion Model for Novel View Synthesis in Laparoscopic Liver Surgery.

Abstract

Despite recent progress in diffusion-based video synthesis, synthesizing accurate novel views from sparse input frames in laparoscopic liver surgery remains challenging due to occlusions, complex shape of anatomical structures and limited field of views. We propose point-guided latent diffusion model, specifically designed for generating high-quality intermediate frames in laparoscopic liver surgery from only the first and last video frames. Our method leverages the powerful generative capability of latent diffusion models combined with geometric cues from 3D point clouds reconstructed via dense stereo matching. To robustly handle occlusions and shape deformation, we use an adaptive camera trajectory planning strategy based on next-best-view algorithms. Furthermore, we introduce a spatial-transformer enhanced decoder to effectively preserve detailed anatomical features from reference frames and minimize visual artefacts in generated views. Extensive experiments on the clinically relevant P2ILF challenge dataset validate our method's effectiveness and superior performance in producing visually coherent and structurally accurate novel views, highlighting its ability for enhancing the quality of surgical scene reconstruction.