EndoGeDE: Generalizable Monocular Depth Estimation with Mixture of Low-Rank Experts for Diverse Endoscopic Scenes

Liangjing Shao 1,2,3, Chenkang Du 1,2, Benshuang Chen 1,2, Xueli Liu 4, Xinrong Chen 1,2 1 College of Biomedical Engineering, Fudan University 2 Shanghai Key laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University 3 Department of Electronic Engineering, The Chinese · 2025

Read EndoGeDE: Generalizable Monocular Depth Estimation with Mixture of Low-Rank Experts for Diverse Endoscopic Scenes by Liangjing Shao 1,2,3, Chenkang Du 1,2, Benshuang Chen 1,2, Xueli Liu 4, Xinrong Chen 1,2 1 College of Biomedical Engineering, Fudan University 2 Shanghai Key laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University 3 Department of Electronic Engineering, The Chinese (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-01206.txt.

Introduction

Estimating depth from a single camera view could give cheap three-dimensional vision inside the body during endoscopy. Liangjing Shao, with four co-authors, tackles the wide range of lighting and tissue scenes that make that task hard. Their framework learns without labeled depth data, a self-supervised setup.

A new block-wise mixture of dynamic low-rank experts fine-tunes a foundation model with few added parameters, choosing experts to suit each block's features. A training scheme also handles shifting brightness and reflections from wet tissue. Tests on the SCARED data set and a simulated colon data set beat prior leading methods. Shao and the team aim for depth sensing that stays reliable across diverse endoscopic scenes.

Catalog record

CREATORLiangjing Shao 1,2,3, Chenkang Du 1,2, Benshuang Chen 1,2, Xueli Liu 4, Xinrong Chen 1,2 1 College of Biomedical Engineering, Fudan University 2 Shanghai Key laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University 3 Department of Electronic Engineering, The Chinese
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE44 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-01206

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

EndoGeDE: Generalizable Monocular Depth Estimation with Mixture of Low-Rank Experts for Diverse Endoscopic Scenes Liangjing Shao 1,2,3, Chenkang Du 1,2, Benshuang Chen 1,2, Xueli Liu 4, Xinrong Chen 1,2 1 College of Biomedical Engineering, Fudan University 2 Shanghai Key laboratory of Medical Image Computing and Computer Assisted Intervention, Fudan University 3 Department of Electronic Engineering, The Chinese University of Hong Kong 4 ENT Institute and Department of Otolaryngology, Eye & ENT Hospital of Fudan University Abstract Self-supervised monocular depth estimation is a significant task for low-cost and efficient 3D scene perception in endoscopy. In recent years, a series of methods are proposed to address the illumination inconsistency, while certain works also focus on the generalization of the model by efficiently finetuning the foundation models. However, the variety of illumination conditions and scene features is still the primary challenges for depth estimation in endoscopic scenes. In this work, a self-supervised framework is proposed for monocular depth estimation in diverse endoscopy. Firstly, considering the diverse features in endoscopic scenes with different tissues, a novel block-wise mixture of dynamic low-rank experts is proposed to efficiently finetune the foundation model for endoscopic depth estimation. In the proposed module, based on the input feature, different experts with a small amount of trainable parameters are adaptively selected for weighted inference, from low-rank experts which are allocated based on the generalization of each block. Moreover, a novel self-supervised training framework is proposed to jointly cope with brightness inconsistency and reflectance interference. The proposed method outperforms state-of-the-art works on SCARED dataset and SimCol dataset. Furthermore, the proposed network also achieves the best generalization based on zero-shot depth estimation on C3VD, Hamlyn and SERV-CT dataset. The outstanding performance of our model is further demonstrated with 3D reconstruction and ego-motion estimation. The proposed method could contribute to accurate endoscopy for minimally invasive measurement and surgery. The evaluation codes will be released upon acceptance, while the demo videos can be found on https://endo-gede.netlify.app/ . 1 Introduction Endoscopy, as one of primary minimally invasive surgical procedures, is widely used in the medical treatment due to its minor additional trauma and short recovery period. However, due to the limited field of view and 2D vision of endoscopes, 3D scene perception has become a significant task for efficient and accurate endoscopy. Depth estimation, as one of the most common 3D vision techniques which has been applied in industrial scenes ( Liu et al. 2024 ) , underwater scenes ( Ye et al. 2023 ) , outdoor driving scenes ( Feng et al. 2023 ; Zhu et al. 2025 ) and indoor scenes ( Wei et al. 2024 ) , has also been a primary 3D scene perception method in endoscopy. Due to the high cost of obtaining depth maps ( Yang et al. 2024c ) and limited inference efficiency of stereo matching ( Wu et al. 2025 ) in endoscopy, self-supervised monocular depth estimation is the main-stream method for endoscopic scenes. Although numerous self-supervised methods, such as MonoViT ( Zhao et al. 2022 ) and LiteMono ( Zhang et al. 2023a ) , have been proposed for monocular depth estimation for natural scenes based on deep learning, there are two critical challenges for the application of such methods in endoscopy. Firstly, illumination conditions in endoscopic scenes often causes brightness and reflectance inconsistency. Moreover, the performance and generalization of the depth estimation model will be highly limited by the variety of scene features due to different surgical tasks and tissues, which is shown in Fig. 1 . Figure 1 : Challenges (left) and performance comparison (right) for depth estimation in diverse endoscopic scenes. The size of the circle denotes the number of trainable parameters in the corresponding depth estimation network. To deal with the illumination inconsistency, Shao et al. ( Shao et al. 2022 ) propose to utilize constraint based on the appearance flow to alleviate brightness inconsistency. Meanwhile, intrinsic image decomposition is also a useful technique to filter the interference of the light, which is also applied in endoscopy ( Li et al. 2024 ) . However, as Fig. 1 shows, the depth at some positions with high brightness on the surface will be estimated as a higher value than the ground truth, while some light spots from reflectance on the surface will highly influence the depth value at the corresponding position. With the development of foundation models for depth estimation, a series of methods are proposed to efficiently finetune the foundation model for adaptation to the endoscopic scenes, including Surgical-DINO ( Cui et al. 2024a ) , EndoDAC ( Cui et al. 2024b ) and DARES ( Zeinoddin et al. 2024 ) . However, most of the existing finetuned foundation models for endoscopic depth estimation are based on Low-Rank Adaptation (LoRA) ( Hu et al. 2022 ) , the performance of which may be constrained by the feature diversity in diverse endoscopic scenes. To this end, a novel self-supervised framework, EndoGeDE, is proposed to boost the performance and generalization of monocular depth estimation in diverse endoscopic scenes. First of all, in the proposed framework, an end-to-end training pipeline based on the integration of a novel intrinsic image registration and brightness constraint ( Shao et al. 2022 ) is utilized to jointly deal with brightness inconsistency and light interference. Specifically, the invariance of the reflectance image to illumination changes and the strong correlations in the gradient domain ( Kim et al. 2016 ) are considered in the proposed intrinsic image decomposition. Furthermore, a novel parameter-efficient finetuning method, Block-wise Mixture of Low-Rank Experts (BW-MoLE), is proposed to finetune the foundation model for adaptive depth estimation in endoscopy based on diverse scene features, which is built upon LoRA and Mixture of Experts (MoE). Note that the low-rank experts are allocated based on the training quality of different Transformer blocks in the encoder of foundation model. The main contribution of this work can be summarized as the following: 1. A novel parameter-efficient finetuning method with block-wise mixture of dynamic low-rank experts is proposed to adaptively utilize the low-rank experts for depth estimation based on diverse features, which can boost the generalization of the model. 2. A novel self-supervised framework for depth estimation is proposed to jointly deal with the inconsistency of brightness and reflectance in endoscopic scenes, which can benefit the depth estimation from observations with diverse illumination conditions. 3. The proposed method is evaluated by self-supervised depth estimation on SCARED ( Allan et al. 2021 ) and SimCol ( Rau et al. 2024 ) dataset, as well as zero-shot depth estimation on C3VD ( Bobrow et al. 2023 ) , Hamlyn ( Recasens et al. 2021 ) and SERV-CT ( Edwards et al. 2022 ) dataset. Our method achieves state-of-the-art performance compared with existing works. Moreover, the proposed method is applied to sim-to-real test ( Azagra et al. 2023 ) and 3D reconstruction. 2 Related Work In this section, the related works for self-supervised monocular depth estimation in endoscopy are reviewed, as well as the application of foundation models in endoscopic depth estimation. 2.1 Self-supervised Depth Estimation for Endoscopy Based on the method proposed by Zhou et al. ( Zhou et al. 2017 ) , Turan et al. ( Turan et al. 2018 ) proposed the

Read the complete text