BronchoGAN: Anatomically consistent and domain-agnostic image-to-image translation for video bronchoscopy
Ahmad Soliman, Ron Keuth, Marian Himstedt · 2025
Read BronchoGAN: Anatomically consistent and domain-agnostic image-to-image translation for video bronchoscopy by Ahmad Soliman, Ron Keuth, Marian Himstedt (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-01387.txt.
Introduction
Ahmad Soliman and co-authors build BronchoGAN, a system that fabricates realistic bronchoscopy images to feed data-hungry training sets. Real images of the airways, taken through a scope, are scarce and hard to share, yet doctors and algorithms need many of them. Their translator takes images from easier sources, such as virtual fly-throughs built from CT scans or lab phantoms, and repaints them to look like genuine human airway footage.
Two ideas carry the design. Depth maps made by a general vision model serve as a bridge representation that behaves the same across sources. Anatomical constraints force the openings of the bronchial branches to stay in place between input and output, which the team verifies with overlap scores that improved by up to 0.43. Tests showed convincing translations from every source tried. The authors aim the tool at building large public training sets the field currently lacks.
Catalog record
| CREATOR | Ahmad Soliman, Ron Keuth, Marian Himstedt |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-01387
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
BronchoGAN: Anatomically consistent and domain-agnostic image-to-image translation for video bronchoscopy Ahmad Soliman, Ron Keuth, Marian Himstedt Abstract Purpose: The limited availability of bronchoscopy images makes image synthesis particularly interesting for training deep learning models. Robust image translation across different domains - virtual bronchoscopy, phantom as well as in-vivo and ex-vivo image data - is pivotal for clinical applications. Methods: This paper proposes BronchoGAN introducing anatomical constraints for image-to-image translation being integrated into a conditional GAN. In particular, we force bronchial orifices to match across input and output images. We further propose to use foundation model-generated depth images as intermediate representation ensuring robustness across a variety of input domains establishing models with substantially less reliance on individual training datasets. Moreover our intermediate depth image representation allows to easily construct paired image data for training. Results: Our experiments showed that input images from different domains (e.g. virtual bronchoscopy, phantoms) can be successfully translated to images mimicking realistic human airway appearance. We demonstrated that anatomical settings (i.e. bronchial orifices) can be robustly preserved with our approach which is shown qualitatively and quantitatively by means of improved FID, SSIM and dice coefficients scores. Our anatomical constraints enabled an improvement in the Dice coefficient of up to 0.43 0.43 for synthetic images. Conclusion: Through foundation models for intermediate depth representations, bronchial orifice segmentation integrated as anatomical constraints into conditional GANs we are able to robustly translate images from different bronchoscopy input domains. BronchoGAN allows to incorporate public CT scan data (virtual bronchoscopy) in order to generate large-scale bronchoscopy image datasets with realistic appearance. BronchoGAN enables to bridge the gap of missing public bronchoscopy images. 2023 CNN Convolutional Neural Network Lite R-ASPP Lite Reduced Atrous Spatial Pyramid Pooling ResNet Residual Network ReLU Rectified Linear Unit CAM Class Activation Map HMM Hidden Markov Model NLL Negative Log Likelihood ECE Estimated Calibration Error MAP Maximum A Posteriori VB virtual bronchoscopy PB phantom bronchoscopy RB real bronchoscopy ICU Intensive Care Unit EMT Electromagnetic Tracking SLAM Simultaneous Localization And Mapping SfM Structure from Motion MSER Maximally Stable Extremal Regions SVM Support Vector Machine keywords Video Bronchoscopy; Image-guided navigation; Depth estimation; Segmentation 1 Introduction Bronchoscopy is a vital procedure frequently performed in pulmonology clinics, with the largest number of cases involving the examination and biopsy of patients suspected of having lung cancer. Beyond this primary application, bronchoscopy is also used for various other medical indications, such as monitoring patients with chronic obstructive pulmonary disease (COPD), performing biopsies in pneumonia cases, and addressing acute respiratory problems in intensive care unit (ICU) settings. While traditional computer vision has been integrated into several certified bronchoscopy-based intervention systems, the application of deep learning-based approaches holds immense potential to enhance these existing systems and enable novel approaches, particularly for navigation assistance as already motivated in keuth2024airway ; sganga_autonomous_2019 . An adequate integration of deep learning-based models in medical products requires superior generalization capabilities which can only be obtained from a large patient cohort covering variability in anatomy and optical appearance. This is currently hampered by the lack of publicly available bronchoscopy images which are rarely recorded in clinical routine. Fortunately, large-scale public thorax CT scan datasets national2011NLST comprising tens of thousands of volumes are available, offering a valuable resource. In addition, phantoms are frequently utilized in research and training visentini-scarzanella_deep_2017 , enabling the simulation of varying camera poses, motion dynamics, and lightning conditions. This paper attempts to close the aforementioned gap by means of image-to-image translation rendering realistic bronchoscopy images from virtual bronchoscopy (VB) constructed from CT scans and phantom images. Medical image translation has been investigated in-depth in recent years. Conditional GANs (cGAN) have been used for e.g. translating PET and CT zhou2023gan , retinal images iqbal2018generative , chest radiographs popp2024adaptive and PET to MRI data chen2024ambient . Wang et al. propose pix2pix wang2018pix2pixhd using paired image data while cycleGAN zhu2017unpaired and CUT park2020contrastive can handle unpaired image sets. In atli2024i2i authors propose an adversarial model for multi-modal synthesis integrating channel-mixed Mamba blocks into a CNN backbone, using selective state space modeling. More recently, denoising diffusion models (DDM) have been investigated for image translation promising more stable training and mode coverage compared to GANs at the cost of increased computational costs kazerouni2023diffusion ; arslan2024self . In arslan2024self , authors introduce SelfRDB which further improves source-to-target alignment using a novel forward process, adaptive noise scheduling, and recursive sampling for enhanced accuracy. Exhaustive surveys are provided by zhou2023gan ; dayarathna2024deep . Image translation has also been used in conjunction with bronchoscopy. Existing approaches banach2021 ; xu2024depth ; yang2024adversarial ; ozbey2023unsupervised primarily focus on GAN-based depth estimation motivated as pivotal fundamental for vision-only guidance. Only banach2021 proposes 3cGAN translating images between multiple input domains - virtual bronchoscopy, real bronchoscopy (RB) and depth - using individual generators and discriminators for each domain. However, also 3cGAN focuses on synthesizing depth images and rather incorporates VB data to ease ground-truth depth image acquisition. To our best knowledge, current state of the art banach2021 ; xu2024depth ; yang2024adversarial ; ozbey2023unsupervised in bronchoscopy image translation misses two fundamental requirements: 1. Anatomical consistency: None of the existing approaches incorporates anatomical constraints, in particular bronchial orifices, in image translation even though the preservation of anatomical structures is indispensable for clinical applications. 2. Domain-robustness: Existing approaches are limited to specific input image domains with limited capabilities to generalize from unseen data (e.g. visual appearance across VB, phantom and RB images) While segmentation priors have already been integrated in other medical applications (e.g. hamghalam2024medical ), it is missing in state-of-the-art approaches to bronchoscopy image translation. Leveraging VB images from large-scale CT scan datasets national2011NLST presents significant opportunities for deep learning-based models, particularly in accommodating the wide variety of lung anatomies incorporated by bronchial orifice segmentation keuth2023weakly . The challenge of domain-robustness is omnipresent in medical imaging particularly for applications with limited readily-available data. Substantial investigations have been done in the field of domain adaptation and generalization guan2021domain as well domain randomization dinkar2022automatic . Another option is the introduction of an intermediate image representation mitigating individual input domain variations and thus enabling domain-agnostic image translation. A first approach to this has been presented in visentini-scarzanella_deep_2017 . Here, the authors propose to translate domain-specific RB images to domain-agnostic VB images in order to synthesize depth images visentini-scarzanella_deep_2017 . We take this idea further by using depth images as intermediate representations to synthesize real from virtual bronchoscopy images while also addressing a fundamental limitation of visentini-scarzanella_deep_2017 : the ability to achieve this intermediate representation as zero-shot inference. Deep learning-based depth estimation has been extensively studied in numerous works (survey: ming2021deep ). Self-supervised approaches godard2019digging have demonstrated superior performance compared to supervised methods, while a significant breakthrough for zero-shot inference has been achieved with the advent of foundation models such as DINO oquab2023dinov2 and DepthAnything yang2024depth . Depth images have found applications in several endoscopy-related tasks cui2024surgical , including bronchoscopy guo2024cgan , since they provide a domain-agnostic representation, where the scene semantic is preserved. However, these applications have primarily motivated on leveraging depth data for direct benefits in 3D-reconstruction as well as visual mapping and tracking, except for wang_bronchial_2021 ; xu2024depth . In this work, we adopt depth images as an intermediate, domain-agnostic representation to address challenges associated with limited training