Renji endoscopic submucosal dissection video data set for Esophagus Authors: Jinnan Chen, Xiangning Zhang, Tang Cao, Chunjiang Gu, Jinneng Wang, Zhao Li, Yiming Song, Liuyi Yang, Zhengjie Zhang, Qingwei Zhang, Dahong Qian, Xiaobo Li Published: 2025 ABSTRACT AbstractProgress in artificial intelligence has markedly improved computer-supported interventions, professional education for surgeons, and post-procedural analysis of operative recordings, considerably enhancing surgical proficiency and clinical outcomes. Phase recognition frameworks for endoscopic procedures utilizing deep learning methodologies depend critically on well-annotated comprehensive datasets. Our study introduces the Renji database containing endoscopic submucosal dissection (ESD) videos for esophageal lesions, comprising 25 procedural recordings with 141,909 phase-specific classifications jointly annotated by a specialized endoscopy team. To the best of our knowledge, this constitutes the inaugural publicly available collection of esophageal ESD videos with comprehensive phase annotations, and we believe this contribution will assist in establishing benchmarks for future esophageal ESD databases. The complete video repository and associated annotations have been made freely accessible to researchers through the Figshare repositorySubject terms: Cancer imaging, Oesophageal cancer Introduction Endoscopic submucosal dissection (ESD) is the preferred treatment method for squamous cell dysplasia/carcinoma of m1-m2 grade and high-grade intraepithelial neoplasia/adenocarcinoma of m1-SM1 grade without adverse features, with curative outcomes when negative margins are achieved1,2. Compared with traditional surgical methods, ESD offers less invasiveness and better post-operative life quality for individuals undergoing treatment3. ESD is not only a technically demanding procedure with a steep learning curve and certain associated complication rates, but also requires endoscopists to accurately assess lesion characteristics and determine the extent of the lesion through real-time imaging4. ESD training follows a progressive pathway from animal models to supervised clinical practice to independent operation5,6. Limited resources and expert guidance create challenges for novices, extending learning periods—especially in Western countries with fewer beginner-appropriate gastric lesions7. Research on ESD learning curves reveals that unsupervised practitioners improve in technical success and en bloc resection rates but show no advancement in R0 or curative resection rates7. This demonstrates that practice without expert guidance has clear limitations for developing critical clinical competencies in ESD. In surgical practice, each technical operation and clinical decision profoundly impacts patient outcomes. Compared to conventional procedural notes, operative video recordings deliver an enhanced objective documentation of surgical interventions, enabling practitioners to systematically evaluate technical performance and establish connections between operative findings and clinical endpoints8–10. Artificial intelligence (AI)-enhanced video assessment systems are transforming this field, providing operators with real-time guidance and analytical support11. This computational approach not only enables second-by-second analysis of procedural progression but also identifies potential complications, thereby improving intervention safety and care standards12,13. For early-career practitioners, AI technologies facilitate competency development through simulation-based training and detailed video assessments, accelerating skill acquisition14. These advantages have led to growing interest in computational surgical workflow analysis from both informatics specialists and healthcare practitioners. Currently, research on phase recognition in the treatment of early gastrointestinal cancers using ESD remains limited. Three studies have significantly advanced this field. The AI-based phase recognition systems they developed using surgical videos demonstrated accuracy rates exceeding 85% in their respective datasets and even in animal models. Nevertheless, these ESD video datasets remain unpublished, and the classification of procedural phases is still inadequate15–17. This is mainly reflected in the following aspects: annotations for preoperative detailed observation of lesions using magnifying endoscopy combined with chromoendoscopy, annotations for intraoperative traction procedures, and annotations for postoperative hemostasis management. Given the scarcity of openly accessible datasets for computerized identification of procedural phases in esophageal ESD, our research sought to develop a freely available resource specifically for ESD phase detection. This project enhances analytical capabilities for ESD procedures through meticulous annotating 25 esophageal ESD endoscopic recordings. The key contributions of this work encompass: Video collection: 25 complete operative recordings of esophageal ESD were systematically collected and processed Video collection: 25 complete operative recordings of esophageal ESD were systematically collected and processed Extensive frame annotation: A total of 141,909 individual frames were precisely categorized according to their corresponding procedural phases. We anticipate that providing open access to this comprehensive resource will significantly catalyze artificial intelligence developments in endoscopic submucosal dissection and facilitate successful implementation of computational analysis tools in real-world clinical practice. Extensive frame annotation: A total of 141,909 individual frames were precisely categorized according to their corresponding procedural phases. We anticipate that providing open access to this comprehensive resource will significantly catalyze artificial intelligence developments in endoscopic submucosal dissection and facilitate successful implementation of computational analysis tools in real-world clinical practice. Methods Data collection The investigation examined information gathered during routine endoscopic submucosal dissection operations performed at the Endoscopy Center of Renji Hospital (affiliated with Shanghai Jiao Tong University School of Medicine) between May and October 2024. All consecutive esophageal ESD procedures performed during this period were included in the dataset without any exclusion criteria to ensure comprehensive data representation. All subjects provided signed informed consent authorizing research use and subsequent open publication of their procedural recordings, following thorough de-identification of personal information to maintain privacy. The institutional Ethics Committee of Renji Hospital approved this research protocol (Approval Number: LY2024-271-B). The dataset comprised 25 ESD procedural videos recorded using the Olympus CV-260/290 endoscopic system with IMH-200 image processing unit, capturing footage at 1920 × 1080 pixel resolution and 50 frames per second (fps). Detailed specifications of endoscopic devices utilized during these procedures are presented in Table 1. All video material underwent comprehensive editing using Adobe Premiere to eliminate non-relevant imagery. Since the recording device begins capturing immediately upon endoscope activation, the initial footage may contain images of medical personnel, patient faces, and other non-procedural content, which were meticulously removed to protect patient privacy and focus solely on the endoscopic procedure. Sensitive identifying information such as patient identifiers and timestamps was meticulously obscured. To ensure procedural consistency and minimize technical variability, all procedures were performed by a single highly experienced senior endoscopist, Dr. Xiaobo Li, who has over 15 years of ESD experience and has performed more than 5,000 ESD procedures. Table 1The instrument used in the ESD surgery.InstrumentEndoscopeOlympus CD-GIF260, CF-GIF290Mucosal Cutting KnifeOlympus KD-612L/U, KD-655L/U, KD-620URMucosal Injection NeedleOlympus NM-400L-0423; NM-400U-0423Hot Biopsy ForcepsCOOK HDBF-2.4-230-SClamp ForcepsOlympus FD-410LR; FD-412LRSoft Tissue ClampMicro-Tech (Nanjing) ROCC-D-26-195-C; ROCC-D 26 -235-CHemostatic ClampBoston Scientific Corporation M00521242Methylene Blue Injection1.5% iodine fuel The instrument used in the ESD surgery. Comprehensive clinical information was systematically collected for each patient included in the study. The clinical data encompassed patient demographics including age and gender, preoperative endoscopic pathology results (when performed), intraoperative endoscopic morphology of the lesions, magnifying endoscopy with narrow-band imaging (ME-NBI) classification according to the Inoue classification system, and detailed postoperative histopathological findings. The postoperative pathological assessment included specimen Size, tumor Size, invasion depth, presence of ulceration, presence of lymphovascular invasion, horizontal and vertical resection margin status (positive or negative), histological tumor type, and molecular markers. Additionally, preoperative contrast-enhanced chest computed tomography (CT) was performed to evaluate TNM staging in conjunction with histopathological findings. All clinical information was systematically documented and organized for inclusion in the dataset. Annotations of video For systematic dataset classification, our research group established a structured framework for categorizing ESD interventions (Fig. 1, Table 2). This framework segments the ESD workflow into 8 distinct phases: (1) preparation: periods when practitioners configure endoscopic apparatus or switch instruments; (2) Estimation: preliminary examination of lesion properties utilizing white light examination, magnification techniques, and narrow-band imaging (NBI) with iodine chromoendoscopy; (3) Marking: the identification of the lesion periphery, with subsequent placement of electrocautery reference points approximately 5 mm external to the targeted region (4) Injection: the submucosal delivery of composite fluid containing physiological saline, adrenaline, or hyaluronic acid derivatives to elevate targeted tissue layers. (5) Incision: circumferential mucosal incision performed 5 mm beyond the previously marked zone. (6) ESD: systematic separation of the submucosal tissue from the deeper muscular layer until complete excision and extraction of the specimen, with bleeding management conducted via electrocoagulation or heated forceps as required, utilizing traction assistance when beneficial; (7) vessel treatment: addressing remaining blood vessels or bleeding sites on the post-resection surface using thermal biopsy instruments; (8) Clip: deployment of endoscopic clips for managing suspected perforations. Given our extensive video collection, recordings were downsampled to 1 fps, extracting the initial frame from each second for annotation purposes. Every extracted frame was assigned exclusively to one of the 8 procedural phases based on identification of the starting and ending frames for each respective phase. Representative visual illustrations showing each separate classification within the 8 recognized phases of endoscopic procedures. Table 2Definition of each phase of ESD for esophagus.PhaseDefinitionStart frameEnd framePreparationDuring the preparation stage, the operator continually switches instruments, adjusts viewing perspectives, and maintains clarity of the visual field.Start: End of the previous phaseEnd: Start of the next phaseEstimation1) Perform a comprehensive examination of the lesion utilizing white light endoscopic visualization.2) Position the lens of the endoscope close to the area of interest and utilize magnification endoscopy with NBI chromoendoscopy to assess patterns of microvasculature of the surface.3) Rinse the area completely with water, then insert a plastic spray catheter through the working channel to apply 20–30 ml of 1.5–2% iodine solution uniformly across the esophageal mucosa, enabling visualization of unstained areas (Lugol-voiding lesions) that may indicate dysplasia or neoplastic changes4) Accurately identify the location, dimensions, characteristics, and depth of infiltration of the lesion to gather essential preoperative information.Start: start of positioning endoscope and inspecting the lesionEnd: Endoscope is retrievedMarking1) Position a Dual Knife against the mucosal surface in proximity to the target lesion.2) Establish electrocoagulation markings roughly 0.5 cm beyond the peripheral edge of the lesion.Start: Start of applying the surgical knife to mucosal tissue surfaceEnd: Surgical knife is retrievedInjection1) Guide an injection needle via the working channel, reveal its tip, and advance it into the submucosal layer.2) Administer a sufficient quantity of solution (hyaluronic acid, dextrose solution, or sodium chloride) beneath the mucosa prior to withdrawing the injection apparatus.Start: Start of inserting injection needle into the submucosal layer;End: Injection needle is retrieved from the submucosal layerIncision1) The endoscopist adjusts settings on the electrosurgical generator, verifying adequate exposure of cutting edges on devices such as triangle-tipped, insulation-tipped, and dual knives.2) The endoscopist maneuvers the knife to create a circular incision approximately 0.5 cm outside the previously established markings.Start: Start pf using knife to cut along the margin of the lesion or marked edgeEnd: the knife stop cut edge of lesion and retrievedESD1) The endoscopist directs the knife to perform submucosal dissection of the target lesion.2) Techniques for achieving hemostasis during the procedure include utilizing the electrosurgical generator’s tip for coagulative effect and selectively employing hemostatic forceps to secure, elevate, and electrically coagulate vessels for bleeding management.3) This stage also involves surgical manipulation methods including clip-and-line and clip-and-snare techniques, which enhance tissue handling and visibility essential for the intervention.Start: Commencement of knife manipulation for submucosal tissue separation, deployment of electrosurgical unit’s tip/hemostatic tools for controlling hemorrhage, or application of clips for traction of tissue.End: The knife is retrieved, bleeding is successfully controlled, or traction maneuvers are finalized.Vessel treatmentFollowing separation of the lesion from the esophageal mucosa, any ongoing bleeding points or visible vessels within the wound surface are addressed using either the electrosurgical device or hemostatic forceps to achieve effective hemostasis.Start: Start of applying electrosurgical unit’s tip or hemostatic forceps to hemorrhage points or vascular structuresEnd: Withdrawal of the electrosurgical device’s tip or hemostatic instrument.ClipsIn cases of wounds with deep infiltration, hemostatic clips are employed to facilitate complete closure.Start: Start of positioning hemostatic clips onto mucosal tissue edgesEnd: the clip has been successfully applied Definition of each phase of ESD for esophagus. During the preparation stage, the operator continually switches instruments, adjusts viewing perspectives, and maintains clarity of the visual field. Start: End of the previous phase End: Start of the next phase 1) Perform a comprehensive examination of the lesion utilizing white light endoscopic visualization. 2) Position the lens of the endoscope close to the area of interest and utilize magnification endoscopy with NBI chromoendoscopy to assess patterns of microvasculature of the surface. 3) Rinse the area completely with water, then insert a plastic spray catheter through the working channel to apply 20–30 ml of 1.5–2% iodine solution uniformly across the esophageal mucosa, enabling visualization of unstained areas (Lugol-voiding lesions) that may indicate dysplasia or neoplastic changes 4) Accurately identify the location, dimensions, characteristics, and depth of infiltration of the lesion to gather essential preoperative information. Start: start of positioning endoscope and inspecting the lesion End: Endoscope is retrieved 1) Position a Dual Knife against the mucosal surface in proximity to the target lesion. 2) Establish electrocoagulation markings roughly 0.5 cm beyond the peripheral edge of the lesion. Start: Start of applying the surgical knife to mucosal tissue surface End: Surgical knife is retrieved 1) Guide an injection needle via the working channel, reveal its tip, and advance it into the submucosal layer. 2) Administer a sufficient quantity of solution (hyaluronic acid, dextrose solution, or sodium chloride) beneath the mucosa prior to withdrawing the injection apparatus. Start: Start of inserting injection needle into the submucosal layer; End: Injection needle is retrieved from the submucosal layer 1) The endoscopist adjusts settings on the electrosurgical generator, verifying adequate exposure of cutting edges on devices such as triangle-tipped, insulation-tipped, and dual knives. 2) The endoscopist maneuvers the knife to create a circular incision approximately 0.5 cm outside the previously established markings. Start: Start pf using knife to cut along the margin of the lesion or marked edge End: the knife stop cut edge of lesion and retrieved 1) The endoscopist directs the knife to perform submucosal dissection of the target lesion. 2) Techniques for achieving hemostasis during the procedure include utilizing the electrosurgical generator’s tip for coagulative effect and selectively employing hemostatic forceps to secure, elevate, and electrically coagulate vessels for bleeding management. 3) This stage also involves surgical manipulation methods including clip-and-line and clip-and-snare techniques, which enhance tissue handling and visibility essential for the intervention. Start: Commencement of knife manipulation for submucosal tissue separation, deployment of electrosurgical unit’s tip/hemostatic tools for controlling hemorrhage, or application of clips for traction of tissue. End: The knife is retrieved, bleeding is successfully controlled, or traction maneuvers are finalized. Following separation of the lesion from the esophageal mucosa, any ongoing bleeding points or visible vessels within the wound surface are addressed using either the electrosurgical device or hemostatic forceps to achieve effective hemostasis. Start: Start of applying electrosurgical unit’s tip or hemostatic forceps to hemorrhage points or vascular structures End: Withdrawal of the electrosurgical device’s tip or hemostatic instrument. In cases of wounds with deep infiltration, hemostatic clips are employed to facilitate complete closure. Start: Start of positioning hemostatic clips onto mucosal tissue edges End: the clip has been successfully applied Data Records Our dataset underwent rigorous quality assurance through multiple verification protocols. The comprehensive archive, now publicly available as a compressed repository on Figshare18, encompasses the original 25 endoscopic videos along with 141,909 phase-labeled sequential frames. For each procedural phase identified within individual recordings, text-based annotations document the precise starting and terminal frames. Figure 2 presents the quantitative distribution of phase classifications across the entire ESD video collection. To facilitate easier access and verification, the dataset has been reorganized into 25 separate compressed files, with each representing an individual clinical case available for independent download. Upon accessing the Figshare repository, researchers will encounter a systematically structured organization where each case-specific compressed file contains the unmodified endoscopic video recording (stored as.mp4 file), the associated phase identification document (in.txt format), and detailed instrument specification documentation (in.doc format). Additionally, comprehensive clinical information for all cases is provided as a separate file (in.xls format named “clinical information”) that can be downloaded independently from the Figshare repository. Furthermore, Fig. 3 depicts the aggregate time duration of annotations for specific phases throughout the complete database. Aggregate time measurements (presented in seconds) of all recorded procedural stages throughout the entire set of 25 annotated ESD video sequences. Visual depiction of phase distribution across 25 ESD interventions. The duration of recordings differs between cases and has been normalized to facilitate comparative visualization. Technical Validation Dataset characteristics The collection encompasses 25 individual procedure videos with accompanying phase classification labels, gathered from an equal number of clinical cases. Participants had a mean age of 60.35 years (SD = 9.02). The demographic distribution included 10 women and 15 men, all of Chinese nationality. Data validation Our classification protocol implemented a three-step verification process. In the preliminary stage, two clinically trained data specialists (Tang Cao and Chunjiang Gu), who were not involved in the surgical procedures and had no prior exposure to the video content, conducted separate and independent classification of two randomly selected video sequences from different patients, containing 3118 and 5184 frames respectively, following established annotation protocols. The degree of agreement between annotators was measured using Cohen’s-Kappa statistical analysis, resulting in coefficients of 0.955 and 0.959 respectively (p < 0.001), demonstrating exceptional inter-rater consistency in applying the classification framework. Upon validation of this reliability, both annotators proceeded to independently categorize the remaining 23 procedural recordings. Following completion of all annotations, thorough quality verification was conducted by two senior endoscopists with substantial clinical expertise (Xiaobo Li and Qingwei Zhang) who separately reviewed all annotated procedures. In cases where discrepancies were identified between the initial annotators, both senior endoscopists reached consensus through discussion and directly implemented corrections to ensure annotation accuracy and consistency across the entire dataset. Limitations of the datasets A significant constraint of our dataset involves its single-institution origin and single-operator design, as all procedures were performed by one experienced endoscopist (Dr. Xiaobo Li), potentially limiting procedural technique variability that would exist across multiple practitioners with diverse training backgrounds and expertise levels. This standardization, while beneficial for consistency, may restrict the model’s ability to generalize across different operator styles and technical approaches. Additionally, uniform equipment specifications including particular endoscopic instruments and lighting arrangements from a single medical center further limit acquisition parameter diversity. Subsequent initiatives will prioritize enlarging our database with multi-operator contributions and developing institutional partnerships with diverse clinical centers. Notwithstanding these acknowledged limitations, as the first publicly accessible resource specifically focused on esophageal ESD procedural documentation currently available, we anticipate this dataset will significantly contribute to the advancement of computer-assisted identification of procedural phases in endoscopic submucosal dissection interventions. Acknowledgements This work was financially supported by the Medical Engineering Cross Research Fund of Shanghai Jiaotong University (YG2025ZD01). Author contributions The development and coordination of the Renji dataset initiative was led by Jinnan Chen, who additionally performed data evaluation, formulated classification guidelines, and prepared the initial manuscript. Xiangning Zhang conducted review of annotated data and created visual representations. Independent phase classification and labeling were carried out by both Chunjiang Gu and Tang Cao. Jineng Wang formulated the documentation for annotation methodology. Procedural video acquisition was managed by Liuyi Yang and Yiming Song, who also contributed written content to the manuscript. Zhao Li oversaw medical device verification and graphic presentation. Zhengjie Zhang supervised the analysis of annotated information and project management. The conceptual structure for phase identification was created and annotation precision was validated by Qingwei Zhang. Dahong Qian provided oversight for annotation concept formulation and project coordination. Xiaobo Li performed the ESD interventions, verified the annotated materials, and provided overall scientific direction for the research initiative. Data availability The dataset is publicly available on Figshare (10.6084/m9.figshare.28777862) and can be freely downloaded. No custom code was developed for this study. Code availability No novel code was used in the construction of Renji dataset. Competing interest The authors affirm that they do not have any known conflicting financial interests or personal relationships that could be perceived as influencing the work reported in this paper. Footnotes Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. These authors contributed equally: Jinnan Chen, Xiangning Zhang, Tang Cao. Contributor Information Qingwei Zhang, Email: zhangqingweif@hotmail.com. Dahong Qian, Email: dahong.qian@sjtu.edu.cn. Xiaobo Li, Email: lxb_1969@163.com. References Associated Data Data Availability Statement The dataset is publicly available on Figshare (10.6084/m9.figshare.28777862) and can be freely downloaded. No custom code was developed for this study. No novel code was used in the construction of Renji dataset.