Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery: A Controlled Study of Vision Foundation Models

Lincoln Spencer, Song Wang, Chen Chen · 2026

Read Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery: A Controlled Study of Vision Foundation Models by Lincoln Spencer, Song Wang, Chen Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-10514.txt.

Introduction

Spencer, Wang, and Chen test cataract phase tagging when labeled video runs thin. Each visual encoder shares one time model on SICS-155 with 19 phases, so only the eyes differ. Supervised nets face large self taught foundation rivals in the same setup. DINOv3 leads the pack at about 83 percent accuracy with an edit score of 87. Unlabeled cataract clips plus light tuning help at times and hurt at others.

Catalog record

CREATORLincoln Spencer, Song Wang, Chen Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE24643
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.10514

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Data-Efficient Surgical Phase Segmentation in Small-Incision Cataract Surgery: A Controlled Study of Vision Foundation Models Lincoln Spencer, Song Wang, Chen Chen Abstract Surgical phase segmentation is central to computer-assisted surgery, yet robust models remain difficult to develop when labeled surgical videos are scarce. We study data-efficient phase segmentation for manual small-incision cataract surgery (SICS) through a controlled comparison of visual representations. To isolate representation quality, we pair each visual encoder with the same temporal model (MS-TCN++) under identical training and evaluation settings on SICS-155 (19 phases). We compare supervised encoders (ResNet-50, I3D) against large self-supervised foundation models (DINOv3, V-JEPA2), and use a cached-feature pipeline that decouples expensive visual encoding from lightweight temporal learning. Foundation-model features improve segmentation performance in this setup, with DINOv3 ViT-7B achieving the best overall results (83.4% accuracy, 87.0 edit score). We further examine cataract-domain transfer using unlabeled videos and lightweight adaptation, and analyze when it helps or hurts. Overall, the study indicates strong transferability of modern vision foundation models to surgical workflow understanding and provides practical guidance for low-label medical video settings. The project website is available at: https://sl2005.github.io/DataEfficient-sics-phase-seg/ 1 Introduction Surgical workflow understanding from video can support intraoperative assistance, automatic documentation, skill assessment, and safety monitoring. In practice, however, progress is often limited by annotation cost: phase segmentation requires dense frame-level labels over long procedures, which are expensive to obtain at scale 13 . This challenge is especially pronounced for manual small-incision cataract surgery (SICS), which is widely performed in low-resource settings where annotation capacity and specialized infrastructure may be limited 11 ; 12 . Phase segmentation is also intrinsically a joint spatial–temporal task. Strong performance depends not only on recognizing instruments and anatomy in individual frames, but also on modeling long-range workflow structure, repeated sub-steps, and brief transitions between clinically similar phases. Motivated by these constraints, we position this work as a systematic controlled study of data-efficient surgical video understanding, rather than as a new temporal architecture. We leverage self-supervised visual foundation models pretrained on large unlabeled corpora to provide strong frame-level features with limited task-specific supervision. To keep comparisons interpretable and deployment-relevant, we fix the temporal learner to MS-TCN++ 6 and vary only the feature extractor. This design (i) isolates representation quality by controlling temporal architecture, optimization, and splits, and (ii) matches a practical workflow where heavy encoder inference is performed once offline (feature caching) and the temporal head is trained quickly on limited labels. Section 3 details this shared protocol. The contributions of this paper are summarized as follows. • We present a controlled benchmark for surgical phase segmentation where supervised encoders and self-supervised foundation models are evaluated under the same cached-feature workflow, shared MS-TCN++ head, and stratified splits. • We study empirical representation scaling and deployment trade-offs across encoder families, while carefully interpreting scaling trends because model size and upstream pretraining scale co-vary in released checkpoints. • We analyze CataractFT , an exploratory cataract-domain transfer workflow combining self-supervised continuation and optional LoRA adaptation before frozen feature extraction on SICS-155 8 , and characterize when transfer is helpful versus harmful. Practical insight. For teams building surgical video systems with limited labels, foundation-model features can be a strong drop-in alternative to supervised encoders when the temporal model is held fixed and evaluation includes both frame-level and segment-level metrics. Small Data Statement. This study is small data in two senses. First, SICS-155 contains only 155 videos, and each sample requires dense frame-level labeling across 19 phases; for rare phases, the effective sample size is much smaller than the raw video count. Second, the target domain is operationally narrow (manual SICS in low-resource settings), making large, task-matched labeled corpora difficult and costly to assemble. To address this setting, we explicitly design for data efficiency. We reuse representations from large self-supervised visual encoders pretrained on external unlabeled data, rather than training video models from scratch on SICS. We keep the temporal backbone fixed so improvements can be attributed to representation quality, use stratified 5-fold cross-validation to reduce split sensitivity, cache features to enable efficient temporal training, and explore CataractFT (self-supervised continuation with optional parameter-efficient adaptation on unlabeled cataract-domain video) as a practical way to exploit additional data without requiring new SICS labels. 2 Method 2.1 Problem setup Given a surgical video with T frames, the goal is to assign a phase label y_{t}\in\{1,\dots,C\} to each frame t . We study how the per-frame representation \mathbf{f}_{t}\in\mathbb{R}^{d} affects downstream temporal segmentation. 2.2 Overall Framework End-to-end training of very large visual encoders on long surgical videos is often computationally prohibitive and prone to overfitting, especially on long-tailed datasets like SICS-155. Instead of proposing a new temporal architecture, we adopt a deployment-oriented factorization: a large visual encoder produces frozen per-frame features, and a lightweight temporal model maps cached features to phase labels. This decoupling enables controlled comparisons, reduces downstream training cost, and matches realistic deployment where expensive feature extraction is run once offline. Our base benchmark pipeline caches features from off-the-shelf encoders and trains a shared MS-TCN++ head on those tensors. We then study CataractFT as an exploratory extension : optional self-supervised continuation on 1,000 unlabeled Cataract-1K videos using each backbone’s native objective (DINO self-distillation or JEPA masked prediction), followed by optional LoRA phase-classification adaptation 5 on 56 annotated Cataract-1K and 101 Cataract-101 surgeries 4 ; 9 . In our implementation, LoRA uses rank r=16 and \alpha=32 for up to 50 epochs. The adapted encoder is then frozen for dense SICS feature extraction, and only the same MS-TCN++ head is trained on SICS-155. Figure 1 summarizes the controlled pipeline, including the shared cached-feature benchmark path and the optional CataractFT transfer branch. Figure 1 : Controlled study pipeline. Encoders are frozen after optional cataract-domain SSL continuation and LoRA adaptation, features are cached once, and the same MS-TCN++ head is trained for all encoders. 2.3 Feature extractors We compare two encoder groups: • Supervised encoders: ResNet-50 pretrained on ImageNet 3 and I3D 2 pretrained on Kinetics. • Self-supervised foundation models: DINOv3 10 (image ViTs) and V-JEPA2 1 (video ViTs). Frame-level features (DINOv3). For DINOv3, we extract one feature per frame with a single forward pass of the image encoder and use the pooled representation as \mathbf{f}_{t} . Clip-to-frame features (V-JEPA2). V-JEPA2 operates on clips (e.g., 64 frames). To obtain per-frame features, we use a sliding-window procedure and assign each clip embedding to its center frame. For tractability (especially ViT-g at 384px), we extract at stride 4 and linearly interpolate back to full frame rate before training MS-TCN++; this may smooth rapid transitions. Implementation details for feature caching, optimization, and the fixed MS-TCN++ configuration are deferred to Section 3.4 . 3 Experiments 3.1 Dataset We evaluate on SICS-155 8 , which contains 155 annotated SICS procedures with frame-level labels over 19 phases. The released train/validation portion available for this study contains 115 labeled videos (32 BL, 50 BN, 33 SD), where BL, BN, and SD denote the three procedure-type prefixes used in the released annotations; the hidden test set is not used. This public subset already reflects key dataset challenges: median video duration is 6.4 minutes (range 2.5–15.0), cautery accounts for only 1.7% of frames and appears in 48/115 videos, cortical wash occupies 13.8% of frames and appears in every video, and short

Read the complete text