Scale-Aware Self-Supervised Learning for Segmentation of Small and Sparse Structures

Jorge Quesada, Ghassan AlRegib · 2026

Read Scale-Aware Self-Supervised Learning for Segmentation of Small and Sparse Structures by Jorge Quesada, Ghassan AlRegib (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-18619.txt.

Introduction

Jorge Quesada and Ghassan AlRegib teach vision tools to mark small, sparse things in shots with few hand marks to learn from. Self-taught tools learn form from raw shots with no tags. That path works well for big, plain zones of a shot. It works far less well when the thing to mark is small, thin, or spread in specks.

A speck can wash out when the tool shrinks the shot to learn its broad form. Their plan makes the tool scale-aware, so fine specks stay in view while it learns. That care should lift marks on roads, crops, and scans where the key zone is tiny. Good marks with few hand tags could cut cost in fields from health to maps. The work names a blind spot in self-taught vision and sets out to fix it.

Catalog record

CREATORJorge Quesada, Ghassan AlRegib
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-18619

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scale-Aware Self-Supervised Learning for Segmentation of Small and Sparse Structures Jorge Quesada, Ghassan AlRegib Abstract Self-supervised learning (SSL) has emerged as a powerful strategy for representation learning under limited annotation regimes, yet its effectiveness remains highly sensitive to many factors, especially the nature of the target task. In segmentation, existing pipelines are typically tuned to large, homogeneous regions, but their performance drops when objects are small, sparse, or locally irregular. In this work, we propose a scale-aware SSL adaptation that integrates small-window cropping into the augmentation pipeline, zooming in on fine-scale structures during pretraining. We evaluate this approach across two domains with markedly different data modalities: seismic imaging, where the goal is to segment sparse faults, and neuroimaging, where the task is to delineate small cellular structures. In both settings, our method yields consistent improvements over standard and state-of-the-art baselines under label constraints, improving accuracy by up to 13% for fault segmentation and 5% for cell delineation. In contrast, large-scale features such as seismic facies or tissue regions see little benefit, underscoring that the value of SSL depends critically on the scale of the target objects. Our findings highlight the need to align SSL design with object size and sparsity, offering a general principle for buil ding more effective representation learning pipelines across scientific imaging domains. Citation J. Quesada, and G. AlRegib, ”Scale-Aware Self-Supervised Learning for Segmentation of Small and Sparse Structures”, accepted at 2026 IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) Review Date of acceptance: 17 January 2026 Code & Data https://github.com/olivesgatech/SASSL Bib @ARTICLE{Quesada2026Scale, author={J. Quesada and G. AlRegib}, journal={IEEE International Conference on Acoustics, Speech, and Signal Processing}, title={Scale-Aware Self-Supervised Learning for Segmentation of Small and Sparse Structures}, year={2026},} Copyright ©Creative Commons Attribution CCBY 4.0 Contact {jpacora3, alregib}@gatech.edu https://alregib.ece.gatech.edu/ Corresponding author [email protected] † † address: OLIVES at the Georgia Institute of Technology 1 Introduction Self-supervised learning (SSL) has become a cornerstone of modern representation learning, enabling models to leverage large unlabeled datasets through pretext tasks rather than costly manual annotations [ 7 , 11 , 24 ] . Its effectiveness has not only driven advances across vision [ 7 , 8 ] , medical imaging [ 20 , 14 ] , and remote sensing [ 23 , 16 ] , but has also laid the foundation for the rise of transformer architectures [ 13 , 18 , 19 ] , whose pretraining paradigms are inherently self-supervised [ 9 , 6 ] . Despite these successes, most existing SSL pipelines implicitly assume that learned features should capture broad, homogeneous patterns. This design choice aligns well with tasks such as classification or object recognition, but its suitability for tasks dominated by small or sparse structures remains unclear. (a) Seismic and brain image slices (b) Ground truth (c) SSL segmentation of large structures (d) SSL segmentation of small structures Figure 1 : Differences in SSL segmentation depend on the nature of the target. Left: Seismic segmentation task with geological facies (large) and seismic faults (small). Right: Neuroimaging segmentation task from a mouse brain cutout, with axons (large) and cell bodies and vessels (small). Segmentation tasks provide a natural testbed for this question. In many domains, targets vary drastically in their spatial scale: large and continuous regions such as organs [ 22 ] or geological facies [ 2 ] contrast sharply with small and rare structures such as neurons, blood vessels [ 20 ] , or seismic faults [ 17 ] . Standard SSL approaches often underperform on the latter, as pretraining objectives dominated by large contextual patterns fail to emphasize the fine-scale cues required for accurate segmentation. Figure 1 illustrates this mismatch using segmentation tasks from geophysics [ 17 ] and neuroscience [ 20 ] : the same SSL pipeline that yields useful representations for large structures struggles to capture narrow or localized ones, either by merging them into a single large object losing granularity (seismic case), or by capturing only the most salient ones (neuroscience case). In this work, we argue that the effectiveness of SSL in segmentation depends critically on the scale and sparsity of the target objects. To address this, we propose a simple yet effective scale-aware strategy that embeds small-window cropping directly into the augmentation process, forcing encoders to “zoom in” on localized features during pretraining. This modification biases representations toward fine-grained structures without requiring architectural changes or additional supervision. We validate this approach across two distinct scientific imaging domains: (i) seismic imaging, where faults appear as narrow, discontinuous anomalies within large volumes, and (ii) neuroimaging, where cells and other microscopic structures are small and spatially sparse relative to the surrounding tissue. Our experiments demonstrate that scale-aware SSL provides consistent improvements in both settings under limited labels, yielding up to 13% accuracy gains for fault segmentation and 5% for blood cell and vessel delineation. By contrast, tasks involving larger-scale features (such as seismic facies or axon regions) show little benefit, underscoring the importance of tailoring SSL strategies to the intrinsic properties of the segmentation target. 2 Related Work Segmentation tasks vary widely in terms of the spatial scale and density of the target structures. Deep learning has achieved strong results for segmenting large to mid-size homogeneous regions such as organs or geological facies [ 2 , 21 ] . In contrast, small or sparse structures continue to be more challenging because of their limited spatial footprint, sparsity, and irregular geometry [ 12 , 15 ] . This gap suggests that scale strongly influences representation quality for downstream segmentation tasks. SSL has made substantial progress with global contrastive methods (SimCLR [ 7 ] , MoCo [ 11 ] ), non-contrastive ones (BYOL [ 10 ] , SimSiam [ 8 ] ), among others for a wide variety of downstream tasks. However, these methods typically assume (or are optimized for) features that capture broad semantic structure. Thus many SSL pipelines achieve good transfer performance on downstream dense prediction tasks when the objects of interest have well-defined and reasonably large spatial extent, as we showcase in Figure 1 . Recent works, however, begin to challenge this assumption. [ 1 ] observes that methods that include multiscale views, patch-based tasks, or dense pixel/patch clustering often behave better when the downstream task involves fine-grained or spatially localized structures. [ 5 ] learns both global and local features to target segmentation tasks, [ 12 ] enforces inter-scale consistency, and [ 15 ] addresses inconsistencies in non-salient regions. These methods highlight the importance of balancing local and global cues during SSL pretraining. While prior work recognizes the value of multi-scale or patch-level cues, few studies systematically analyze the impact of object scale and sparsity on SSL performance across domains. Our work addresses this gap by explicitly embedding small-window cropping into SSL pretraining and validating its effect on small, sparse targets in both seismic and neuroimaging data. 3 Methodology 3.1 Self-Supervised Pretraining We adopt a general SSL framework where an encoder f_{\theta} learns representations from unlabeled images by enforcing consistency between multiple augmented views of the same sample. Given an input image x\in\mathbb{R}^{H\times W} , two stochastic augmentations t_{1},t_{2}\sim\mathcal{T} produce correlated views x_{1}=t_{1}(x),\;x_{2}=t_{2}(x) . Each view is passed through the encoder and optionally a projection head g_{\phi} , yielding embeddings z_{i}=g_{\phi}(f_{\theta}(x_{i})) . Different

Read the complete text