OASIC: Occlusion-Agnostic and Severity-Informed Classification

Kay Gijzen, Gertjan J. Burghouts, Daniël M. Pelt · 2026

Read OASIC: Occlusion-Agnostic and Severity-Informed Classification by Kay Gijzen, Gertjan J. Burghouts, Daniël M. Pelt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-04012.txt.

Introduction

Kay Gijzen and colleagues built OASIC, a classifier for heavily occluded objects that first masks the occluder, then picks a model tuned to the occlusion level. Occlusion hurts in two ways: visible detail is lost, and occluder patterns distract. Masking occluders as visual anomalies at test time removes the distraction, while training with random masks at varied levels covers the detail loss. The team also showed occlusion severity can be estimated at test time. Combining gray masking with adaptive model choice beat standard training by wide margins.

Catalog record

CREATORKay Gijzen, Gertjan J. Burghouts, Daniël M. Pelt
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE28086
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.04012

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

OASIC: Occlusion-Agnostic and Severity-Informed Classification Kay Gijzen, Gertjan J. Burghouts, Daniël M. Pelt Abstract Severe occlusions of objects pose a major challenge for computer vision. We show that two root causes are (1) the loss of visible information and (2) the distracting patterns caused by the occluders. Our approach addresses both causes at the same time. First, the distracting patterns are removed at test-time, via masking of the occluding patterns. This masking is independent of the type of occlusion, by handling the occlusion through the lens of visual anomalies w.r.t. the object of interest. Second, to deal with less visual details, we follow standard practice by masking random parts of the object during training [ 13 ] , for various degrees of occlusions. We discover that (a) it is possible to estimate the degree of the occlusion (i.e. severity) at test-time, and (b) that a model optimized for a specific degree of occlusion also performs best on a similar degree during test-time. Combining these two insights brings us to a severity-informed classification model called OASIC: Occlusion Agnostic Severity Informed Classification. We estimate the severity of occlusion for a test image, mask the occluder, and select the model that is optimized for the degree of occlusion. This strategy performs better than any single model optimized for any smaller or broader range of occlusion severities. Experiments show that combining gray masking with adaptive model selection improves AUC occ \text{AUC}_{\text{occ}} by +18.5 over standard training on occluded images and +23.7 over finetuning on unoccluded images. 1 Introduction Modern computer vision models perform impressively on clean, fully visible images. In practice, however, objects are often partially hidden or obscured, e.g., foliage and smoke. Finegrained classification is challenging under severe occlusions, because less of the details are visible which are needed to distinguish the classes. We consider up to 90% occlusion, i.e., 10% of the finegrained class is visible. Finegrained classification under such severe occlusions is a under-explored area, but essential for real-world deployment. The gap between clean training images and occluded real-world conditions is a significant challenge in computer vision [ 23 , 25 ] . Occlusion is challenging for two reasons: it reduces the visible regions of an object and introduces distracting patterns that can mislead models [ 31 , 24 ] . This is confirmed for finegrained classification, by the findings in this paper. Datasets containing annotated occlusion are scarce, and real-world occlusions are often unpredictable. That makes robust finegrained classification under severe occlusions a particularly difficult yet interesting problem. To address the challenges posed by occlusion, we introduce OASIC (Occlusion-Agnostic Severity-Informed Classification), a method designed to explicitly tackle the two fundamental difficulties of occlusion: (i) the loss of visible information and (ii) the visual distraction introduced by the occluders themselves. OASIC handles occlusion as a form of visual anomaly with respect to the object of interest, allowing it to handle any type of occlusion—such as vegetation, smoke, or other visual obstructions—without requiring prior knowledge of the occluder. Our method employs pixel-level occlusion likelihood maps, enabling what we term occlusion-agnostic segmentation. From these maps, we derive segmentation masks that serve two complementary roles. First, they mitigate visual distraction by replacing occluded regions with a neutral tone, effectively masking out the misleading texture cues introduced by the occluders. Second, the same segmentation maps allow us to estimate the severity of occlusion, defined as the fraction of the image affected by occlusion. During training, OASIC follows standard practice by applying random masks to simulate varying degrees of occlusion, ensuring robustness across different severity levels. Through this process, we discover two key properties: (a) the degree of occlusion can be estimated reliably at test time, and (b) a model optimized for a specific severity level performs best when tested on similar levels of occlusion. Combining these insights, OASIC operates as a severity-informed classification framework. At inference time, the method estimates the occlusion severity for each test image, masks the occluding patterns, and dynamically selects the model trained for the corresponding degree of occlusion. This strategy consistently outperforms any single model trained on a fixed or broader range of occlusion severities, achieving robust and adaptable performance under diverse occlusion conditions. The main contributions of this work are threefold. (1) We introduce OASIC, a unified framework that addresses both the loss of visual information and the distracting patterns caused by occluders. (2) Occlusion-agnostic segmentation by interpreting occluded regions as visual anomalies, enabling the localization of arbitrary occluders without prior knowledge of their appearance, suppressing distraction. (3) A severity-informed model selection strategy that leverages the estimated occlusion severity to dynamically select the most suitable model from a set of models which were trained across varying occlusion levels. This adaptive approach leads to consistently improved classification robustness under diverse and unpredictable occlusion conditions. 2 Related Work To handle occlusion, data augmentation techniques create modified training samples that discourage reliance on specific image regions. Mixup [ 30 ] blends two images through pixel averaging, while CutMix [ 28 ] replaces patches from one image with another. Hide-and-Seek [ 13 ] randomly hides image regions to force attention to alternative cues. These methods promote part-based learning by exposing models to partial views. TransMix [ 1 ] , extending CutMix, uses transformer attention to weight label contributions, further enhancing part-based robustness. Such augmentations improve recognition under occlusion by linking labels to visible regions, but they mainly mitigate information loss and do not fully resolve confusion from misleading occluder textures. Part-based methods explicitly model object structure. CompositionalNets [ 11 ] include an occlusion localization module and part dictionaries that reason over visibility, improving recognition of partially visible objects. TDMPNet [ 27 ] estimates visibility maps and suppresses occluded features via top-down attention, yielding cleaner feature representations. Despite their strengths, these CNN-based methods remain limited by local receptive fields and poor long-range reasoning, making them sensitive to structured occlusion [ 4 , 3 ] . Transformer-based models (ViTs) demonstrate superior robustness to occlusion [ 16 , 8 ] , largely due to self-supervised pretraining. Masked Image Modeling (MIM) [ 10 ] , as used in MAE [ 7 ] and iBOT [ 32 ] , trains models to reconstruct masked regions, inherently promoting occlusion resilience. Large-scale vision transformers such as CLIP [ 20 ] and DINOv2 [ 17 ] further benefit from massive, diverse pretraining, yielding general-purpose visual representations adaptable to many downstream tasks. However, transformer robustness is largely incidental: while ViTs retain accuracy under partial occlusion [ 16 ] , their performance degrades under strong or structured occlusions [ 8 ] . This highlights the need for complementary strategies beyond pretraining to achieve reliable occlusion handling. To our knowledge, no prior work addresses occlusion segmentation in an occlusion-agnostic manner using pixel-level anomaly maps. We repurpose anomaly detection methods such as AnomalyDINO [ 2 ] , PatchCore [ 22 ] , and DRAEM [ 29 ] , which localize visual irregularities via per-pixel anomaly likelihoods. By interpreting high anomaly scores as occluded regions, we achieve occlusion segmentation without explicit supervision, relying only on image-level labels. Unlike segmentation models such as SAM2 [ 21 ] or OVSeg [ 14 ] , which require prior knowledge or prompting for each occluder type [ 26 ] ,

Read the complete text