Divide and Conquer: Object Co-occurrence Helps Mitigate Simplicity Bias in OOD Detection
Boyang Dai, Chaoqi Chen, Yizhou Yu · 2026
Read Divide and Conquer: Object Co-occurrence Helps Mitigate Simplicity Bias in OOD Detection by Boyang Dai, Chaoqi Chen, Yizhou Yu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07821.txt.
Introduction
Boyang Dai and two co-authors detect out-of-distribution images by reading the company objects keep. Detectors usually fight over entangled representations, ignoring context, and they struggle most on near-out-of-distribution cases, where a model's simplicity bias locks onto easy regions. Human vision, by contrast, uses object co-occurrence in natural scenes to understand what belongs.
Their framework, OCO, predicts disentangled representations for a test image, sorts its co-occurrence pattern into one of three scenarios learned from in-distribution data, and detects outliers within each scenario separately. Across challenging and full-spectrum test settings the method stayed competitive and handled both semantic and covariate shifts, precisely because it judges scenes by their object relationships rather than surface simplicity.
Catalog record
| CREATOR | Boyang Dai, Chaoqi Chen, Yizhou Yu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 44 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07821
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Divide and Conquer: Object Co-occurrence Helps Mitigate Simplicity Bias in OOD Detection Boyang Dai, Chaoqi Chen, Yizhou Yu Abstract Out-of-distribution (OOD) detection is crucial for ensuring the reliability of deep learning models. Existing methods mostly focus on regular entangled representations to discriminate in-distribution (ID) and OOD data, neglecting the rich contextual information within images. This issue is particularly challenging for detecting near-OOD, as models with simplicity bias struggle to learn discriminative features in disentangled representations. The human visual system can use the co-occurrence of objects in the natural environment to facilitate scene understanding. Inspired by this, we propose an Object-Centric OOD detection framework that learns to capture O bject CO -occurrence (OCO) patterns within images. The proposed method introduces a new OOD detection paradigm that understands object co-occurrence within an image by predicting disentangled representations for the test sample, then adaptively divides patterns into three scenarios based on object co-occurrence patterns observed in ID training data, and finally performs OOD detection in a divide-and-conquer manner. By doing so, OCO can distinguish near-OOD by considering the semantic contextual relationships present in their images, avoiding the tendency to focus solely on simple, easily learnable regions. We evaluate OCO through experiments across challenging and full-spectrum OOD settings, demonstrating competitive results and confirming its ability to address both semantic and covariate shifts. Code is released at https://github.com/Michael-McQueen/OCO . 1 Introduction Deep learning has achieved remarkable success across various domains over the past decade 28 ; 11 . Despite deep learning models exhibiting extraordinary performance on in-distribution (ID) data, they face significant challenges when distributional shifts occur 55 ; 32 . To solve this issue, out-of-distribution (OOD) detection has been extensively investigated, aiming to make deep models more reliable when deployed in non-stationary test environments. Existing OOD detection methods predominantly rely on the distinctive characteristics between ID and OOD data in the latent feature space 36 ; 51 , logit outputs 32 ; 38 , or a combination of both 59 ; 44 ; 69 . Although effective for far-OOD scenarios where distribution shifts are drastic, it often falters in near-OOD settings, where subtle semantic deviations make differentiation more challenging 24 ; 22 ; 41 ; 58 ; 66 ; 6 ; 72 ; 34 ; 14 ; 13 . Figure 1 : Attention visualization of vanilla method and object-centric prediction paradigms on mixed anomalous samples, dog is the ID (blue box), and shark is the OOD (red box). This limitation stems from the models’ simplicity bias – when processing entangled representations, deep networks tend to prioritize easily learnable local cues while overlooking complex semantic relationships that are crucial for scene understanding 48 ; 54 ; 70 ; 5 ; 53 ; 15 . As shown in Fig. 1 , for the anomalous image mixing ID and OOD samples, depicting an oceanic scene where a dog (ID) and a shark (OOD) co-occur. For the conventional approach, simplicity bias causes predictions based on the entangled feature 1 1 1 This refers to the predictive features from the model’s final layer. to focus solely on the dog’s discriminative parts, neglecting the scene’s contextual incongruity. This results in overconfident predictions, with a confidence of 52.90%. In contrast, humans can easily recognize the semantic incompatibility between a dog and a shark co-occurring in an oceanic setting. Cognitive science research suggests that the human visual system leverages object relationships within scenes to make predictions about the world 43 ; 8 ; 3 . This gap between human and machine perception raises a critical question: How can object co-occurrence patterns be leveraged to mitigate simplicity bias in OOD detection? Recent studies in object-centric representation learning show that an image’s entangled features can be decomposed into compositions of distinct objects 62 ; 73 ; 35 . Motivated by this insight, we argue that an effective OOD detection method should focus on the underlying compositions and patterns within an image, rather than treating it as a unified entity when distinguishing between ID and OOD data. In response, we propose OCO , a novel OOD detection framework that addresses simplicity bias through object co-occurrence (OCO) modeling. We utilize Slot Attention to abstract disentangled representations into slots that capture object co-occurrence. These slots are then aligned with semantics to form OCO patterns, which mirror human perceptual logic. As shown in Fig. 1 (right), each slot specializes in specific visual concepts—such as facial features, anatomical details, and contextual elements (e.g., ocean scene in our example)—while aggregated predictions help uncover semantic contextual incompatibility, reducing overconfidence from 52.90% to 20.80% in ambiguous cases. For OOD detection, we propose more effective detection scores by incorporating object co-occurrence pattern information. Based on the nature of test samples’ co-occurrence patterns, we categorize all possible patterns into three scenarios: (1) single patterns, (2) typical patterns, and (3) atypical patterns. Adopting this divide-and-conquer approach, rather than a one-size-fits-all paradigm, we devise tailored OOD detection scores for each scenario based on its distinct characteristics. The main contributions are summarized as follows: • We propose OCO, a novel object-centric framework for OOD detection that leverages object co-occurrence patterns to learn complex scene-object semantic correlations, effectively mitigating the simplicity bias inherent in traditional models. • Unlike methods that use a unified OOD score for all data, OCO adaptively categorizes test samples into different groups based on their co-occurrence patterns, assigning specific OOD scores to each group. • We demonstrate that OCO achieves competitive performance against state-of-the-art baselines on both challenging OOD and Full-spectrum OOD 65 benchmarks, showing its ability to handle both semantic and covariate shifts effectively. Figure 2 : Overview of our OCO . We first establish ID training data object co-occurrence pattern statistics ( \mathcal{F}_{train} ). During OOD detection, patterns are grouped according to \mathcal{F}_{t} co-occurrences to compute OOD scores. The slot-wise prediction can learn object co-occurrence patterns. In the above example, dog , cat , camel , and alcidae are ID, while penguin is OOD, where alcidae and penguins have very similar semantic features. 2 Preliminary 2.1 Preliminary: Slot Attention Slot attention (SA) 39 has emerged as a mainstream object-centric method that can iteratively extract object features into several slots in an unsupervised manner. Given an input feature map \bm{x}\in\mathbb{R}^{H\times W\times C} , where H and W are the size of the feature map, and C is the feature map dimension. In SA , the query \bm{Q} is generated by sampling K slots \bm{S}\in\mathbb{R}^{K\times d} from a learnable Gaussian distribution, where d is the dimension of each slot. The input feature map \bm{x} is then transformed into the key \bm{K} and value \bm{V} , which together with \bm{Q} enable the cross-attention mechanism 57 that competitively abstracts object features into slots: \bm{Q}^{(t)}=\bm{S}^{(t)}\bm{W}_{Q},\hskip 20.00003pt\bm{K}=\bm{xW}_{K},\hskip 20.00003pt\bm{V}=\bm{xW}_{V}, \bm{W}_{Q} , \bm{W}_{K} , and \bm{W}_{V} are learnable projection matrices. Then, slot features are iteratively refined through a GRU DBLP:conf/ssst/ChoMBB14 that serves as a memory-like update mechanism: \bm{A}^{(t)}=\textit{softmax}(\frac{\bm{Q}\bm{K}^{T}}{\sqrt{D}}),\hskip 10.00002pt\bm{S}^{(t+1)}=\text{GRU}(\bm{S}^{(t)},\bm{A}^{(t)}\bm{V}). 2.2 Problem Setup Let \mathcal{D}_{\text{train}}=\{(\bm{X}_{i},y_{i})\}_{i=1}^{n} be a set of training instances sampled independently and identically from the source distribution \mathcal{P}_{\text{in}} , where y_{i}\in\{1,\ldots,M\} denotes the corresponding class label. In the context of OOD detection, a model is first trained on \mathcal{D}_{\text{train}} , then deployed to identify samples drawn from an unknown target distribution \mathcal{P}_{\text{out}} . The detector \mathcal{G} can be formally