One Model, Many Behaviors: Training-Induced Effects on Out-of-Distribution Detection
Gerhard Krumpl, Henning Avenhaus, Horst Possegger, [0.4em] Institute of Visual Computing, Graz University of Technology, Austria · 2026
Read One Model, Many Behaviors: Training-Induced Effects on Out-of-Distribution Detection by Gerhard Krumpl, Henning Avenhaus, Horst Possegger, [0.4em] Institute of Visual Computing, Graz University of Technology, Austria (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-10836.txt.
Introduction
Gerhard Krumpl, Henning Avenhaus, and Horst Possegger study how training choices shape out-of-distribution detection. Detectors must flag inputs that differ from what a model was trained on. They are key to safe deployment in open worlds. Yet most work tunes detectors alone, apart from the training pipelines that boost in-distribution accuracy.
The authors ask whether one model can show many behaviors: strong on known data, yet honest about strange data. Their empirical study links training recipes to detector performance. Some tricks that raise accuracy may quietly harm detection. The findings guide teams that need both skill and self-knowledge from the same model.
Catalog record
| CREATOR | Gerhard Krumpl, Henning Avenhaus, Horst Possegger, [0.4em] Institute of Visual Computing, Graz University of Technology, Austria |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-10836
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
One Model, Many Behaviors: Training-Induced Effects on Out-of-Distribution Detection Gerhard Krumpl, Henning Avenhaus, Horst Possegger, [0.4em] Institute of Visual Computing, Graz University of Technology, Austria Abstract Out-of-distribution (OOD) detection is crucial for deploying robust and reliable machine-learning systems in open-world settings. Despite steady advances in OOD detectors, their interplay with modern training pipelines that maximize in‑distribution (ID) accuracy and generalization remains under-explored. We investigate this link through a comprehensive empirical study. Fixing the architecture to the widely adopted ResNet‑50, we benchmark 21 post-hoc, state-of-the-art OOD detection methods across 56 ImageNet-trained models obtained via diverse training strategies and evaluate them on eight OOD test sets. Contrary to the common assumption that higher ID accuracy implies better OOD detection performance, we uncover a non‑monotonic relationship: OOD performance initially improves with accuracy but declines once advanced training recipes push accuracy beyond the baseline. Moreover, we observe a strong interdependence between training strategy, detector choice, and resulting OOD performance, indicating that no single method is universally optimal. 1 Introduction Figure 1 : Top-1 classification accuracy is an unreliable indicator for OOD detection performance. This figure shows the relationship between in-distribution (ID) classification accuracy and out-of-distribution (OOD) detection performance for 56 ResNet-50 models, all sharing the same architecture but trained with different strategies. Each point represents one model and reports the mean AUROC (Area Under the Receiver Operating Characteristic Curve) over 21 OOD detection methods and eight OOD datasets. Color indicates the model’s training category, while the marker shape uniquely identifies each model within that category. The robust deployment of machine learning (ML) systems in real-world environments depends on models that not only perform well on in-distribution (ID) data but can also reliably detect inputs that differ from their training distribution—such as novel object categories, unexpected environmental conditions, or sensor corruptions. When exposed to such out-of-distribution (OOD) inputs, modern ML models often produce high-confidence yet incorrect predictions [ 52 , 50 , 27 ] . This behavior poses significant risks in safety-critical settings, such as autonomous driving, healthcare, or industrial quality assurance. For example, an autonomous car might find itself in an unexpected emergency scene, or a food-sorting stream could encounter unexpected (potentially allergy-inducing) foreign materials. Post-hoc OOD detection methods offer a promising solution. These methods are appealing in practice: i) they are simple to deploy on pretrained models, avoiding costly retraining and preserving ID accuracy, and ii) they require no prior exposure to OOD data, which, by definition, are unknown before deployment. A wide variety of methods have emerged, ranging from simple confidence scores [ 30 ] to more sophisticated approaches based on feature-space distances [ 40 , 62 , 56 ] and model enhancements [ 60 , 15 ] . Concurrently, modern classification training pipelines have become increasingly diverse and powerful. Augmentations such as MixUp [ 78 ] or CutMix [ 77 ] , regularization techniques like label smoothing [ 63 ] or Exponential Moving Averages (EMA), contrastive/self-supervised pretraining, and extended training schedules have all been shown to improve ID accuracy and generalization [ 70 , 26 , 67 ] . Yet, surprisingly, the impact of such training strategies on post-hoc OOD detection has received little attention. Most prior evaluations continue to benchmark OOD detection methods on models trained with vanilla strategies—failing to reflect the diversity of models used in practice. A widely held assumption is that higher ID accuracy naturally translates into stronger OOD detection [ 66 , 19 , 35 ] . This belief has shaped experimental design and practical implementation alike: train a more accurate classifier and get stronger OOD detection performance for free . However, this assumption is rarely questioned—and its validity under diverse training strategies remains unclear. In this work, we revisit this and other assumptions through the first large-scale, architecture- and data-controlled study on the effects of training strategies on post-hoc OOD detection. By fixing the architecture (ResNet-50 [ 25 ] ) and dataset (ImageNet [ 14 ] ), we isolate the impact of training strategy alone. We evaluate 21 post-hoc OOD detection methods across 56 models spanning four training families: data augmentation, contrastive/self-supervised learning, adversarial training, and improved training schedules—as summarized in Fig. 1 . Performance is assessed using eight benchmark OOD datasets that cover near-, far-, extreme-, and synthetic-OOD categories. This study provides new insights into how training influences OOD detection and offers guidance for developing more robust, generalizable OOD detection methods. We summarize our contribution as follows: • We conduct a large-scale, architecture- and data-controlled study on the influence of training strategies on post-hoc OOD detection, covering 56 models, 21 OOD detectors, and eight OOD test sets. We further analyze the correlation between ID accuracy and OOD detection performance across methods and training strategies. • We show that the commonly assumed correlation between ID accuracy and OOD robustness is not generally valid. Prior studies often focus on low-to-moderate accuracy regimes or vary model architectures, where capacity effects confound the relationship. • We identify substantial variance in method robustness: many post-hoc detectors overfit to vanilla training recipes and degrade under diverse strategies, while methods that utilize richer internal representations generalize better. 2 Related Work Out-of-distribution (OOD) detection methods are broadly categorized into training-based and post-hoc approaches [ 75 ] . Training-based methods modify model architectures, loss functions, or regularization strategies to improve OOD robustness [ 44 , 17 , 33 ] . Recent training-based works also focus on improving OOD generalization and detection [ 2 , 83 ] . In contrast, post-hoc methods operate on pretrained models without requiring retraining or access to OOD samples. We focus on post-hoc, sample-free methods due to their practical advantages: they are model-agnostic, preserve the model’s in-distribution (ID) performance, and scale efficiently to large deployments, where retraining is costly or infeasible [ 80 ] . Post-hoc methods can be further grouped into score-based and model enhancement methods. Score-based methods assign an OOD score to each input based on model outputs or internal activations. Within this category, classification-based methods were pioneered by Hendrycks et al . [ 30 ] , who introduced Maximum Softmax Probability (MSP), which uses the highest softmax output as a baseline OOD score. Subsequent variants, such as Maximum Logit Score (MLS) [ 28 ] and Energy-Based OOD (EBO) [ 46 ] , refine this approach by operating directly on the model’s logits or their transformations. ODIN [ 42 ] further improves separation by combining temperature scaling with input perturbations. Feature-based methods instead operate on internal representations: Mahalanobis distance score (MDS) [ 40 ] , Simplified Hopfield Energy (SHE) [ 81 ] , and KNN-based methods [ 62 ] compute distances in feature space, while GRAM [ 58 ] and GradNorm [ 34 ] leverage intermediate statistics or input gradients. Model enhancement methods modify inference-time activations to improve OOD detection robustness without changing model weights. ReAct [ 60 ] improves robustness by clipping abnormal activations, ASH [ 15 ] prunes and rescales feature activations to down-weight irrelevant neurons, and SCALE [ 73 ] shapes internal representations. These are typically paired with energy-based scoring and have demonstrated strong performance in large-scale benchmarks [ 80 ] . ID accuracy vs. OOD detection performance. Prior works [ 66 , 19 , 35 ] have reported a positive correlation between ID