Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection

Zhexiao Huang, Weihao He, Shutao Deng, Junzhe Chen, Chao Yuan, Hongxin Wang, Changsheng Zhou* · 2025

Read Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection by Zhexiao Huang, Weihao He, Shutao Deng, Junzhe Chen, Chao Yuan, Hongxin Wang, Changsheng Zhou* (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-00849.txt.

Introduction

Zhexiao Huang and colleagues propose P-OCS, a method for flagging out of distribution inputs, data unlike anything a model was trained on. Spotting such inputs matters when deep learning systems run in open environments, where strange cases appear without warning.

The method works in the space left over after the main feature directions of normal data are removed. A single small nudge applied only inside that leftover space sharpens the faint differences between normal and foreign samples, without disturbing the geometry of the normal ones. The authors prove a one step update suffices in the small nudge regime and give convergence guarantees. Across designs and data sets, they report top detection scores at almost no extra computing cost and with no retraining.

Catalog record

CREATORZhexiao Huang, Weihao He, Shutao Deng, Junzhe Chen, Chao Yuan, Hongxin Wang, Changsheng Zhou*
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE25 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-00849

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Perturbations in the Orthogonal Complement Subspace for Efficient Out-of-Distribution Detection Zhexiao Huang, Weihao He, Shutao Deng, Junzhe Chen, Chao Yuan, Hongxin Wang, Changsheng Zhou* Abstract Out-of-distribution (OOD) detection is indispensable for the reliable deployment of deep learning models in open-world environments. Existing approaches, including energy-based scoring and gradient-projection methods, typically exploit high-dimensional representations to separate in-distribution (ID) from OOD samples. We present P-OCS (Perturbations in the Orthogonal Complement Subspace), a lightweight and theoretically grounded method that operates within the orthogonal complement of the principal subspace spanned by ID features. P-OCS applies a single projected perturbation confined to this complementary subspace, selectively amplifying subtle ID–OOD discrepancies while preserving the geometry of ID representations. We show that, in the small-perturbation limit, a one-step update is sufficient and provide convergence guarantees for the resulting detection score. Extensive experiments across diverse architectures and datasets demonstrate that P-OCS achieves state-of-the-art OOD detection with negligible computational overhead, without requiring model retraining, access to OOD data, or architectural modifications. Keywords out-of-distribution detection \cdot orthogonal complement subspace \cdot perturbation \cdot robustness \cdot deep learning 1 Introduction Deep neural networks frequently produce highly confident predictions when confronted with samples drawn from distributions beyond their training regime. As such, identifying out-of-distribution (OOD) samples is essential for the safe deployment of models in open-world settings. A substantial body of prior work has explored post-hoc scoring mechanisms—such as ODIN [ 1 ] , the Mahalanobis-distance method [ 2 ] , energy-based scores [ 3 ] , and GradOrth [ 4 ] —to distinguish in-distribution (ID) from OOD samples using features of pretrained classifiers. These methods tend to perform well when OOD data exhibit substantial covariate or feature shift (i.e., far-OOD ), but their performance often degrades in more challenging regimes of semantic shift with feature overlap (i.e., near-OOD ), even when aided by post-processing strategies such as ReAct or PCA-based removal of dominant principal components. In our investigation, we observe a consistent geometric phenomenon in the penultimate-layer feature space : ID samples concentrate variance within a dominant principal subspace, whereas OOD samples distribute variance more broadly into the orthogonal complement of that subspace. This observation motivates our central insight: a single perturbation restricted to the orthogonal complement can expose intrinsic separability between ID and OOD samples, including in near-OOD regimes with strong feature overlap . Accordingly, we propose P-OCS ( P erturbations in the O rthogonal C omplement S ubspace) — a minimalist yet theoretically grounded framework that performs a one-step orthogonal perturbation in the complement of the ID principal subspace, computed at the penultimate layer. Empirically, this single iteration achieves near-optimal discriminative power with negligible computational overhead. To better visualize the difference between traditional post-hoc scores and our proposed P-OCS, we compare the score distributions of ID and OOD samples under four representative scoring schemes: (a) ReAct-processed Maximum Softmax Probability (MSP), (b) ReAct-processed Energy score, (c) ReAct-processed Mahalanobis distance, and (d) our proposed P-OCS score. As shown in Fig. 7 , existing methods exhibit substantial overlap between ID and OOD distributions in near-OOD regimes, while P-OCS yields a distinct and well-separated score boundary. Figure 1: PCA projection of ID and Near-OOD samples. ID samples (green) concentrate along the principal components, while Near-OOD samples (blue) exhibit greater dispersion across orthogonal directions. This illustrates the challenge in separating Near-OOD samples from ID using standard methods. Figure 2: Further PCA analysis highlighting the challenge of distinguishing Near-OOD from ID samples. The right plot shows how Near-OOD samples (blue) are dispersed across the feature space, making it difficult for traditional models to separate them from ID (green) samples without additional processing. Figure 3: * (a) ReAct + MSP Figure 4: * (b) ReAct + Energy Figure 5: * (c) ReAct + Mahalanobis Figure 6: * (d) P-OCS Figure 7: Comparison of score distributions for ID (green) and Near-OOD (blue) samples under different post-hoc scoring schemes. Existing methods (a–c) show significant overlap between ID and OOD scores, indicating poor separability. In contrast, our proposed P-OCS (d) yields a clear margin between the two distributions with minimal computation. Contributions. 1. We introduce P-OCS , a one-step orthogonal-complement perturbation method for OOD detection that operates on penultimate-layer features, requiring no additional training or architectural modification. 2. We provide theoretical justification that a single perturbation step suffices under small-perturbation regimes, yielding a principled detection score. 3. We analyze the variance structure of the orthogonal complement and establish its connection to distributional separability, clarifying why near-OOD cases benefit markedly. 4. We demonstrate state-of-the-art detection performance and computational efficiency across multiple benchmarks and architectures, with consistent gains on near-OOD as well as strong results on far-OOD . 2 Related Work 2.1 Out-of-Distribution Detection Classical OOD detection approaches primarily rely on post-hoc confidence estimation over pretrained classifiers. Early methods include the maximum softmax probability (MSP) baseline [ 5 ] , temperature scaling and input perturbation in ODIN [ 1 ] , Mahalanobis distance-based detectors [ 2 ] , and energy-based scoring [ 3 ] . Recent advances, such as ReAct [ 6 ] , DICE [ 7 ] , and GradOrth [ 4 ] , enhance robustness by modifying activation statistics or gradient representations. While these techniques improve detection in far-OOD settings, many remain computationally expensive or lack geometric interpretability, especially for semantically shifted near-OOD cases. 2.1.1 Subspace and Spectral Methods Principal component analysis (PCA) and singular value decomposition (SVD) have been widely used to characterize the intrinsic structure of in-distribution (ID) features [ 8 , 9 ] . For instance, GradOrth [ 4 ] projects gradients onto dominant singular directions to suppress irrelevant variance, aligning feature responses along stable axes. Our work extends this perspective by explicitly leveraging the orthogonal complement subspace , which captures variance components where OOD deviations predominantly reside. This formulation bridges subspace geometry with probabilistic separability in OOD detection. 2.1.2 Perturbation and Feature Stability Perturbation-based and adversarial methods have long studied the stability of model predictions under small input or feature perturbations [ 10 , 11 ] . Unlike input-space perturbations, our approach introduces a iterative perturbation directly in the learned orthogonal feature manifold. This design leads to a mathematically interpretable dynamic process that exposes intrinsic ID–OOD separability without requiring adversarial optimization or retraining. 3 Method 3.1 Preliminaries Let X_{\mathrm{ID}}\in\mathbb{R}^{N\times d} denote the features extracted from an in-distribution (ID) dataset. We first compute its mean and perform principal component analysis (PCA) [ 12 , 13 ] : X_{c}=X_{\mathrm{ID}}-\mu,\quad X_{c}=U\Sigma V^{\top}, (1) where U=[U_{k},U_{\perp}] separates the feature space into the principal subspace U_{k} and its orthogonal complement U_{\perp} . We define the projection operators: P=U_{k}U_{k}^{\top},\quad P_{\perp}=U_{\perp}U_{\perp}^{\top}. (2) To visualize the variance ratio explained by the ID and OOD projections onto the ID basis and the complement space, we present the following two figures. These figures show how the variance is distributed across the first 80 components of the ID and complement spaces, highlighting the differences between the ID and OOD distributions. Figure 8: ID vs OOD explained variance ratio (first 80 components) — on ID basis. Figure 9: ID vs OOD explained variance ratio in complement space (first 80 components). Explanation of Figures: The first figure ( 9 ) presents the explained variance ratio for the first 80 principal components of the ID space. As expected, the ID

Read the complete text