Out-of-Distribution Detection with Adaptive Top-K Logits Integration

Hikaru Shijo, Yutaka Yoshihama, Kenichi Yadani, Norifumi Murata · 2025

Read Out-of-Distribution Detection with Adaptive Top-K Logits Integration by Hikaru Shijo, Yutaka Yoshihama, Kenichi Yadani, Norifumi Murata (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00368.txt.

Introduction

Hikaru Shijo, Norifumi Murata and their co-authors propose ATLI, a method to detect out-of-distribution inputs for neural networks in 2025. Networks often give overconfident answers on such inputs, so spotting them is key to safe machine learning.

The best simple method uses only the largest logit score from a model. The team found that a few other top logits also carry useful signal. ATLI picks, for each model, which top logits help and blends them with the maximum. On a large image benchmark, it cut false positives well below the simple method and below other leading methods.

Catalog record

CREATORHikaru Shijo, Yutaka Yoshihama, Kenichi Yadani, Norifumi Murata
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-00368

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Out-of-Distribution Detection with Adaptive Top-K Logits Integration Hikaru Shijo, Yutaka Yoshihama, Kenichi Yadani, Norifumi Murata Abstract Neural networks often make overconfident predictions from out-of-distribution (OOD) samples. Detection of OOD data is therefore crucial to improve the safety of machine learning. The simplest and most powerful method for OOD detection is MaxLogit, which uses the model’s maximum logit to provide an OOD score. We have discovered that, in addition to the maximum logit, some other logits are also useful for OOD detection. Based on this finding, we propose a new method called ATLI (Adaptive Top-k Logits Integration), which adaptively determines effective top-k logits that are specific to each model and combines the maximum logit with the other top-k logits. In this study we evaluate our proposed method using ImageNet-1K benchmark. Extensive experiments showed our proposed method to reduce the false positive rate (FPR95) by 6.73% compared to the MaxLogit approach, and decreased FPR95 by an additional 2.67% compared to other state-of-the-art methods. keywords Out-of-Distribution, Image Classification 1 Introduction Out-of-distribution (OOD) detection is a critical task for improving the reliability and safety of machine learning models, particularly in real-world applications such as autonomous driving and medical diagnostics. This is because models tend to make overly confident predictions when faced with data that diverges from their training distribution. Several existing methods for OOD detection are commonly employed, including Maximum Softmax Probability (MSP) ( Hendrycks and Gimpel 2017 ) , MaxLogit ( Hendrycks et al. 2022 ) , and Energy ( Liu et al. 2020 ) . MSP uses maximum softmax probability, MaxLogit uses maximum logits, and Energy uses the logsumexp function for logits. These methods compute scores based on logit space. MaxLogit considers only maximum logits, whereas MSP and Energy consider all class logits. These methods are simple and powerful, but they are not state-of-the-art. However, since logits contain high-level semantic information, there is still potential for their use. In this study, we discovered that considering all logits or only maximum logits can result in low scores. In Figure 3(a) (b) and (c), we show the distribution of in-distribution (ID) and OOD with different top-k logits. The leftmost figure shows the distribution of the top-1 logit (MaxLogit), which is typically used in OOD detection. The other figures show the distributions of different top-k logits. In Eva (b), the top-2 and top-350 logits largely overlap between ID and OOD. However, the top-190 and top-900 logits separate ID and OOD as effectively as MaxLogit. In ResNet-50d (c), however, the top-2 largely overlap between ID and OOD, whereas the other top-190, top-350, and top-900 separate ID and OOD as effectively as MaxLogit. This pattern is also observed in other models. We thus find that the top-k logits effective for OOD detection vary across models. These findings suggest that the OOD score should contain only the effective top-k logits in each model. In this paper, we propose ATLI, which adaptively selects a set of effective top-k logits for each trained model, excluding the top-1 logit, and combines them with the maximum logit to compute the OOD score. To select effective top-k logits, we use a pseudo-OOD sample created from a training sample and evaluated our proposed method using various models and datasets. The results of the experiments reveal that the latest methods depend on the model, and in cases with poorly compatible models, the accuracy is below that of MaxLogit, which we use as our baseline. However, our proposed method consistently outperforms existing baselines across a variety of trained models, indicating its low dependency on specific trained models. Our contributions are summarized as follows. • We reveal that there are several top-k logits that can separate ID and OOD as effectively as MaxLogit for each trained model. • We develop a methodology for identifying effective top-k logits for each trained model by utilizing pseudo-OOD. • Our proposed method has a very simple implementation and outperforms other methods. (a) Illustration of our idea and the actual distribution of model logits. (a) is a conceptual diagram for calculating logits, illustrating that top-k logits other than MaxLogit can separate ID and OOD. Please also note that logits are sorted for each sample. (b) and (c) show the distribution of top-k logits for ID and OOD in the actual model. These models were trained using ImageNet-1K as the ID dataset. INaturalist is used as an OOD dataset to measure the performance of OOD detection. The red area represents ImageNet-1K and the blue area represents INaturalist. 2 Related Work 2.1 Score Design method The most basic approach to OOD detection is to design a scoring function that can separate ID and OOD based on the output of a pre-trained neural network model. Hendrycks et al. ( Hendrycks and Gimpel 2017 ) adopt a simple baseline using the maximum softmax probability. Similarly, MaxLogit ( Hendrycks et al. 2022 ) uses the maximum value of the predicted logits as the score. The energy score ( Liu et al. 2020 ) computes the logsumexp of logits. These methods are based on logits or the probability of neural networks. On the other hand, some studies use the features of the penultimate layer for OOD detection. Lee et al. ( Lee et al. 2018 ) use the Mahalanobis distance, which computes the distance of class-wise Gaussian distributions on training data. Sun et al. ( Sun et al. 2022 ) use the KNN method for OOD detection. In recent years, Wang et al. ( Wang et al. 2022 ) have proposed ViM, which is both logit-based and feature-based. In another straightforward approach, Yu et al. ( Yu et al. 2023 ) selected the valid layer for OOD detection using a feature ratio. More recently, GEN ( Liu et al. 2023 ) uses the top 10 percentile of sorted probabilities. TRIM ( Kim et al. 2024 ) uses the top-7 to top-16 sorted probabilities of models for OOD detection. However, softmax normalization compresses the logit distribution, especially when logits are close in magnitude. This obscures fine-grained differences among classes and may result in the loss of valuable semantic cues needed for OOD detection. Therefore we use the region of logits which contains high-level semantic information. 2.2 Training method Another approach to OOD detection is to focus on training model by OOD sample. Hendrycks et al. ( Hendrycks et al. 2019 ) tackle this by re-training using a new loss function that incorporates class label loss and out-of-distribution loss. As follow-up work, OECC ( Papadopoulos et al. 2021 ) were able to suppress excessive confidence in the model by adding a loss term that aligns confidence for in-distribution training samples with training accuracy. However, these training methods require real OOD data. VOS ( Du et al. 2022 ) uses pseudo-OOD sampled from the low-likelihood region of the class-conditional distribution. NPOS ( Tao et al. 2023 ) use non-parametric outlier synthesis, which does not make any distributional assumptions as to the ID embeddings. The use of a training method is the most effective strategy for OOD detection; however, it requires time-consuming procedures such as model re-training. And training with additional OOD datasets may negatively affect the model’s accuracy. 2.3 Enhancement methods Some studies focus on improving the accuracy of OOD detection

Read the complete text