Just Noticeable Difference Modeling for Deep Visual Features

Rui Zhao, Wenrui Li, Lin Zhu, Yajing Zheng, Weisi Lin · 2026

Read Just Noticeable Difference Modeling for Deep Visual Features by Rui Zhao, Wenrui Li, Lin Zhu, Yajing Zheng, Weisi Lin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21933.txt.

Introduction

Rui Zhao and co-authors ask how much a deep visual feature can be distorted before the change becomes noticeable. Deep features, the internal representations of vision networks, now serve as the interface in many systems. Machines, not just eyes, consume images and features.

The just noticeable difference is the largest distortion that stays imperceptible, whether to human or machine vision. Modeling it for deep features, as this 2026 preprint does, lets engineers describe feature quality and control it. That control matters for compression and transmission: bits spent below the noticeable threshold are wasted. While damage above it degrades the systems that read the features downstream.

Catalog record

CREATORRui Zhao, Wenrui Li, Lin Zhu, Yajing Zheng, Weisi Lin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21933

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Just Noticeable Difference Modeling for Deep Visual Features Rui Zhao, Wenrui Li, Lin Zhu, Yajing Zheng, Weisi Lin Abstract Deep visual features are increasingly used as the interface in vision systems, motivating the need to describe feature characteristics and control feature quality for machine perception. Just noticeable difference (JND) characterizes the maximum imperceptible distortion for images under human or machine vision. Extending it to deep visual features naturally meets the above demand by providing a task-aligned tolerance boundary in feature space, offering a practical reference for controlling feature quality under constrained resources. We propose FeatJND , a task-aligned JND formulation that predicts the maximum tolerable per-feature perturbation map while preserving downstream task performance. We propose a FeatJND estimator at standardized split points and validate it across image classification, detection, and instance segmentation. Under matched distortion strength, FeatJND-based distortions consistently preserve higher task performance than unstructured Gaussian perturbations, and attribution visualizations suggest FeatJND can suppress non-critical feature regions. As an application, we further apply FeatJND to token-wise dynamic quantization and show that FeatJND-guided step-size allocation yields clear gains over random step-size permutation and global uniform step size under the same noise budget. Our code will be released after publication. 1 Introduction As computer vision models and systems rapidly evolve, an increasing number of applications no longer treat raw images as the only exchangeable entity. Instead, deep visual features are becoming a common interface both within and across systems ( 11 ) . In edge-cloud deployments, features are extracted on-device and transmitted for downstream inference ( 39 ) ; in foundation-model and multi-task pipelines, modules are composed and reused through deep features ( 6 ) ; and in distributed training and inference, intermediate states often become a dominant bottleneck ( 18 ) . As a result, features are emerging as the “information currency” of modern vision systems. They carry the semantic and structural content that governs task performance, while simultaneously driving bandwidth, latency, memory, and compute costs. This motivates the need for a task-aligned tolerance threshold that characterizes the maximum feature distortion permissible without inducing a perceptible degradation in downstream performance. This can also guide us to control feature quality under constrained resources. Figure 1 : Illustration of the just noticeable difference for deep visual features (FeatJND). In this paper, we first construct the concept of FeatJND, and then evaluate it on different tasks, and verify its effectiveness on a token-wise quantization application, which means spatially-varying step sizes shared across channels. In practical feature-based vision systems, we often need to compress ( 10 ) , quantize ( 17 ) , or downsample intermediate features to meet system constraints such as bandwidth and latency. A simple approach is to measure feature distortion using simple metrics, e.g., mean squared error (MSE). However, such metrics are often insufficient to reflect the task-level impact. Distortions of the same magnitude can lead to drastically different drops in downstream accuracy depending on where they occur in the representation, while in other cases, large numeric discrepancies may leave the final prediction almost unchanged. In other words, the relationship between feature-space distortion and task performance is not stable, making an “acceptable distortion boundary” difficult to define. As a result, system-level resource allocation and quality control are often driven by heuristic tuning, which can limit the interpretability and generalization across tasks and models. Essentially, we need a threshold-based characterization of feature quality for machine vision. In the image domain, perceptual thresholding under the human vision system (HVS) provides a canonical paradigm: Just Noticeable Difference (JND) ( 27 ) describes the boundary at which distortions become just perceptible to humans. Building on this, several works have developed methods to predict JND maps ( 22 ; 34 ) and to leverage them in vision tasks ( 36 ; 42 ) , where they allocate more bits or computation to perceptually sensitive regions. The success of JND highlights the value of threshold-based formulations: they offer an operational notion of a quality boundary that can directly guide system-level resource allocation. Traditional JND is defined for image distortions under human perception, while in machine vision, the relevant perception is reflected by downstream task performance. Some recent studies explore JND for machine vision ( 23 ; 41 ) , typically by defining imperceptibility through negligible changes in task metrics. However, they remain in the image domain. As deep features become the objects that are directly transmitted, reused, and compressed, a key question arises: Does a JND-like tolerance boundary also exist in deep feature space? In this paper, we explore Feature-JND (FeatJND) to model the feature quality boundary for machine perception. Given a feature and its downstream task, we predict a FeatJND map as a perturbation of the feature, where we aim to keep the task performance drop within a small tolerance. To probe the machine-perception boundary, we also require the predicted FeatJND perturbation to be as large as possible while remaining imperceptible to the downstream task. Our contributions can be summarized as follows. (1) We propose FeatJND, a just noticeable difference formulation for deep visual features that predicts the maximum tolerable perturbation while preserving downstream outputs. (2) We develop a simple but effective FeatJND estimator and validate it on image classification, detection, and instance segmentation with diverse backbones. (3) We show the effectiveness of FeatJND via matched-distortion-strength evaluations, attribution visualizations, and an application in token-wise feature quantization. 2 Related Work Just Noticeable Difference (JND). JND originated from the Human Visual System (HVS) ( 21 ) , aiming to characterize the boundary at which distortions become just perceptible. In the image domain, several works have explored obtaining fine JND maps, ranging from modeling of visual mechanisms ( 40 ; 4 ) to data-driven approaches that incorporate semantic cues ( 35 ) , frequency information ( 34 ) , etc. The JND maps have been adopted in applications such as image compression ( 36 ; 42 ; 25 ) and image quality assessment ( 32 ) to distinguish and handle areas with different sensitivities. More recently, a few studies have explored JND from a machine-perception perspective ( 23 ; 41 ) , formulating the boundary as the maximum perturbation under a performance constraint. However, these efforts primarily focus on the image domain, and feature JND remains relatively underexplored. Feature Quality Modeling. Feature-JND can be regarded as a way of characterizing feature quality. In recent machine-oriented image/video compression and collaborative inference research ( 8 ) , several works have attempted to measure feature quality. Some works simply use the MSE between original and compressed features as the quality metric for system optimization ( 5 ; 24 ) , assuming that smaller numeric deviation implies better representations. Some other works tie feature quality to downstream task performance, using the drop in task accuracy after compression to quantify feature quality ( 1 ; 9 ) . Both types of formulations typically depend on a specific network architecture and task setup, making them difficult to transfer and reuse across tasks and scenarios. Adversarial Attack. Adversarial attacks study how to optimize a targeted perturbation to maximize the loss or induce erroneous predictions under given constraints, which can be categorized as white-box and black-box. In the white-box setting ( 29 ; 30 ; 2 )

Read the complete text