Deep Psychovisual Image Representations

Wendi Ma, Aryaman Sharma, Wei Dai, Shekhar S. Chandra · 2026

Read Deep Psychovisual Image Representations by Wendi Ma, Aryaman Sharma, Wei Dai, Shekhar S. Chandra (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-29260.txt.

Introduction

Wendi Ma proposes Deep Visual Coding, a frequency-domain code inspired by human vision. Psychovisual models suggest vision first forms middle-level abstractions, yet deep vision models usually stack uniform spatial layers, leaving choices opaque. Deep Visual Coding instead learns spectral filters that encode key structures in distinct sub-bands, paired with complex-valued images. Salience checks show these models extract readable object parts, unlike the blurry regions from standard convolutional networks. They are also less depth dependent than standard networks.

Catalog record

CREATORWendi Ma, Aryaman Sharma, Wei Dai, Shekhar S. Chandra
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70040
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.29260

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deep Psychovisual Image Representations Wendi Ma, Aryaman Sharma, Wei Dai, Shekhar S. Chandra Abstract Psychovisual models suggest human vision decouples low-level feature extraction from higher cognition by first forming intermediate abstractions. In contrast, deep learning-based vision models routinely extract and aggregate features using homogeneous stacks of spatial layers, rendering their decision-making processes opaque. In this paper, we propose DVC ( DVC ), a learned frequency-domain representation inspired by 1990s image codes that quantised perceptually salient frequencies, which together with complex-valued image representations produces psychovisual-style abstractions. This approach enables the first psychovisual-based deep learning framework, utilizing data-driven spectral filters that learn to encode task-relevant semantic structures within distinct frequency sub-bands. Salience analyses reveal that our psychovisual models extract highly interpretable object parts compared to the amorphous regions produced by regular CNN . Furthermore, we find that our models are less depth dependent than CNN for model scaling, since our complex-valued representations and learned abstractions subsume the role of deep spatial layers. Together, these findings demonstrate that psychovisual coding provides a promising path toward more efficient and transparent vision models. Figure 1 : (a) Standard CNN use deep stacks of homogeneous spatial layers. (b) Psychovisual models suggest human vision uses explicit intermediate abstractions. (c) Our psychovisual deep learning framework first produces rich complex-valued representations of interpretable semantic regions. They are then encoded in the frequency domain by DVC ( DVC ) , a data-driven adaptation of hand-crafted psychovisual coding schemes, to introduce psychovisual-like abstractions into deep learning models. 1 Introduction Ever since the ImageNet challenge popularised deep learning for computer vision [ 31 , 9 ] , architectural advances have focused on the design of spatial domain feature extractors, from convolution layers [ 22 , 60 , 24 , 40 ] to more recent token mixers based on attention mechanisms [ 11 , 56 , 13 , 47 ] . These models refine and aggregate image features through deep, homogeneous stacks of spatial layers, where feature processing emerges implicitly through training (Figure 1 (a)). As a result, intermediate representations are typically produced ad hoc with little explicit structure or organisation. Psychovisual processing (Figure 1 (b)), the way human vision encodes and interprets visual information, separates feature extraction from higher cognition using intermediate abstractions, such as objects, relations, and categories, providing a natural basis for reasoning [ 45 , 30 , 46 , 33 ] . In the 1990s, psychovisual coding schemes were developed that targeted the perceptually salient frequencies found by human vision studies [ 52 , 19 , 50 ] (see Appendix B.1 for a brief introduction). We hypothesize that these perceptually salient frequency selections may implicitly capture the high-level semantic abstractions utilized by human cognition. In this work, we develop a data-driven adaptation of psychovisual coding that introduces abstraction-like representations into deep learning frameworks (Figure 1 (c)). To the best of our knowledge, our work represents the first data-driven exploration of the frequency domain for high-level representation learning in vision, whereas previous studies primarily focused on lower-level feature learning or parameterizing spatial models [ 5 , 49 , 61 ] . Furthermore, our approach processes feature abstractions almost entirely in the frequency domain without the need for repeated spatial-to-spectral transformations. To summarise, the key contributions of this work are: • We propose DVC ( DVC ) , a first-of-its-kind data-driven adaptation of psychovisual coding schemes designed to introduce human-vision-like abstractions into deep learning models. It employs learnable, band-limited frequency filters that learn representations of high-level semantic image information, emerging as sparse selections of coronal frequency sub-bands in the DFT ( DFT ) (see Appendix B.2 for a brief introduction). • We introduce Phasor Block which has the ability to learn complex-valued features as phase information from real-valued coloured signals to build complex-valued image features that are well suited to the Fourier domain and the proposed DVC . • We assemble DVC and Phasor Blocks into a cohesive psychovisual deep learning framework. We demonstrate that this pipeline enables interpretable psychovisual-like processing, maintaining or improving performance across multiple classification benchmarks. • We provide salience analysis that demonstrate evidence of semantic decision making that reveal intermediate spatial layers consistently focus on meaningful object parts. These features are then encoded by DVC and used for final output prediction, mirroring the use of abstractions to support reasoning in a psychovisual model of human vision. 2 Background Biologically-inspired Vision. Biologically inspired approaches in computer vision predominantly focus on modelling early vision stages. In particular, much attention has been given to RF - regions of visual stimuli that elicit strong neural responses in the visual cortex. Mammalian RF are known to act as directional differential operators, closely resembling traditional image processing functions like wavelets and Gabor filters [ 42 , 26 , 48 ] . These parallels motivated their use in approximating low-level human vision, serving as effective feature extractors for basic visual structures like edges and shapes. In deep learning, these functions have been used to build neural networks that mimic cortical pathways [ 38 ] , and early-layer CNN kernels also perform similar directional operations [ 31 , 49 ] . However, networks that directly incorporate these functions, for example as fixed filter banks or bases for convolution kernels, have generally been restricted to small-scale problems and do not scale well to modern vision tasks [ 39 ] . Beyond RF , human vision has also been studied from a frequency domain perspective in previous image coding literature. In particular, our work draws upon psychovisual coding, introduced further below, as the basis for modelling abstractions used in higher vision stages and supports models which do scale to modern tasks. Frequency Domain Learning. Deep learning computer vision models have largely focused on processing in the spatial domain, which expresses localized relationships through contiguous pixel neighborhoods. The frequency (Fourier) domain, conversely, projects signals onto orthogonal basis functions, inherently capturing global representations (a review of the frequency domain and Fourier Transforms is presented in Appendix B.2 ). Formulated in this space, traditional image processing functions like ridgelets [ 3 ] , curvelets [ 55 ] and contourlets [ 10 ] have appealing sparse representations. In fact, they bear strong resemblances to psychovisual codes since they target specific selections of sub-bands, corresponding to features from different spatial scales. In deep learning, the frequency domain has primarily been used to exploit the Convolution Theorem [ 14 ] , whereby spatial circular convolution becomes simple element-wise multiplication in the frequency domain. Many works leveraging this property are performance-driven: [ 36 , 5 , 17 ] use frequency domain filters to accelerate CNN and incorporate global context, while [ 47 , 35 , 25 ] employ global frequency filters as lightweight and effective token mixers for transformer-style models. Other studies use frequency domain re-parameterizations of CNN to analyse model properties and behaviours [ 49 , 16 , 27 ] , such as optimal kernel structures. However, while there is extensive work exploiting the frequency domain for computational efficiency and model analysis, it has remained largely unexplored as an explicit representation space in its own right, particularly for high-level semantic features. Psychovisual Coding. Research conducted in the 1990s by Dominique Barba and colleagues used psychophysical experiments to characterise the frequency sensitivities of the

Read the complete text