Interpreting V1 Population Activity via Image–Neural Latent Representation Alignment

2026

Read Interpreting V1 Population Activity via Image–Neural Latent Representation Alignment (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-04309.txt.

Introduction

Xin Wang and five co-authors present a framework for reading what populations of neurons in the primary visual cortex are doing. Recent methods can decode images from brain activity well, but they reveal little about the computations behind that success.

Their dual-tower system aligns images and neural responses in a shared space at the level of intermediate feature maps, keeping those maps open to inspection. On large mouse imaging data, it decodes accurately while showing that coarse, low-level visual structure carries most of the signal, rather than fine detail or object category. That insight helps link decoding back to biology.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.04309

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Interpreting V1 Population Activity via Image–Neural Latent Representation Alignment Abstract Understanding the neural mechanisms underlying visual computation has long been a central challenge in neuroscience. Recent alignment-based approaches have improved the accuracy of decoding visual stimuli from brain activity, yet they provide limited insight into the neural computations that give rise to these improvements. To address this gap, we propose Dual-Tower Image–Neural Alignment (DINA), an interpretable contrastive framework for analyzing population-level visual computations in primary visual cortex (V1). DINA jointly trains a biologically motivated dual-tower architecture that aligns visual stimuli and corresponding V1 population responses in a shared latent space at the level of intermediate feature maps, enabling both accurate decoding and direct access to interpretable feature maps. Evaluated on large-scale two-photon calcium imaging data from mouse V1, DINA achieves accurate neural-based decoding while revealing that decoding performance is primarily supported by coarse, low-level visual structure, rather than semantic category information or fine-grained details. Further analysis reveals that alignable feature maps emerge from multiple spatially distributed image regions, capturing both shape and texture cues, and are predominantly reconstructed by sparse subsets of strongly responsive neurons and their functional interactions. Together, these results confirm that, beyond enabling accurate decoding, DINA provides a principled framework for probing the computational mechanisms underlying visual processing in V1. Code is available at GitHub repository . Xin Wang 1 Zhuangzhi Gao 2,3,4 Hongyi Qin 5,3 Zhongli Wu 3 Feixiang Zhou †,3,4 He Zhao †,3,4 1 School of Psychological and Cognitive Sciences, Peking University, Beijing, China 2 Department of Primary Care and Mental Health, University of Liverpool, Liverpool, United Kingdom 3 Liverpool Centre for Cardiovascular Science, University of Liverpool, Liverpool, United Kingdom 4 Department of Eye and Vision Sciences, University of Liverpool, Liverpool, United Kingdom 5 Institute of Life Course & Medical Sciences, University of Liverpool, Liverpool, United Kingdom † Corresponding authors: Feixiang Zhou [email protected] ; He Zhao [email protected] Figure 1: Illustration of feature-level contrastive alignment. Visual stimuli are transformed into intermediate image feature maps, while neural activities reconstruct intermediate neural feature maps. Encoding and decoding can be viewed as aligning these heterogeneous representations in a shared feature space, raising the central question of how image-derived and neural-derived feature maps can be aligned. Moreover, the resulting explainable feature maps enable interpretable analyses of population-level neural computation mechanisms. 1 Introduction Understanding how visual information is processed by neural populations is a central goal of visual neuroscience. A key system for studying this problem is the primary visual cortex (V1), the first cortical stage of visual processing in the brain. V1 receives input from the retina via the thalamus and transforms visual signals into structured neural representations. Extensive prior work has established that V1 neurons exhibit selective tuning to basic visual features such as edges, orientations ( 13 ; 14 ) , and spatial frequency ( 6 ) . While these studies have provided important insights, they have largely focused on the tuning properties of individual neurons. Recent advances in recording technologies, particularly large-scale two-photon calcium imaging, now allow simultaneous measurements from large neural populations, shifting the study of neural computation from the single-neuron level to the population level. At the population level, two complementary approaches are commonly used to study neural computation ( 18 ; 23 ; 22 ) . Encoding approaches aim to predict neural responses from visual stimuli, providing insight into how visual information is represented in the brain. Decoding approaches, in contrast, attempt to reconstruct or identify visual stimuli from neural population activity, revealing what information can be read out from neural representations. While both paradigms have proven useful, decoding performance alone provides limited insight into the structure of the underlying representations or the computational strategies that give rise to them. Existing encoding and decoding approaches typically model the relationship between visual stimuli and neural responses at the level of raw images ( 38 ; 29 ; 37 ; 7 ) . However, neural responses do not directly represent visual images themselves; instead, they encode intermediate feature representations of the stimulus. This distinction is particularly important in early visual areas such as V1, where neural activities are shaped by local receptive fields and predominantly encode low-level visual structure. As illustrated in Fig. 1, both encoding and decoding can be viewed as operating on feature maps, rather than on raw images. Thus, a central question is how to align visual feature representations extracted from images with feature maps read out from neural population activity, such that corresponding image-neural pairs are close in latent space, while non-corresponding pairs are well separated. This feature-map-level alignment provides a natural entry point for probing population-level computational principles beyond decoding accuracy. More broadly, this perspective suggests that contrastive alignment should not be viewed merely as a means to improve decoding performance, but as a general analytical tool for dissecting population-level neural computation. To operationalize this idea, we leverage large-scale two-photon calcium imaging recordings from mouse V1 and propose Dual-Tower Image–Neural Alignment (DINA), a framework that explicitly mirrors key computational motifs of the visual system. This biologically motivated design reflects the structure of cortical visual processing, such that feature maps extracted by the model are more likely to resemble those computed by the brain. The image tower incorporates parallel local and global processing pathways, and the neural tower models neuron-specific properties and population-level interactions. By aligning the latent feature maps produced by the two towers using a contrastive objective, the framework enables accurate decoding while obtaining intermediate representations suitable for probing population-level computational mechanisms in V1. Using this framework, we show that successful image-neural alignment in V1 is primarily supported by coarse, low-dimensional visual structure rather than semantic category or high-frequency detail. Through systematic and multi-level interpretability analyses of the aligned feature maps, we further find that image–neural alignment arises from multiple spatially distributed regions of the stimulus, within which diverse visual features, including both shape and texture cues, are jointly represented. These feature maps are read out predominantly by sparse subsets of strongly responsive neurons and a small number of functionally important interactions. Figure 2: Architecture of the DINA model. An image tower and a neural tower independently project visual stimuli and V1 population responses into feature maps that are matched in dimensionality and aligned using a contrastive objective. Our key contributions are as follows: • We formulate that contrastive image-neural alignment can be used not only for decoding, but as a general analytical tool for probing population-level neural computation. • We propose a biologically grounded dual-tower image-neural model inspired by cortical visual processing, which achieves reliable image decoding while producing aligned feature maps in V1. • Multi-level interpretability analyses of aligned image-neural feature maps reveal low-level visual structure, spatially distributed stimulus contributions, and sparse neural readout mechanisms in V1 population coding. 2 Related Work Prior work has largely focused on visual encoding, decoding, and alignment-based approaches as separate directions, with an emphasis on predictive performance. This section reviews these lines of work and shows how our approach bridges them through an interpretable alignment framework to probe population-level visual computation in primary visual cortex. 2.1 Encoding and Decoding Models in Visual Cortex Early encoding models were based on hand-crafted feature representations, such as linear–nonlinear (LN) models ( 15 ) and Gabor-based

Read the complete text