Untrained CNNs Match Backpropagation at V1: A Systematic RSA Comparison of Four Learning Rules Against Human fMRI

Nils Leutenegger · 2026

Read Untrained CNNs Match Backpropagation at V1: A Systematic RSA Comparison of Four Learning Rules Against Human fMRI by Nils Leutenegger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-16875.txt.

Introduction

Leutenegger tests four learning rules in same networks against human fMRI using pattern match study. A fresh baseline splits the architecture's share, with models trained at 32 pixels and tested at 224 pixels. At V1 the untrained base exceeds backpropagation, zero point zero five three versus zero point zero two two, and the same holds at V2. That edge dies at the 32-pixel training resolution, showing it depends on evaluation scale. At LOC, BP, guess coding and plasticity exceed the base, while at IT no conditions can be told.

Catalog record

CREATORNils Leutenegger
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34280
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.16875

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Untrained CNNs Match Backpropagation at V1: A Systematic RSA Comparison of Four Learning Rules Against Human fMRI Nils Leutenegger Abstract A central question in computational neuroscience is whether the learning rule used to train a neural network determines how well its internal representations align with those of the human visual cortex. We present a systematic comparison of four learning rules—backpropagation (BP), feedback alignment (FA), predictive coding (PC), and spike-timing-dependent plasticity (STDP)—applied to identical convolutional architectures and evaluated against human fMRI data from the THINGS-fMRI dataset (720 stimuli, 3 subjects) using Representational Similarity Analysis (RSA). All models process stimuli at 224 × 224 224\times 224 resolution; results are averaged across 5 random seeds. Crucially, we include an untrained random-weights baseline that reveals the dominant role of architecture. At V1/V2, the untrained baseline exceeds backpropagation ( ρ = 0.076 \rho=0.076 vs. ρ = 0.034 \rho=0.034 ; Δ ​ ρ = + 0.044 \Delta\rho=+0.044 , p < 0.001 p<0.001 ), and STDP achieves the highest V1 alignment among trained rules ( ρ = 0.064 \rho=0.064 ). At LOC, only BP reliably exceeds the random baseline ( ρ = 0.012 \rho=0.012 vs. − 0.005 -0.005 , p < 0.001 p<0.001 ). At IT, all five conditions converge ( ρ = 0.008 \rho=0.008 – 0.014 0.014 ) with no significant pairwise differences among trained rules ( p > 0.05 p>0.05 , FDR-corrected). FA consistently produces the lowest alignment at V1, V2, and LOC ( ρ = 0.012 \rho=0.012 at V1, below all other conditions). Partial RSA confirms all effects survive pixel-similarity control. Seed variability is small relative to between-rule differences at V1/V2. These results demonstrate that early visual alignment is architecture-driven, learning rules differentiate only at intermediate areas, and all rules converge at the highest levels of the hierarchy. Keywords: representational similarity analysis, visual cortex, biologically plausible learning, random baseline, STDP, feedback alignment, predictive coding, fMRI, THINGS 1 Introduction Deep neural networks (DNNs) trained with backpropagation have become the dominant computational models of the ventral visual stream ( Yamins and DiCarlo, 2016 ) . A well-established finding is that the hierarchy of features learned by DNNs mirrors the hierarchy of representations from V1 through IT cortex, as quantified by representational similarity analysis (RSA) and related metrics ( Kriegeskorte et al., 2008 ; Schrimpf et al., 2020 ) . This correspondence has been interpreted as evidence that the objectives and inductive biases of deep learning capture something fundamental about visual processing. Yet backpropagation is neurobiologically implausible in at least three well-known respects: it requires symmetric forward and feedback weights (the weight transport problem), it relies on a global error signal propagated backward through the network, and the forward and backward passes must be temporally separated ( Crick, 1989 ; Lillicrap et al., 2020 ) . These observations have motivated work on biologically plausible alternatives, including feedback alignment ( Lillicrap et al., 2016 ) , predictive coding ( Rao and Ballard, 1999 ; Whittington and Bogacz, 2017 ) , and spike-timing-dependent plasticity ( Bi and Poo, 1998 ) . A key question is: does the choice of learning rule matter for brain alignment? If alignment arises from the architecture and objective rather than the weight-update rule, then alternative rules—or even no training at all—should yield comparable alignment. Prior work has compared individual learning rules to brain data ( Schrimpf et al., 2020 ) , but to our knowledge no study has systematically compared all four rules together with an untrained random-weights baseline on the same architecture, stimuli, and RSA pipeline. We address this gap with five conditions: BP, FA, PC, STDP, and a random-weights (untrained) baseline, all applied to identical CNNs trained on CIFAR-10, with layer-wise RDMs compared to human fMRI from the THINGS-fMRI dataset. Our main contributions are: (1) a head-to-head RSA comparison of four learning rules plus a random baseline at 224\times 224 resolution, averaged across 5 seeds; (2) the finding that V1/V2 alignment is architecture-driven, with the untrained CNN exceeding BP at V1 ( \Delta\rho=+0.044 , p<0.001 ); (3) the result that at IT, all trained rules converge and are statistically indistinguishable from each other; (4) the finding that only LOC shows a reliable learning effect, with BP being the sole rule to significantly exceed the random baseline; and (5) the consistent finding that FA produces the lowest brain alignment at V1, V2, and LOC. 2 Related Work RSA as a model evaluation tool. Kriegeskorte et al., 2008 introduced RSA as a framework for comparing representations across measurement modalities. The core idea is to represent each system’s response to a stimulus set as a pairwise dissimilarity matrix (RDM) and to assess model–brain alignment as the rank correlation between RDMs. RSA has since become a standard tool for comparing DNNs to neural data ( Yamins and DiCarlo, 2016 ; Schrimpf et al., 2020 ) . DNNs and the visual hierarchy. Task-optimized DNNs reproduce the representational hierarchy of the ventral visual stream, with early layers matching V1 and deeper layers matching IT ( Yamins and DiCarlo, 2016 ) . The Brain-Score benchmark ( Schrimpf et al., 2020 ) systematizes this evaluation across architectures and datasets. Most high-scoring models are trained with standard BP on ImageNet. Importantly, Schrimpf et al., 2020 noted that untrained networks retain some brain alignment, but did not systematically compare this against multiple learning rules. Biologically plausible learning rules. Feedback alignment ( Lillicrap et al., 2016 ) replaces the transpose weight matrix in the backward pass with fixed random matrices, removing the weight transport problem. Lillicrap et al., 2016 showed that FA still allows learning in fully-connected networks, though its effectiveness in convolutional architectures is less established. Predictive coding ( Rao and Ballard, 1999 ) frames inference as hierarchical prediction-error minimization; Whittington and Bogacz, 2017 showed that PC implements an approximation to BP under certain conditions, with learning driven entirely by local Hebbian updates. Spike-timing-dependent plasticity ( Bi and Poo, 1998 ) updates synaptic weights based on the relative timing of pre- and postsynaptic spikes, and is one of the best-characterized forms of synaptic plasticity in mammalian cortex. Each of these rules has been studied individually in the context of neural representation learning, but direct comparisons on identical architectures and evaluation pipelines are rare. Random networks and early visual cortex. Saxe et al., 2011 demonstrated that random-weight networks can exhibit non-trivial visual structure. Concurrently with our work, Truzzi and Cusack, 2025 showed that random-weight CNNs predict early visual cortex responses when model complexity is sufficient. Our work extends these observations by systematically comparing four distinct learning rules against the random baseline, and by demonstrating that the architecture-dominance effect is not limited to V1: at IT, all rules—including random—converge with no significant pairwise differences. Sparse coding and V1. Olshausen and Field, 1996 showed that unsupervised learning of a sparse code for natural images produces receptive fields resembling V1 simple cells, establishing that correlation-based learning from image statistics suffices to produce V1-like representations without supervised objectives. 3 Methods 3.1 Dataset We used the THINGS-fMRI dataset ( Gifford and Cichy, 2022 ) , comprising 7T fMRI responses to images from the THINGS object concept database ( Hebart et al., 2019 ) . We selected 720 stimuli (one image per concept). fMRI responses were available for 3

Read the complete text