What Cohort INRs Encode and Where to Freeze Them

Vasiliki Sideri-Lampretsa, Sophie Starck, Robbie Holland, Julian McGinnis, Daniel Rueckert · 2026

Read What Cohort INRs Encode and Where to Freeze Them by Vasiliki Sideri-Lampretsa, Sophie Starck, Robbie Holland, Julian McGinnis, Daniel Rueckert (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08298.txt.

Introduction

Vasiliki Sideri-Lampretsa, Sophie Starck, Robbie Holland, Julian McGinnis, and coauthors study reusable neural signal encoders. Early layers of cohort-trained implicit neural representations can seed new signals. That speeds fitting and improves it. But nobody knew which layers transfer, or what they encode.

The team swept the freeze depth in SIREN and Fourier-feature networks. The best layer to freeze is the one with the highest weight stable rank. Freezing there matches or beats standard fine-tuning. The team then applied sparse autoencoders to encoder activations. That is the first such decomposition for these models. The results tell practitioners where to cut an encoder for reuse.

Catalog record

CREATORVasiliki Sideri-Lampretsa, Sophie Starck, Robbie Holland, Julian McGinnis, Daniel Rueckert
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08298

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What Cohort INRs Encode and Where to Freeze Them Vasiliki Sideri-Lampretsa, Sophie Starck, Robbie Holland, Julian McGinnis, Daniel Rueckert Abstract Reusing the early layers of cohort-trained INRs as initialization for new signals has been shown to accelerate and improve signal fitting, yet it remains unclear which layers of the shared encoder learn transferable representations and what those representations encode. We address both questions for two standard backbones, SIREN and Fourier-feature MLPs (FFMLP). First, sweeping the freeze depth across the shared encoder at test time, we find that the optimum coincides with the layer of highest weight stable rank. Moreover, freezing at this depth matches or improves on the standard fine-tuning recipe across all our experiments. Second, identifying which layer transfers does not characterize what that layer encodes. To address this we adopt sparse autoencoders (SAEs), the dominant tool in mechanistic interpretability, and present the first SAE decomposition of INR activations into sparse dictionary atoms. Interestingly, SIREN and FFMLP achieve comparable cohort-fitting quality, but learn qualitatively different dictionaries. Cohort SIREN’s atoms are localized, tiling the coordinate plane such that each atom fires in a confined region independent of cohort content. Cohort FFMLP’s atoms are image-spanning, tracing the contours of memorized cohort signals. Single-atom ablations confirm causal use of these dictionaries: a single FFMLP atom out of 4096 can drop PSNR by up to 10.6 dB across the image, while SIREN ablations remain confined to where the atom fires. Together, these results give the first mechanistic account of what transfers in cohort-trained INRs and turn their activations into inspectable dictionary atoms. These tools open a path towards characterizing what INRs encode and towards architectures designed for generalization rather than memorization. We plan to release our code upon acceptance. 1 Introduction Implicit neural representations (INRs) are coordinate-based multi-layer perceptrons (MLPs) that learn a continuous, functional approximation of signals. Popularized by their usage in NeRFs [ 42 ] , INRs have demonstrated their utility across diverse signal modalities including images [ 63 , 60 , 41 , 66 , 67 ] , scenes [ 42 , 1 ] , shapes [ 49 , 23 , 12 ] , videos [ 7 , 9 ] , medical data [ 39 , 58 , 20 , 68 ] , and signal compression [ 15 , 35 , 62 , 73 ] . A key distinction of INRs is that each network fits a single signal, with learned features finely tuned to that signal’s morphology, and every new signal requires optimization from scratch. Reducing this per-signal cost, by reusing structure learned from related signals, would significantly broaden the applicability of INRs. A recent line of work, STRAINER [ 66 , 67 ] , addresses this by training a cohort of INRs that share early encoder layers but maintain per-signal decoders. Once trained, the shared encoder serves as initialization for new signals. STRAINER’s test-time protocol updates every layer, so pretrained structure and test-time fitting cannot be separated, leaving per-layer transferability unmeasured. This raises two questions: which layers carry transferable representations, and what do those representations encode? Drawing inspiration from the pretrained vision encoder literature, where frozen-feature analysis has localized transferable representations to intermediate rather than output layers [ 4 , 17 , 53 ] , we sweep the test-time freeze boundary across all encoder layers of a cohort-trained INR. Whether this pattern holds for cohort-trained INRs has not been studied. The two dominant backbones, SIREN [ 60 ] and Fourier-feature MLPs (FFMLP) [ 63 ] , may not localize transferable structure at the same depth since they encode coordinate position through fundamentally different mechanisms (sinusoidal activations at every layer versus a random Fourier projection followed by ReLU activations). Figure 1 summarizes our two findings. Figure 1 : Which layers transfer, and what do they encode? Left: the optimal freeze depth \tau^{\star} in cohort-trained INRs coincides with the layer of peak weight stable rank in the shared encoder ( \tau^{\star}=1 for cohort SIREN, shown). Right: sparse autoencoders recover qualitatively different dictionaries: SIREN atoms are localized, tiling the coordinate plane. FFMLP atoms are image-shaped and fire diffusely, tracing memorized cohort content. To address the which question, we train cohort INRs on three datasets (CelebA-HQ, OASIS-MRI, Kodak) with two backbones (SIREN and FFMLP), and evaluate whether frozen features transfer from each source to each of the three target distributions. Sweeping the freeze boundary at test time, we find that the optimal freeze depth coincides with the layer with the peak stable rank of the weights in the cohort-trained encoder [ 54 , 40 ] . The weight stable rank is computable from the encoder alone, with no target dataset access or hyperparameter search. Although the two backbones encode coordinate position differently, both transfer best when frozen up to the first high-rank layer. In FFMLP this layer is the first trainable layer after the fixed Fourier projection, and its features reflect the Fourier basis [ 46 ] . In SIREN, where no such fixed basis exists, the first high-rank layer itself can be viewed as an emergent positional embedding learned from the data. To address the what question, we adopt sparse autoencoders (SAEs) [ 5 , 10 , 21 ] , the dominant tool for interpreting hidden activations in language and vision-language models. We apply SAEs to trained INRs, learning an overcomplete dictionary that decomposes activations into discrete atoms. Despite training on the same cohort to comparable quality, SIREN and FFMLP recover qualitatively different dictionaries (Figure 3 ). SIREN’s atoms tend to be localized, tiling the coordinate plane independently of cohort content. FFMLP’s atoms tend to be image-shaped, tracing memorized cohort signals. Single-atom ablations confirm the dichotomy is causal: SIREN ablations stay confined to where each atom fires, while FFMLP ablations propagate across the entire image, with single atoms causing PSNR drops of up to 10.6 dB (Fig. 5 ). We make the following contributions: 1. We establish that frozen features transfer across cohort-trained INRs. The optimal freeze depth coincides with the layer with the highest weight stable rank in the encoder. Freezing at this depth matches or improves on the standard fine-tuning recipe across all our experiments. 2. We present the first SAE decomposition of INR activations into sparse atoms, showing that SIREN and FFMLP recover qualitatively different dictionaries: localized atoms for SIREN vs. image-spanning atoms tracing memorized cohort content for FFMLP. 3. We demonstrate, through single-atom ablations, that these dictionaries causally drive the network’s output: SIREN ablations stay local to where the atom fires. FFMLP ablations propagate across the entire image, with single atoms causing PSNR drops of up to 10.8 dB. 2 Related Work Beyond the single-signal INR regime. Recent work tackles the cost of training INRs from scratch by learning priors over INR weights. Hypernetwork [ 60 , 8 ] and meta-learning [ 64 , 59 , 33 ] approaches map a code or initialization to target weights. Auto-decoder approaches [ 49 , 16 , 20 , 41 ] learn a shared network conditioned on per-signal codes or modulations, and a separate line embeds a learned prior into the INR itself by training on related signals [ 57 , 61 , 11 , 32 ] . Most relevant to our setup, STRAINER [ 66 ] trains a cohort

Read the complete text