Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization

Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin · 2026

Read Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization by Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07483.txt.

Introduction

Leonel Aguilar and three co-authors ask when a neural network can generalize beyond its training data, and when it provably cannot. From one training window, they argue, the true process behind the data is not identifiable: many different processes fit the training data equally well yet diverge outside it. No test using only training data can choose between them.

What decides success, they show, is the structural commitment built into features, labels, and model class, often smuggled in through architecture, pretraining, or data formats. Changing only the representation changed out-of-distribution error by a factor of about 520 in their tests, at unchanged training loss. They check the same mechanism in chemistry, exoplanet, and DNA settings, plus a 264-run study of positional encodings.

Catalog record

CREATORLeonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE95 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07483

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Does Your Neural Network Extrapolate? Feature Engineering as Identifiability Bias for OOD Generalization Leonel Aguilar, Jan Nagler, Christoph Hoelscher, Nino Antulov-Fantulin Abstract Successful deep neural networks discover salient features of data. We show when and why they fail to learn out-of-distribution (OOD)-relevant representations from an in-distribution (ID) training window. This requires decoupling feature learning from data-generating-process (DGP) identifiability. From a single training window, OOD extrapolation is non-identifiable: infinitely many DGPs are ε \varepsilon -observationally equivalent on the training data but diverge arbitrarily outside it, and no in-distribution criterion alone reliably breaks the tie. A structural commitment, the feature map, label map, and model class ( φ , ψ , ℳ ) (\varphi,\psi,\mathcal{M}) , dictates the assumed DGP and governs OOD generalization while leaving ID performance essentially unchanged. When architecture, pretraining, augmentation, input formats, or domain knowledge implicitly inject the missing commitment, the model succeeds. When it cannot infer OOD-relevant structure from ID evidence, it fails. Changing only the representation can make the same architecture, at the same in-distribution loss, differ by ∼ 520 × {\sim}520\times out of distribution. When the commitment is correct and identifiable, OOD error vanishes. For example, Fourier coordinates turn periodic extrapolation into interpolation on 𝕊 1 \mathbb{S}^{1} . The same mechanism predicts outcomes in three natural-science settings (mass-action chemistry; Kepler’s-third-law exoplanet prediction, n = 2,362 n=2{,}362 ; and cross-species coding-DNA detection) and in a 264-run positional-encoding study across Transformer, Mamba, and S4D. Finally, a controlled study shows: correct features are necessary but not sufficient. The model class must express the target, and the transformed training data must cover the relevant representation space. Thus, feature engineering is not a departure from what makes deep learning successful. It is the explicit structural commitment that makes extrapolation identifiable. 1 Introduction Figure 1: Left: \sin(x) (green) and its degree-9 Taylor polynomial (black) can be observationally equivalent (depending on noise and amount of samples) on the training window and diverge on [\pi,3\pi] . Models (MLP, SINDy, TabPFN, TimesFM) trained/evaluated on the Original (raw) space without noise are shown. Right: in Fourier coordinates \varphi(x)=(\sin x,\cos x) , both (ID, OOD) map to the same circle \mathbb{S}^{1} , turning OOD extrapolation into interpolation. While the Universal Approximation Theorem guarantees that neural networks are exceptional interpolators within the support of their training data, they struggle with out-of-distribution extrapolation. Frameworks from Statistical Learning Theory provide guarantees for in-distribution generalization, however, achieving mathematically guaranteed extrapolation in deep learning remains an unsolved problem. We believe it is the right time to ask the question: When can neural networks extrapolate? We start our journey with a good old \sin(x) function. A neural network, 5 -hidden-layer MLP with 256 Tanh units, trained on raw f(x)=\sin(x) on [-\pi,\pi] ( Figure 1 ) extrapolates as a smooth, eventually-levelling curve with high OOD relative error while an identical architecture trained on \varphi(x)=(\sin x,\cos x) reaches OOD relative error below 1\% ( Table 1 ). The two have indistinguishable training loss; they differ only in the feature map applied before the model. Two assumptions, periodic and polynomial-like (Taylor is a valid hypothesis on [-\pi,\pi] ), can be observationally equivalent on the training window and diverge outside it. The indistinguishability problem is structural. For any finite training window W , infinitely many processes agree on W to any fixed tolerance and disagree arbitrarily outside it. Section 3 formalises this as a worst-case lower bound on any in-distribution test, and cross-validation (CV) falls short empirically even outside that worst case ( Section 6.1 ). The modeller’s choice of representation space and model class breaks the tie. Correctness is relative to the model class. (\sin x,\cos x) is the correct \varphi for \sin(x) only because the downstream class favours linear combinations: it renders the target linearly expressible. A Fourier-series regressor would collapse \varphi to the identity. Correctness is a joint property of (\varphi,\psi,\mathcal{M}) , which is why we treat them as a single commitment ( Section 3 ). Concrete failures of correct \varphi paired with non-matching \mathcal{M} appear in Section 4 . Three conditions are jointly necessary: (a) a representation (\varphi,\psi) aligned with the DGP, (b) a model class that expresses the target between \mathcal{Z}_{x} and \mathcal{Z}_{y} , and (c) coverage of \mathcal{Z}_{x} sufficient to identify the model. (a)+(b) constitute the DGP commitment; (c) is its evidential prerequisite ( Section 7 ). The headline comparisons hold (b) and (c) fixed, isolating (a). Contributions: (i) We formalise observational equivalence on a finite window, identify (\varphi,\psi,\mathcal{M}) as the DGP commitment, prove ERM on W is structurally blind to the choice, and characterise near-boundary data of width \delta\geq\varepsilon/r as the minimum heuristic that breaks the blindness ( Section 3 ). (ii) We isolate \varphi , \psi , and \mathcal{M} in controlled experiments ( Section 4 ) on minimal pairs and on pretrained foundation models (TimesFM ( 4 ) , TabPFN ( 7 ) , governed by the same mechanism via input format). (iii) We validate cross-domain ( Section 5 ) on chemistry (mass-action kinetics), physics (Kepler exoplanets, n{=}2{,}362 ), and biology (cross-species coding-DNA-sequence (CDS) detection on six organisms), same prediction, all confirmed. (iv) We deliver three practitioner artefacts ( Section 6 ): near-boundary selection, the SINDy (sparse identification of nonlinear dynamics) ( 3 ) \delta_{\mathrm{OOD}} diagnostic, and a content/position-embedding (CE/PE) reframing verified architecture-invariant on Transformer ( 16 ) / Mamba ( 5 ) / S4D ( 6 ) ( 264 runs). (v) We map scope and limitations on a d -torus benchmark, a periodic synthetic testbed ( Section 7 ): correct \varphi is necessary but not sufficient; model class and coverage continue to constrain OOD generalisation. Extended versions of tables are given in the appendix. 2 Related Work Multi-environment domain generalisation, formalised by IRM ( 1 ) and catalogued by 9 , breaks ties through invariance penalties across training distributions. Our single-window setting offers no such auxiliary environments, leaving OOD risk non-identifiable from data alone, and the commitment (\varphi,\psi,\mathcal{M}) supplies the structural information that multi-environment access supplies elsewhere. The Rashomon set ( 2 ; 17 ) captures the same multiplicity at the model side: members agree on training data but disagree off-distribution, and (\varphi,\psi) acts as a canonicaliser that selects a Rashomon element whose extension matches the assumed DGP. A second body of work supplies the commitment implicitly via architecture or input format. PINNs ( 12 ) and Fourier Neural Operators ( 10 ) bake in a differential operator (constraint on \psi ) or a Fourier basis (a \varphi that turns extrapolation into spectral interpolation), helping precisely when the assumed structure matches the DGP and degrading otherwise ( Section 7 ). Sinusoidal PE ( 16 ) , RoPE ( 13 ) , and ALiBi ( 11 ) are partial periodic feature maps at the token-position level. Section 6.3 evaluates state-of-the-art architectures: Mamba ( 5 ) and S4D ( 6 ) . On the activation side, 18 addresses the periodic-extrapolation failures by crafting an activation function with explicit periodic inductive bias, an \mathcal{M} -side commitment. In our controlled experiments, we show foundation models, TabPFN ( 7 ) and TimesFM ( 4 ) , and how they inherit the same commitment problem through their input format: identical pretrained weights swing from chance to near-perfect when the user supplies a transformation (

Read the complete text