Separable neural architectures as a primitive for unified predictive and generative intelligence

Reza T. Batley, Apurba Sarker, Rajib Mostakim, Andrew Klichine, Sourav Saha · 2026

Read Separable neural architectures as a primitive for unified predictive and generative intelligence by Reza T. Batley, Apurba Sarker, Rajib Mostakim, Andrew Klichine, Sourav Saha (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12244.txt.

Introduction

Batley, Sarker, Mostakim, Klichine, and Saha propose separable neural architectures as a shared base unit for AI. Many systems hide structure that splits into factors, which monolithic networks never state outright. Their class unifies additive, quadratic, and tensor-decomposed models under one form. Limits on interaction order and tensor rank split big maps into low-arity parts. Separability often appears only after a change of coordinates, a point their frame makes central. Tests span navigation, microstructure design, turbulent flow, and language modeling.

Catalog record

CREATORReza T. Batley, Apurba Sarker, Rajib Mostakim, Andrew Klichine, Sourav Saha
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE85026
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.12244

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Separable neural architectures as a primitive for unified predictive and generative intelligence Reza T. Batley, Apurba Sarker, Rajib Mostakim, Andrew Klichine, Sourav Saha Abstract Intelligent systems across physics, language and perception often exhibit factorisable structure, yet are typically modelled by monolithic neural architectures that do not explicitly exploit this structure. The separable neural architecture (SNA) addresses this by formalising a representational class that unifies additive, quadratic and tensor-decomposed neural models. By constraining interaction order and tensor rank, SNAs impose a structural inductive bias that factorises high-dimensional mappings into low-arity components. Separability need not be a property of the system itself: it often emerges in the coordinates or representations through which the system is expressed. Crucially, this coordinate-aware formulation reveals a structural analogy between chaotic spatiotemporal dynamics and linguistic autoregression. By treating continuous physical states as smooth, separable embeddings, SNAs enable distributional modelling of chaotic systems. This approach mitigates the nonphysical drift characteristics of deterministic operators whilst remaining applicable to discrete sequences. The compositional versatility of this approach is demonstrated across four domains: autonomous waypoint navigation via reinforcement learning, inverse generation of multifunctional microstructures, distributional modelling of turbulent flow and neural language modelling. These results establish the separable neural architecture as a domain-agnostic primitive for predictive and generative intelligence, capable of unifying both deterministic and distributional representations. Keywords: separable neural architectures, tensor decomposition, generative modelling, turbulence, metamaterials Figure 1: The separable neural architecture (SNA) as a unified primitive for predictive and generative intelligence. Top: The SNA formalises a representational class that constructs high-dimensional mappings by combining lower-arity learnable components (atoms) selected by an interaction tensor. By constraining interaction order and tensor rank, this formalism subsumes generalised additive, quadratic and tensor-decomposed neural models. Monolithic neural architectures have transformed artificial intelligence. The Transformer, with its ability to model long-range interactions across sequences, has achieved ubiquity in language modelling. Convolutional architectures remain highly effective for local feature extraction. However, systems across physical, linguistic and perceptual domains often exhibit latent factorisable structure that monolithic architectures leave implicit rather than exploit. Moreover, separability is often not a property of a system itself but of the coordinates or representations through which it is expressed. Accordingly, this work introduces the separable neural architecture (SNA) as a neural primitive: a rank- and interaction-controlled operator that serves as (i) a standalone model, (ii) a variational trial space, or (iii) a compositional module within larger intelligent systems. Formally, SNAs construct high-dimensional mappings from low-arity learnable components – atoms – whose interactions are governed by an interaction object that can be embedded as a sparse tensor. Expressivity is governed by two controls on the interaction object: its rank r and interaction order k , which together control the capacity and sparsity of the learned representation. SNAs thus define a representational class subsuming additive, quadratic and tensor-decomposed neural models within a single formalism. This primitive permits separable structure to be exploited where it arises, whether explicit, induced by a latent coordinate system or embedded within a larger architecture. When applied at the level of representation, SNAs enable continuous token embeddings that preserve neighbourhood relations in the underlying state space [ 3 ] . Adjacent physical states are thus adjacent in representation, a feature lacking in the discrete lookup embeddings of prevailing neural sequence models. For chaos precludes stable pointwise prediction over extended horizons; modelling must therefore be distributional to avoid nonphysical drift. Under this view, chaotic spatiotemporal dynamics and linguistic autoregression become structurally analogous: both benefit from modelling conditional distributions over sequentially revealed states. As a standalone model, the separable neural architecture realises compact predictive–generative intelligence in the form of KHRONOS [ 2 , 37 ] . KHRONOS instantiates an SNA whose low-rank, separable structure across all dimensions yields a smooth, cheaply invertible interpolant over the input space. Despite containing only hundreds of trainable parameters, it supports accurate prediction and rapid generative inversion to recover entire manifolds of admissible inputs consistent with a queried output. KHRONOS demonstrates that separable primitives can unify prediction and inversion within a single lightweight architecture able to operate in real time on commodity hardware. This same primitive extends naturally from predictive–generative modelling to variational learning by reinterpreting KHRONOS as a structured Galerkin trial space. In this setting, the SNA is trained directly from a governing operator, yielding a variational separable neural architecture (VSNA) over spatiotemporal-parametric domains. This demonstrates that SNAs may serve as physics-faithful computational representations capable of learning high-dimensional fields from governing operators. Thus, the present work introduces the SNA as a neural primitive for exploiting latent factorisable structure across intelligent systems (Fig. 1 ). The SNA is shown to unify predictive and generative modelling across domains. It serves as a standalone architecture enabling parsimonious predictive–invertive learning (KHRONOS); as a trial space for operator-driven learning of high-dimensional spatiotemporal-parametric fields (VSNAs); and as a compositional module within larger intelligent systems, enabling efficient autonomous navigation agents (SPAN), generative inversion of bicontinuous multiscale metamaterials (Janus) and continuous token embeddings for probabilistic sequence modelling (Leviathan). Separable neural architectures as a predictive–generative primitive Predictive and generative modelling The predictive–generative capacity of separable neural architectures is grounded in a specific subclass, arising when full interaction is permitted ( k=d ) and the interaction tensor is factorised into a rank- r canonical polyadic (CP) decomposition. In this CP-class, atoms factorise into products of univariate sub-atoms \psi . Consistent with the convention in generalised decompositions, each atom \phi constitutes a single mode : a standalone functional contribution to the global representation. Letting c^{(j)} denote modal weights and \rho an activation function, an element of this class may therefore be written \displaystyle f_{r}(x;\Theta_{CP})=\rho\left(\sum_{j=1}^{r}c^{(j)}\prod_{i=1}^{d}\psi^{(j)}(x_{i};\theta_{i}^{(k)})\right). (1) The CP-class is grounded in a concrete setting through KHRONOS [ 2 , 37 ] , a CP-class SNA adopting identity activation ( \rho(x)=x ), unit modal weights ( c^{(j)}\equiv 1 ) and B-spline subatoms. This particular CP-class network structure traces its lineage to the interpolating neural network (INN) [ 30 ] and its Hierarchical Deep Learning Neural Network (HiDeNN) predecessors [ 36 , 49 ] . KHRONOS has separately demonstrated 100-fold gains over Kolmogorov-Arnold Networks [ 23 ] on canonical PDE benchmarks [ 2 ] . On multi-fidelity aerodynamic field prediction, it achieves accuracy comparable to multilayer perceptrons (MLPs), graph neural networks (GNNs) and physics informed neural networks (PINNs) [ 34 ] with 94-98\% fewer parameters [ 37 ] . KHRONOS yields a smooth, separable interpolant over the input coordinate space. For a spline basis of order P over C_{i} interior cells in each dimension d , each subatom takes the form \displaystyle\psi^{(j)}(x_{i};\theta^{(j)}_{i})=\sum_{c=1}^{C_{i}+P}\alpha^{(j)}_{i,c}B^{P}_{c}(x_{i}). (2) The extended index C_{i}+P accounts for domain-exterior “ghost” cells required to preserve partition of unity [ 32 ] . KHRONOS is demonstrated on a process–structure inverse modelling problem investigated originally in [ 46 , 14 ] , linking thermal histories recorded during directed energy deposition (DED) of Inconel 718 thin-walls to spatially varying mechanical properties of the resultant print. Inverse modelling of 3D printed structures has been a challenge for engineers, as seemingly minor perturbations in the process may result in drastically varying mechanical properties within a single part. A schematic of the experimental setup for this problem is presented in Fig. 2 a . The raw thermal signals, recorded by infrared (IR) sensors, are stochastic and

Read the complete text