Flowers: A Warp Drive for Neural PDE Solvers Till Muser, Alexandra Spitzer, Matti Lassas, Maarten V. de Hoop, Ivan Dokmanić Abstract We introduce Flowers , a neural architecture for learning PDE solution operators built entirely from multihead warps. Aside from pointwise channel mixing and a multiscale scaffold, Flowers use no Fourier multipliers, no dot-product attention, and no convolutional mixing. Each head predicts a displacement field and warps the mixed input features. Motivated by physics and computational efficiency, displacements are predicted pointwise, without any spatial aggregation, and nonlocality enters only through sparse sampling at source coordinates, one per head. Stacking warps in multiscale residual blocks yields Flowers , which implement adaptive, global interactions at linear cost. We theoretically motivate this design through three complementary lenses: flow maps for conservation laws, waves in inhomogeneous media, and a kinetic-theoretic continuum limit. Flowers achieve excellent performance on a broad suite of 2D and 3D time-dependent PDE benchmarks, particularly flows and waves. A compact 17M-parameter model consistently outperforms Fourier, convolution, and attention-based baselines of similar size, while a 150M-parameter variant improves over recent transformer-based foundation models with much more parameters, data, and training compute. 1 Introduction Figure 1 : The multi-head Selfwarp layer. (a) Given an input feature map u with C channels, (b) for each pixel x in the image the layer computes (c) a local per-head interaction v^{(h)}(x)\in\mathbb{R}^{C_{h}} via a linear (affine) projection of the input u(x) and (d) a displacement field (one per head) \varrho^{(h)}(x) , also pointwise, using a small MLP. Each C_{h} -channel per-head interaction field v^{(h)} (e) is warped along the same per-head displacement field \varrho^{(h)} (f) to compute the warped head output (g). The outputs of all heads are concatenated at the output (h). Deep neural networks have become credible surrogate solvers for PDEs. After training, they run fast, they are differentiable, and they can be conditioned on parameters, forcing, or geometry. What is less settled is what it takes for such models to behave physically rather than as data interpolants, to capture qualitative structure, roll out stably, and remain effective across physical regimes. Most neural PDE backbones rely on Fourier mixing, convolutions, or attention. Fourier layers and attention provide global coupling, but through dense mixing that treats long-range interactions as equally plausible a priori. Convolutions build a locality prior, but interactions are simple linear stencils with weak expressivity at the fringe of the receptive field. For many conservative phenomena u(t,x) depends on local interactions between “channels” plus a small set of upstream sources along characteristics or rays, in an adaptive, state-dependent way. Here we introduce Flower , a neural PDE architecture built (near-)exclusively from learned non-linear coordinate warps (or pullbacks). At each target location x , we predict a set of per-head displacements (\varrho^{(h)}(x))_{h=1}^{H} using only pointwise information at x , and induces nonlocal interaction by sampling features at the displaced coordinates x+\varrho^{(h)}(x) . Heads are mixed pointwise and composed in depth and scale. With a fixed number of heads the cost is linear in the number of grid points so scaling to 3D is easy. What we find is somewhat surprising: coordinate warps are a sufficient and scalable primitive to build state-of-the-art neural PDE solvers. One way to see why warps work is by considering the wave equation. In the high-frequency regime, the solution operator is a Fourier integral operator (FIO): it routes localized wave packets along rays (see Figure 2 ), which can be read as a sum of pullbacks, or warps, of the initial data. This observation motivated neural networks for waves ( 22 ) , but it illustrates a common pattern in scientific ML: while models that closely mirror mathematical structure of an operator class usually excel in a narrow set of tasks, they are usually not good neural networks in an engineering sense and thus have limited impact. Here our goal is to extract the key computational primitives from this line of work, guided by physics, and replace the inessential parts with careful deep learning engineering. Warps are not new in deep learning, but they appear as augmentations of more complex pipelines which involve convolution and/or attention layers (similarly as with FIO networks). The canonical example is the spatial transformer ( 19 ) . The same sampling primitive underlies deformable convolutions ( 10 ) , and reappears in deformable transformers ( 54 ) , where each query predicts a set of sampling offsets, which is shown to improve encoder–decoder object detection transformers. Related ideas have been used for dynamic upsampling ( 31 ) , image registration ( 2 ) , and scientific forecasting ( 53 ) ; in all cases, warping is a component of a convolutional or transformer pipeline. Motivated by physics, Flowers adopt a different perspective. We make warping the only adaptive nonlocal mixing primitive: each layer predicts a single per-head displacement \varrho^{(h)}(x) using only pointwise information at the target location u(x) , and induces nonlocal interaction by sampling features at x+\varrho^{(h)}(x) , one displacement per head. This keeps the operation lightweight, scalable, and grounded in characteristic structure of PDEs, with nonlocality entering only through sparse pullbacks. We then build a modern backbone around this primitive—multi-head organization, residual connections, and a U-Net-like multiscale scaffold. The empirical takeaway is that this one primitive is enough to obtain state-of-the-art neural PDE solvers across 2D and 3D benchmarks, without Fourier multipliers or dense dot-product attention. As we show, even a modestly sized Flower outperforms strong Fourier-, convolutional-, and attention-based baselines across diverse benchmarks spanning fluid dynamics, wave propagation, and complex geometries. On the The Well benchmark suite ( 45 ) , 17M-parameter Flowers achieve state-of-the-art supervised performance while being efficient to train and run. Scaling to \sim 100–150M parameters closes the gap to, and even improves over, very recent foundation-scale PDE models trained with far more parameters, data, and compute. (On several datasets even our “tiny” model achieves stronger performance.) This makes Flowers a plausible new family of scientific ML backbones, with considerable headroom: we believe that performance will only improve as the community explores the new design space. 1.1 Contributions (i) We introduce the Selfwarp layer and the Flower architecture, showing how to build effective neural PDE solvers from lightweight pointwise warps. (ii) We connect the design of Flowers to characteristic/ray structure of solution operators to key PDEs and to a natural kinetic theory (phase space) limit. (iii) We show that with a U-Net-like scaffold, Flowers outperform strong baselines across datasets and regimes including on almost entire “The Well”. (iv) We show that Flower easily scale to 3D. We also show that increasing the number of parameters to 150M makes them competitive with much larger and more resources foundation models. Finally, we mention that the design of Flowers makes them more interpretable than generic architectures. In Figure 6 we show an example of a warp field predicted by a trained Flower rather than computed “by hand”. It is remarkable that this field indeed “wants to create vortices”. 2 The Flower Architecture The design of Flowers is inspired by the structure of PDEs known as systems of conservation laws. Such systems are very general and include hydrodynamics (e.g. the Navier–Stokes equations). While the resulting architecture applies well beyond conservation laws, they provide a clean narrative to describe the core mechanics. We write u(t,x)\in\mathbb{R}^{C} for the vector of “conserved quantities”, also known as the state, whose evolution we want to predict; for compressible Navier–Stokes, u comprises density, momentum, and energy. A prototypical system is 1 1 1 Perhaps augmented with source and diffusive terms. \partial_{t}u(x,t)+\nabla_{x}\cdot G(u(x,t))=0, where G(u) are the so-called fluxes which describe how the conserved quantities flow in space. As we show in Section 3.1 , a key component of the solution operators for such systems is a simple pullback, or warp. We now introduce this basic component and the resulting architecture before moving to a more substantial theoretical motivation. 2.1 Architecture For clarity we present the architecture in continuous coordinates. Let \Omega\subset\mathbb{R}^{d} be the domain of interest and \mathcal{F}^{C}(\Omega):=\{u:\Omega\to\mathbb{R}^{C}\} the space of C -channel fields on \Omega . Given a \tau:\Omega\to\mathbb{R}^{d} we define the pullback (\tau^{*}v)(x):=v(\tau(x)) . We apply (broadcast) finite-dimensional maps pointwise in the usual way: for a C -channel field u and h:\mathbb{R}^{C}\to\mathbb{R}^{C^{\prime}} , we define (h[u])(x):=h(u(x)) . Concatenation along the channel dimension is denoted (u\oplus v)(x)=u(x)\oplus v(x) . Multihead warp (pullback layer) The main component of Flowers is a multihead warp. We begin with a single head. The layer \textsc{Selfwarp}:\mathcal{F}^{C_{\mathrm{in}}}(\Omega)\to\mathcal{F}^{C_{\mathrm{out}}}(\Omega) maps a C_{\mathrm{in}} -channel input field u to a C_{\mathrm{out}} -channel output w as \displaystyle w(x) \displaystyle=\textsc{Selfwarp}_{V,g}[u](x) \displaystyle\overset{\text{def}}{=}Vu(x+\varrho(x)) where \varrho(x)=g(u(x)) ; in concise pullback notation, u\mapsto w=(\mathrm{Id}+\varrho)^{*}(Vu). Crucially, the map u\mapsto\varrho is local in a strong sense, i.e, pointwise: \varrho(x) depends on u(x) but not on u(x^{\prime}) for x^{\prime}\neq x . The warping itself is nonlocal and nonlinear, but sparse , as it samples the input at a single point. A single deformation has limited expressivity so we build a multihead variant in the spirit of multihead attention. Fix H heads and C_{\mathrm{out}}=HC_{h} . Let f(a)=Va be a pointwise “value map” and let g:\mathbb{R}^{C_{\mathrm{in}}}\to(\mathbb{R}^{d})^{H} be a pointwise displacement map with components g^{(h)} . For u\in\mathcal{F}^{C_{\mathrm{in}}}(\Omega) define v:=f[u] and split it into heads v=v^{(1)}\oplus\cdots\oplus v^{(H)}\quad\text{with}\quad v^{(h)}\in\mathcal{F}^{C_{h}}(\Omega). We then (a) predict the H displacement fields, (\varrho^{(1)},\dots,\varrho^{(H)}):=g[u] We emphasize that (i) each \varrho^{(h)} may depend on all input channels (not only channels in head h ); and (ii) locality is preserved, i.e., \varrho^{(h)}(x) only depends on u(x) but not u(x^{\prime}) for x^{\prime}\neq x . (b) warp, or pull back the input features along the computed displacements, w^{(h)}:=(\mathrm{Id}+\varrho^{(h)})^{*}v^{(h)} (c) compute output by concatenating the heads \textsc{Selfwarp}_{V,g}[u]:=w^{(1)}\oplus\cdots\oplus w^{(H)}. All nonlocality enters through evaluating v^{(h)} at x+\varrho^{(h)}(x) , implemented as differentiable interpolation. Residual pullback block (flower block) We embed the multihead warp in a residual block. Given an input u we first computed a warped update \textsc{Selfwarp}_{V,g}[u] and add a skip connection via a 1\times 1 projection, u\mapsto\phi\circ\mathrm{Norm}\circ(\mathrm{Warp}_{V,g}[u]+\mathrm{IdProj})[u] where \mathrm{IdProj} is a 1\times 1 convolution, \phi is GELU, and \mathrm{Norm} is a GroupNorm. Multiscale composition Simply stacking these residual blocks already yields a strong backbone across a range of forecasting tasks (see Table 8 ), outperforming baselines and underscoring that learned warps alone are a sufficient primitive for effective PDE solvers. To facilitate modeling multiscale and long-range interactions present in complex dynamics we additionally scaffold FlowerBlocks in a U-Net-like multiscale composition. This is similar to multiscale vision transformers ( 32 ) and related to classical considerations from numerical PDEs. The details of this construction are laid out in Appendix D . As we show, it leads to the best empirical performance. 3 Theoretical Motivation Why are Flowers a natural architecture for learning solution operators of time-dependent PDEs, especially those governing flows and waves? We sketch three complementary perspectives that connect the architecture to classical structure in PDEs and numerical schemes. These arguments are primarily motivational: we do not claim that any one of those perspectives causally explains the architecture’s empirical success, but together they provide a coherent intuition and suggest routes towards more rigorous theory. Our hope is that the reader will find at least one of the three vignettes helpful and illuminating. 3.1 Scalar conservation laws Consider a scalar conservation law for u , \begin{cases}\partial_{t}u(t,x)+\nabla_{x}\cdot(G(u(t,x)))=0,\quad u(0,x)=u^{0}(x),\\[6.0pt] u:\mathbb{R}_{+}\times\mathbb{R}^{d}\to\mathbb{R},\qquad{G:\mathbb{R}\to\mathbb{R}^{d}}.\end{cases} Canonical examples include the Burgers equation and traffic-flow models. Since G depends on x only through u(t,x) , the chain rules yields the equivalent quasilinear transport form, sometimes called a nonlinear optical flow ( 1 ) , \partial_{t}u(t,x)+A(u(t,x))\cdot\nabla_{x}u(t,x)=0 where A(u)=\tfrac{d}{du}G(u) . This lets us introduce characteristics: define \Phi_{t}(x) by \partial_{t}\Phi_{t}(x)=A(u(t,\Phi_{t}(x))),\quad\Phi_{0}(x)=x. As long as a classical solution exists (i.e., prior to shock formation), u is constant along characteristics, u(t,\Phi_{t}(x))=u(0,x). Equivalently, u(t,x)=(\Phi_{t}^{-1})^{*}u^{0}(x). More generally, writing \Phi_{t\leftarrow s}\overset{\text{def}}{=}\Phi_{t}\circ\Phi_{s}^{-1} for the flow map from time s to t , we have u(t,\cdot)=(\Phi_{t\leftarrow s}^{-1})^{*}u(s,\cdot). The relevance to Flowers is immediate: the solution operator (locally in time) is a pullback of the state by a warping map \Phi_{t\leftarrow s}^{-1} . More can be said. Over a short step \Delta t we have the expansion \Phi_{t+\Delta t\leftarrow t}^{-1}(x)=x-\Delta tA(u(t,x))+O(\Delta t^{2}) so to first order the displacement needed to compute u(t+\Delta t,x) only depends on the local state u(t,x) , not on values u(t,x^{\prime}) at other locations x^{\prime}\neq x . This is precisely what our Selfwarp primitive does. Figure 6 provides empirical evidence: on the shear flow dataset, the learned displacements visibly align with the underlying fluid velocity, suggesting that Flower discovers physically meaningful transport directions. From a deep learning engineering point of view, the locality of the flow map results in parameter and compute efficiency. Generalizing these ideas to systems of conservation laws—especially those with constraints and additional structure, as in compressible Euler and Navier–Stokes—is substantially more delicate. In Appendix B we outline how Flowers can express the requisite components (multiple characteristic families, coupling, and source-like effects), by employing more than one head. In the next section we outline a different, wave perspective which requires multiple heads. 3.2 Waves and geometric optics Figure 2 : Wavefronts and rays picture: high-frequency waves propagate along rays. The solution at position x and time t is a sum of contributions along the wavefront, which can be indexed by head \eta . Consider the variable-speed wave equation \partial_{t}^{2}u-c(x)^{2}\Delta u=0, with Cauchy data u(0,\cdot)=u_{0} , \partial_{t}u(0,\cdot)=u_{1} . In the geometric-optics regime (and away from caustics), the solution at (t,x) is well approximated by a superposition of contributions arriving along rays. Equivalently, writing \Gamma_{-t}(x,\eta) for the point reached by tracing a ray backwards from x for time t with initial direction \eta\in\mathbb{S}^{d-1} , one obtains the pullback representation \displaystyle u(t,x)\;\approx \displaystyle\int_{\mathbb{S}^{d-1}}\widehat{W}_{0}(t,x,\eta)\,u_{0}\!(\Gamma_{-t}(x,\eta))\,d\eta (1) \displaystyle+\int_{\mathbb{S}^{d-1}}\widehat{W}_{1}(t,x,\eta)\,u_{1}\!(\Gamma_{-t}(x,\eta))\,d\eta. for smooth weights \widehat{W}_{0,1} (absorbing Jacobian/amplitude factors); Appendix F derives this standard parametrix from a Green’s function/WKB ansatz. Equation ( 1 ) matches a multihead pullback layer: each head samples the input at a displaced coordinate, and pointwise channel mixing supplies the weights. Approximating the angular integral by a learned quadrature yields \sum_{h=1}^{H}\alpha_{0,1}^{(h)}(t,x)\;u_{0,1}\!(\Gamma_{-t}(x,\eta^{(h)}(t,x))), (2) i.e., a sum of pullbacks. In Flowers , both the effective directions \eta^{(h)} ( \equiv displacements) and weights \alpha^{(h)} are produced pointwise from local lifted features at x (state, coefficients/geometry channels, coordinates, and metadata such as \Delta t ), yielding an adaptive alternative to uniform angular discretization. Multiple heads also naturally represent multiple arrivals/branches, with subsequent local mixing modeling interference and mode coupling. 3.3 A kinetic theory perspective Our third perspective is the natural continuum limit of Flowers . In this limit, the architecture can be interpreted as a discretization of a generalized Boltzmann-type equation, with a key twist: the streaming velocity is self-adaptive , i.e., it depends on the current lifted state over heads. This viewpoint is appealing because such kinetic (Boltzmann/Vlasov-type) equations serve as “master” descriptions from which many macroscopic models—including hydrodynamics—can be derived via suitable moments and closures. Moreover, the related kinetic numerical schemes such as the lattice-Boltzmann method ( 25 ) have a stream-and-collide ( \equiv locally interact) structure similar to Flowers ( 4 ) . We work formally and refer to Appendix E for details. Consider a single-scale cascade of flower blocks indexed by n=0,\dots,N , with time step \Delta t\overset{\text{def}}{=}T/N and t_{n}\overset{\text{def}}{=}n\Delta t . We take a regime in which the number of heads grows and the discrete head index can be replaced by a continuous phase variable \eta\in\mathbb{R}^{k} (more generally, \eta may live on a manifold that parameterizes transport modes). The network state becomes a lifted field u_{n}(x,\eta) , and in the joint limit N\to\infty , \Delta t\to 0 (with appropriate scaling of the per-layer displacements), one obtains a continuum evolution u_{n}(x,\eta)\to u(t,x,\eta) governed by \displaystyle\partial_{t}u(t,x,\eta) \displaystyle+\mathcal{B}\big(\eta;u(t,x,\cdot)\big)\cdot\nabla_{x}u(t,x,\eta) (3) \displaystyle=\mathcal{Q}_{t}\big(\eta;u(t,x,\cdot)\big). Here \mathcal{B} plays the role of a streaming (transport) velocity, while \mathcal{Q}_{t} is an interaction term capturing local channel mixing, head coupling, and any additional pointwise nonlinearities. A notable distinction from the classical Boltzmann equation is that \mathcal{B} is not fixed by an a priori velocity variable: it is predicted from the lifted state u(t,x,\cdot) itself, enabling the transport directions and speeds to adapt to the evolving solution. This suggests a very nice interpretation of heads as a phase-space lift: the phase variable \eta indexes transport modes, and Flowers can be viewed as learning dynamics in a lifted phase space (x,\eta) before projecting back to physical space. In particular, macroscopic quantities can be recovered as moments or learned projections over \eta , which parallels the kinetic-to-hydrodynamic viewpoint, while the adaptive streaming provides additional expressive power beyond standard fixed-velocity kinetic models. 4 Results Next-step 1:20 Rollout Collection Dataset FNO CNUnet scOT Flower FNO CNUnet scOT Flower The Well acoustic_scattering_maze 0.1454 0.0129 0.0361 0.0064 0.4197 0.0874 0.1996 0.0489 active_matter 0.1749 0.0650 0.1050 0.0249 3.2862 1.7781 4.1055 1.3905 gray_scott_reaction_diffusion 0.0372 0.0188 0.0673 0.0102 0.9125 0.3059 0.4169 0.2074 MHD_64 0.3403 0.2062 – 0.1165 1.3007 0.9534 – 0.7580 planetswe 0.0070 0.0027 0.0041 0.0007 0.1316 0.0624 0.0518 0.0187 post_neutron_star_merger 0.4452 0.3391 – 0.3269 0.5980 0.6529 – 0.6223 rayleigh_benard 0.2104 0.2171 0.1863 0.0807 39.038 12.507 5.6486 2.1661 rayleigh_taylor_instability 0.1714 0.1351 – 0.0491 3.0057 5.3894 – 0.5862 shear_flow 0.0769 0.0594 0.1093 0.0463 1.1245 0.7632 0.8930 0.2246 supernova_explosion 0.4326 0.4316 – 0.2888 1.0162 2.2913 – 0.8113 turbulence_gravity_cooling 0.2720 0.2113 – 0.1700 3.2583 2.0510 – 1.2636 turbulent_radiative_layer_2D 0.3250 0.2559 0.3555 0.1930 1.2328 0.7051 0.8299 0.5491 turbulent_radiative_layer_3D 0.3261 0.3322 – 0.2073 0.9203 0.7139 – 0.6840 viscoelastic_instability † 0.1914 0.1623 0.2017 0.0624 0.4284 0.3592 0.4890 0.3465 PDEBench Diffusion-Reaction 0.0191 0.0033 0.0150 0.0015 0.7563 0.0279 0.0949 0.0241 Shallow Water 0.0019 0.0044 0.0187 0.0010 0.2499 0.1427 0.1164 0.0076 Table 1: VRMSE for unconditioned 4→1 next-step prediction and 1:20 rollout. Best in bold, second best colored green. † Using a fixed viscoelastic_instability dataset, see A.3 . In this section we evaluate Flower on a diverse suite of operator-learning benchmarks drawn from The Well ( 45 ) and additional collections, and compare against strong baselines. Table 1 is the main quantitative comparsion; Figures 3 – 5 give representative qualitative comparisons, including next-step predictions in challenging rheology (viscoelastic instability), autoregressive rollouts, and 3D buoyancy-driven dynamics (Rayleigh–Taylor). We organize the experiments as follows: • Section 4.1 benchmarks 15 – 20 M parameter models on unconditioned 4\to 1 next-step prediction, following the evaluation protocol of The Well. • Section 4.2 studies scaling to larger networks and compares against a recent very large foundation model for PDEs ( 16 ) . • Section 4.3 ablates key architectural choices in Flowers . Additional results are deferred to the appendix, including a conditioned 1\to 1 task with simulation parameters provided separately (Table 7 ) and a time-independent PDE benchmark (Table 4 , Figure 11 ). 4.1 4-to-1 Benchmarking We start with The Well benchmark, where the goal is to predict the state at time index n from the previous four frames u_{n-4},\ldots,u_{n-1} . A key difficulty is that each dataset contains trajectories from the same underlying PDE family but with varying simulation parameters (e.g., coefficients, boundary or forcing settings). These parameters are not provided to the model and must therefore be inferred implicitly from the short history window, making the task both a forecasting and a latent system-identification problem ( 48 ) . Table 1 summarizes next-step prediction performance across 16 datasets with diverse physics. Flower achieves the lowest error overall and is typically best by a substantial margin. This in particular indicates robustness to parameter variation within each PDE family. We next evaluate long-horizon behavior via autoregressive rollouts: we iteratively apply the one-step predictor to generate extended trajectories. The rollout results (second block of columns in Table 1 ) largely mirror the one-step ranking: Flower remains best on most datasets, with the main exception of the neutron star merger aftermath simulation, where it is slightly outperformed by FNO. Finally, these accuracy gains come with high practical throughput. As shown in Figure 7 , Flower matches or exceeds the throughput of FNO and scOT across resolutions, while substantially outperforming CNextU-Net , especially in 3D. 4.2 Scaling Flowers We next study how Flower scales on the Euler multi quadrants (periodic) dataset from The Well. Beyond the 17M-parameter model used throughout the main benchmark ( Flower -Tiny), we train two larger variants: Flower -Small (70M parameters) and Flower -Medium (156M parameters). All models are trained from scratch for 40 epochs; in wall-clock terms this corresponds to roughly a week of training on 1, 2, and 4 H200 GPUs, respectively. We compare with Poseidon-L ( 16 ) , a large foundation model based on the scOT architecture, pre-trained on a diverse collection of fluid-dynamics problems. This model was similarly trained for 40 epochs on 4 H200s. Figure 3 : Comparison of model predictions on the viscoelastic instability dataset (4→1 unconditioned setting), showing the conformation tensor entry C_{zz} . The top row shows the ground truth and the prediction of each model, bottom row shows the last of the four input frames and prediction errors. Figure 4 : Comparison of autoregressive rollout model predictions on the shear flow dataset (4→1 unconditioned setting), showing the tracer. Figure 5 : Comparison of model predictions on the Rayleigh-Taylor instability dataset (4→1 unconditioned setting), showing the density. Visualized using vape4d ( 20 ) . Model Parameters 1-Step 1:20 Rollout Flower -Tiny 17.3M 0.0160 0.1114 Flower -Small 69.3M 0.0124 0.0850 Flower -Medium 155.8M 0.0108 0.0739 Poseidon-L 628.6M 0.0194 0.1114 Table 2 : Model scaling study on an compressible Euler equations dataset, loss given as VRMSE. Table 2 reports the resulting errors. Performance improves smoothly with model size, indicating that Flower can indeed benefit from the additional capacity, without a change in training recipe or architecture class. We also note that even a tiny Flower delivers results comparable to a large attention-based model. 4.3 Ablations Model Variant 1-Step 1:20 Rollout w/o warping 0.4974 0.6911 single-head flow 0.1241 0.6719 w/o coord encoding in lift 0.0638 0.4065 single conv flow (no MLP) 0.0834 0.3998 w/o IdProj 0.0712 0.3816 w/o GroupNorm 0.0622 0.3760 Full Model 0.0624 0.3465 vs. CNUnet 0.1623 0.3592 Table 3 : Ablation study on Flower components, evaluated on the viscoelastic instability dataset (via VRMSE loss), sorted by 1:20 rollout loss. CNUnet values given for comparison. We ablate Flower on the viscoelastic instability dataset to assess which components matter in practice. Table 3 reports one-step prediction error and 1:20 autoregressive rollout error (VRMSE), with variants sorted by rollout performance. Warping is clearly non-negotiable. Removing it leaves a U-Net-like multiscale backbone without a spatial interaction mechanism, and error increases dramatically. Interestingly, a single-head warp already recovers much of the gain for one-step prediction; indeed it is competitive with (and here exceeds) CNUnet . This is consistent with the conservation-law viewpoint in Appendix B : a single locally predicted deformation can capture a substantial part of a transport-dominated update. Conversely, even without the U-net scaffold, a single-scale Flower operating at fixed resolution remains competitive with baselines (Table 8 ). For long-horizon rollouts, however, a single head is not sufficient: it yields poor 1:20 error even when one-step performance is reasonable. Multiple heads are therefore important for stability and accuracy over repeated composition, likely because they allow the model to represent several concurrently active transport modes rather than forcing a single displacement field to explain all motion. Other components have smaller but consistent effects. Coordinate encodings in the lifted representation improve both one-step and rollout performance, indicating that geometric context is useful even when displacements are predicted pointwise. Replacing the displacement MLP with a single convolution degrades results, suggesting that the headwise motion benefits from richer pointwise nonlinearities. Finally, the residual projection ( \mathrm{IdProj} ) and normalization mainly affect rollouts: removing \mathrm{IdProj} or GroupNorm typically worsens 1:20 performance, even when the one-step metric changes only modestly. Notably, omitting GroupNorm slightly improves one-step error while harming rollouts, consistent with normalization primarily stabilizing repeated composition. We emphasize that the relative importance of these components depends on the problem. For example, coordinate encodings may matter more with complex boundaries or discontinuous coefficients. Likewise, multiple heads should matter more when several transport families are active (e.g., multi-field coupling, mixed wave types, or strongly heterogeneous media). Figure 6 : Visualization of a learned self-warp for the Shear Flow dataset (cf. Fig. 4 ). The black arrows show the learned displacement field \varrho^{(1)}(x) of the first head in Flower ’s first downsampling block, overlaid on the tracer field. Without explicit supervision, the model learns displacements that align remarkably well with the underlying fluid velocity, providing evidence that the strong performance of Flower comes from discovering physically-meaningful transport directions. 5 Discussion Our experimental results show that multihead warping is a strong primitive for PDE learning on a broad range of phenomena. It yields lower error in forecasting complex dynamics than baselines of comparable size, and sometimes than much larger and much more resourced models. It also scales well and admits high throughput at high resolution, even in 3D, which is often challening. 5.1 Limitations What we understand reasonably well is why warps are a natural mechanism for waves and flows: characteristics for conservation laws and rays in geometric optics both imply pullback-like updates, and the continuum limit offers a kinetic view in which heads index transport modes. What we do not understand is when the learned warps align with physically meaningful transports, how the architecture handles non-hyperbolic physics, and how to stabilize rollouts which are currently not great. Indeed, while Flowers can outperform larger models on transport-dominated phenomena, they do not reach Poseidon ( 16 ) and Walrus ( 41 ) on more diffusive problems like Gray–Scott and active matter. They do, however, outperform strong baselines of comparable size on those problems and we currently do not have a great explanation why. Warps can implement convolutions and averaging but whether that is what is going on remains to be understood. Even more mystifying is solid performance on the time-independent Helmholtz equation (Table 4 ), where we lack a satisfying explanation. This means that our motivational vignettes should be read as intuition builders but not as explanations. Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. Map of Appendices • Appendix A contains detailed information about the experiments, models and datasets, as well as additional experimental results, including the time-independent Helmholtz equation (wave speed to solution map) where Flower surprisingly performs well. It also compares the computational efficiency of Flowers to other architectures. • Appendix B expands the discussion of the relationship between Flowers and conservation laws. • Appendix D gives a detailed description of the Flower architecture. • Appendix E derives the kinetic continuum limit of cascaded Flower blocks. • Appendix F shows that geometric optics solutions to the linear wave equation naturally yield a multihead pullback structure. • Appendix G describes related work on neural PDE solvers and neural operators. Appendix A Experiment Details A.1 Benchmark Setup In our experiments we generally follow the benchmark settings of The Well: we aim for a batch size utilizing above 90% of GPU memory (or the maximum possible otherwise), and limit training to 24 hours on a single NVIDIA H200 GPU (The Well uses 12 hours on an H100). This evaluates performance under a fixed wall-clock budget. As The Well contains datasets where simulation parameters varies across trajectory, we either need to provide these parameters directly to the model, or teach the model to infer them from a history of snapshots. We do the former in a 1-input-1-output setting, where the model receives simulation parameters as a separate tensor for use in conditioning FiLM layers. For the latter, we follow the benchmark setting of The Well, and provide 4 inputs to each model, stacked along the channel axis. For the 4→1 setting depicted in Table 1 , all models were tested on three different learning rates ( 1\text{\times}{10}^{-3} , 5\text{\times}{10}^{-4} , 1\text{\times}{10}^{-4} ) and the weights of the best validation VRMSE was selected. See Table 5 for the optimal learning rate per model and problem. For the 1→1 setting, where we give simulation parameters directly to the models, we ran a limited suite of experiments with a single learning rate per model, with the results depicted in Table 7 . FNO CNextU-net scOT Flower Optimal LR 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} VRMSE 0.0987 0.1425 0.3068 0.0463 Table 4: Predictions on the 15Hz acoustic time-harmonic Helmholtz equation problem from WaveBench. Collection Dataset FNO CNUnet scOT Flower The Well acoustic_scattering_maze 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} active_matter 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} gray_scott_reaction_diffusion 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} MHD_64 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} – 5\text{\times}{10}^{-4} planetswe 1\text{\times}{10}^{-4} 1\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} post_neutron_star_merger 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} – 5\text{\times}{10}^{-4} rayleigh_benard 1\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} rayleigh_taylor_instability 5\text{\times}{10}^{-4} 5\text{\times}{10}^{-4} – 1\text{\times}{10}^{-3} shear_flow 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} supernova_explosion_128 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} – 1\text{\times}{10}^{-3} turbulence_gravity_cooling 5\text{\times}{10}^{-4} 5\text{\times}{10}^{-4} – 5\text{\times}{10}^{-4} turbulent_radiative_layer_2D 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} turbulent_radiative_layer_3D 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-4} – 1\text{\times}{10}^{-3} viscoelastic_instability 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} PDEBench diffusion_reaction 1\text{\times}{10}^{-4} 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} shallow_water 1\text{\times}{10}^{-3} 5\text{\times}{10}^{-4} 1\text{\times}{10}^{-3} 1\text{\times}{10}^{-3} Table 5: Best learning rate 4→1 next-step prediction. A.2 Models In this section we describe the models used for comparison. Following the experimental setup of the The Well benchmark, we scale all models to between 15 and 20 million parameters for 2D problems. We naturally extend these models to 3D (as is done in The Well), which leads to modest increases in parameter size for the CNUnet (approx. 22M parameters) and Flower (approx. 24M parameters), but causes the FNO to reach almost 300M parameters. As scOT does not support 3D-data, we omit it from 3D benchmarks. We train all models using AdamW ( 33 ) and a linear warm up of the learning rate over 5 epochs, followed by cosine decay. All models were trained for 100 epochs or until 24 hours of training time on an H200 GPU, whichever came first. This time constraint intentionally biases our comparison toward more efficient models and follows the spirit of The Well’s benchmark setup. Model-specific hyperparameters (learning rate and weight decay) were set according to their respective original papers, with details provided in the following subsection. A.2.1 FNO We employ a standard FNO architecture consisting of three main components: an initial lifting layer that projects inputs to a high-dimensional latent space, a series of FNO blocks that perform spectral convolutions, and a final projection layer that maps back to the desired number of output channels. Following the neuralops ( 21 ; 23 ) implementation, we use a local 2-layer MLP for both the lifting and projection operations. The lifting layer employs an inverted bottleneck structure, where the hidden dimension of the MLP is 2\times the latent dimension. Similarly, each FNO block follows the neuralops architecture, with one exception: we insert an InstanceNorm layer before the non-linearity, which we expand to a FiLM layer in case of conditioning. Specifically, each block performs a spectral convolution restricted to low-frequency modes in parallel with a local linear skip connection. The outputs of these two operations are summed and passed through the normalization layer, then through a GELU activation function. Following POSEIDON’s ( 16 ) FNO implementation, we use InstanceNorm rather than alternatives such as GroupNorm or LayerNorm. As input, we concatenate the initial field with a positional grid and the conditioning information. We choose a lifting dimension of 180, 16 fourier modes and use 4 blocks, resulting in a total of approximately 18,929,704 parameters. In The Well benchmark, the median optimal learning rate for the FNO was 1\text{\times}{10}^{-3} , which we adopt here. We also copy the weight decay of 1\text{\times}{10}^{-2} . A.2.2 CNextU-net We use the CNextU-Net architecture as described in the Well benchmark, with one modification: when conditioning is applied, we replace the standard LayerNorm in each ConvNeXt block with a FiLM LayerNorm layer. As input, we concatenate the initial field with the conditioning information. We adopt the hyperparameters specified in the Well benchmark, resulting in a model with approximately 18,570,807 parameters. In The Well benchmark, the median optimal learning rate for the CNextU-Net was 1\text{\times}{10}^{-3} , which we adopt here. We also copy the weight decay of 1\text{\times}{10}^{-2} . A.2.3 scOT We include scOT ( 16 ) for comparison with an attention-based method. While the original scOT implementation expects square 2D domains, we extend the model to handle rectangular 2D domains. Since scOT already employs time-conditioned LayerNorm layers for lead-time prediction, we repurpose these for general conditioning. We configure the model with a patch size of 4, embedding dimension of 32, depths of [8, 8, 8, 8], attention heads of [2, 4, 8, 16], one residual block per skip connection stage, window size of 7, and MLP ratio of 4.0, yielding a model with approximately 17,779,868 parameters. We use a learning rate of 5\text{\times}{10}^{-4} and weight decay of 1\text{\times}{10}^{-6} as reported in the original paper ( 16 ) . A.2.4 Flower For a detailed description of the architecture we refer to Section 2 and in particular Appendix D . Flower , as used in the benchmark setting, has an initial lift to 160 channels, and 4 levels encoding/decoding. We choose 40 heads and set the number of GroupNorm groups to 40. This model has approximately 17,329,362 parameters. For our scaling analysis, we constructed versions of this model with increased channel and head numbers, scaled in proportion. Flower -S has an initial lift to 320 channels with 80 heads, giving approximately 69,274,725 parameters and Flower -M lifts to 480 channels with 120 heads, giving approximately 155,828,885 parameters. A.3 Datasets We adapt the codebase from The Well ( 45 ) and use most datasets ( 38 ; 37 ; 6 ; 40 ; 42 ; 43 ; 44 ; 9 ; 35 ; 7 ; 18 ; 17 ; 12 ) as they are. The exception is the viscoelastic instability ( 3 ) dataset, which erroneously contained duplicate frames that we removed. This improves the performance of every model. Our results on other datasets may differ from The Well’s published scores due to a bug in the original benchmark code. Once corrected, we expect the results to align. We also tested our model on two datasets from PDEBench ( 50 ) and a time-independent Helmholtz problem from WaveBench ( 30 ) . We included the latter as an expected failure case: we did not anticipate Flower to perform well on this problem. Surprisingly, it does, and we do not yet fully understand why. A.4 More Results Figure 7 : Comparison of model throughput (samples/sec) across resolutions for 2D and 3D problems, measured during the training runs of Table 1 . Throughput is calculated as batch size divided by time per batch and includes both forward and backward passes. Error bars indicate standard deviation across datasets. Figure 8 : Comparison of autoregressive model predictions on the viscoelastic instability dataset (4→1 unconditioned setting), showing the conformation tensor entry C_{zz} . The top row shows the ground truth and the prediction of each model, bottom row shows the last of the four input frames and prediction errors. Figure 9 : Comparison of model predictions on the Rayleigh-Bénard dataset (4→1 unconditioned setting), showing the buoyancy. We use blow-outs to enhance regions of interest. Figure 10 : Comparison of autoregressive model predictions on the active matter dataset (4→1 unconditioned setting), showing the concentration. The top row shows the ground truth and the prediction of each model, bottom row shows the last of the four input frames and prediction errors. FNO CNUnet scOT Flower VRMSE 0.0987 0.1425 0.3068 0.0463 Table 6: Results on WaveBench helmholtz_anisotropic . Best learning rates: FNO 1\text{\times}{10}^{-3} , CNextU-net 1\text{\times}{10}^{-3} , scOT 5\text{\times}{10}^{-4} , Flower 1\text{\times}{10}^{-3} . Figure 11 : Comparison of model predictions on WaveBench’s helmholtz_anisotropic 15Hz dataset. The top row shows the ground truth and the prediction of each model, bottom row shows the input (background wavespeed) and prediction errors. Collection Dataset FNO CNextU-net scOT Flower The Well acoustic_scattering (maze) 0.1650 0.0126 0.0378 0.0077 active_matter 0.2200 0.0918 0.1934 0.0397 MHD_64 0.3543 0.1895 – 0.1378 planetswe 0.0255 0.0041 0.0050 0.0018 rayleigh_benard 0.2589 0.1395 0.2295 0.0706 shear_flow 0.0791 0.0650 0.1383 0.0285 supernova_explosion_128 0.5911 0.5038 – 0.3440 turbulent_radiative_layer_2D 0.3326 0.2652 0.3823 0.1907 Table 7: VRMSE loss for conditioned 1→1 next-step prediction. Best results marked in bold. We did not test multiple learning rates in this setting, instead all models trained with a learning rate of 1\text{\times}{10}^{-3} , except scOT where 5\text{\times}{10}^{-4} was used, as reported in the original paper ( 16 ) . FNO CNUnet scOT Flower Flower (single-scale) Next-step VRMSE 0.2104 0.2171 0.1863 0.0807 0.1565 1:20 Rollout VRMSE 39.038 12.507 5.6486 2.1661 2.8323 Table 8: Results for a single-scale Flower variant on rayleigh_benard . The model embeds inputs into 16\times 16 patches and processes them at fixed resolution until de-embedding, rather than using the U-net multiscale structure. Since channel counts remain constant, IdProj is unnecessary; we reallocate this convolution to a projection after the warp and rearrange operations for best performance, giving: u\mapsto u+\phi\circ\mathrm{Proj}\circ\mathrm{Warp}_{V,g}\circ\mathrm{Norm}[u] . Other model’s results are taken from Table 1 and added for comparison. Appendix B Motivation via Conservation Laws Flowers achieve excellent performance on forecasting solutions to nonlinear systems of conservation laws. In this appendix we motivate the main primitive—a local pullback, or warping—using a simple scalar conservation law. Our goal is to build intuition, and the theory for systems is far more complicated. In this appendix, we show that short time solutions of the scalar conservation laws can approximated using a single layer warping network. On long time intervals the shocks can appear in the solutions of the conservation laws and hence the classical solutions may not exist. However, one can define generalized viscosity solutions that exist beyond the appearance of shocks. We consider x\in\mathbb{R}^{d} . The relevant equation is \begin{cases}\partial_{t}u(t,x)+\nabla_{x}\cdot(G(u(t,x)))=0,\quad u(0,x)=u^{0}(x),\\[6.0pt] u:\mathbb{R}_{+}\times\mathbb{R}^{d}\to\mathbb{R},\qquad{G:\mathbb{R}\to\mathbb{R}^{d}}.\end{cases} (4) We write this as a nonlinear optical flow \begin{cases}\partial_{t}u(t,x)+A(u(t,x))\cdot\nabla_{x}u(t,x)=0,\quad u(0,x)=u^{0}(x),\\[6.0pt] u:\mathbb{R}_{+}\times\mathbb{R}^{d}\to\mathbb{R}^{d},\qquad A:\mathbb{R}^{d}\to\mathbb{R}^{d},\end{cases} (5) where A(u)={\frac{d}{du}G(u)}. A generalization would be to let A explicitly depend on t and x but we do not consider this here. As a point of departure, characteristics are given by \frac{\rm d}{{\rm d}t}\Phi_{t}(x)=A(u(t,\Phi_{t}(x))),\qquad\Phi_{0}(x)=x. (6) Solutions of ( 5 ) are constant along characteristic curves t\to(t,\Phi_{t}(x)) . Indeed, substituting u(t,\Phi_{t}(x)) into equation ( 5 ), \begin{split}\partial_{t}u(t,\Phi_{t}(x))&=[\partial_{t}u(t,z)]_{z=\Phi_{t}(x)}+[\nabla_{z}u(t,z)]_{z=\Phi_{t}(x)}\cdot\partial_{t}\Phi_{t}(x)\\ &=[\partial_{t}u(t,z)]_{z=\Phi_{t}(x)}+[\nabla_{z}u(t,z)]_{z=\Phi_{t}(x)}\cdot[A(u(t,z))]_{z=\Phi_{t}(x)}\\ &=[\partial_{t}u(t,z)+A(u(t,z))\cdot\nabla_{z}u(t,z)]_{z=\Phi_{t}(x)}\\ &=0\end{split} (7) and thus we see that \frac{\rm d}{{\rm d}t}u(t,\Phi_{t}(x))=0 so that u(t,\Phi_{t}(x))=u^{0}(x). (8) Substituting ( 8 ) into ( 6 ), we find that the characteristics depend on u(t,x) only through the initial condition u_{0}(x) : \frac{\rm d}{{\rm d}t}\Phi_{t}(x)=A(u^{0}(x)),\qquad\Phi_{0}(x)=x. (9) As \Phi_{t}(x) depends only on u^{0}(x) (that is, not on values of u^{0}(x^{\prime}) at points x^{\prime}\not=x ), we write it as \Phi_{t}(x)=\Phi_{t}^{u^{0}(x)}(x), (10) and moreover, \Phi_{t}(x)=\Phi_{t}^{u^{0}(x)}(x)=x+t\,A(u^{0}(x)). (11) This shows that ( 8 ) is equivalent to u(t,x+t\,A(u^{0}(x)))=u^{0}(x). (12) Consider the case where t>0 is so small that t\cdot\hbox{Lip}(A(u_{0}(\cdot))<1 . Then, by Banach fixed point theorem, the function F_{t}:x\to x+tA(u^{0}(x)) is invertible. Let G_{t}=F_{t}^{-1}:\mathbb{R}^{d}\to\mathbb{R}^{d} be the inverse function of F_{t}:\mathbb{R}^{d}\to\mathbb{R}^{d} . Up to this point our discussion has been precise. In what follows we will discuss, somewhat non-rigorously, why Flowers can be used to approximate the function G_{t} . Let B=A\circ u^{0}:\mathbb{R}^{d}\to\mathbb{R}^{d} . Below, we assume that B is C^{3}(\mathbb{R}^{d}) . By using ( 11 ), we see that G_{t} satisfies the equation G_{t}(x)=x-tB(G_{t}(x)). Then, u(t,x)=u^{0}(G_{t}(x)). (13) Let us consider the Taylor expansion of G_{t}(x) in variable t around the time zero, G_{t}(x)=x+tg_{1}(x)+t^{2}g_{2}(x)+O(t^{3}). By operating on both sides of this equation with B and using Taylor-expansion around x , B(y)=B(x)+B^{\prime}(x)(y-x)+O(|y-x|^{2}),\quad B^{\prime}(x)=DB(x)\in\mathbb{R}^{d\times d}, we obtain B(G_{t}(x))=B(x+tg_{1}(x)+t^{2}g_{2}(x))=B(x)+tB^{\prime}(x)g_{1}(x)+O(t^{2}). Hence, x+tg_{1}(x)+t^{2}g_{2}(x)=x-t(B(x)+tB^{\prime}(x)g_{1}(x)+O(t^{2}))=x-tB(x)-t^{2}B^{\prime}(x)g_{1}(x)+O(t^{3}). Matching coefficients we get g_{1}(x)=-B(x)=-A(u^{0}(x)),\qquad g_{2}(x)=B^{\prime}(x)B(x). Combining the above formulas, we obtain \displaystyle G_{t}(x) \displaystyle=x-tB(x)+t^{2}B^{\prime}(x)B(x)+O(t^{3}) \displaystyle=x-tA(u^{0}(x))+t^{2}D[A\circ u^{0}](x)\,A(u^{0}(x))+O(t^{3}), where by the chain rule, the Jacobian matrix DB(x)=D[A\circ u^{0}](x) is given by D[A\circ u^{0}](x)=A^{\prime}(u^{0}(x))(\nabla u^{0}(x))^{\top}, where A^{\prime} is the derivative of the map of the map A:\mathbb{R}\to\mathbb{R}^{d} . Thus the 1st order approximation of G_{t}(x) in t is \displaystyle G_{t}(x) \displaystyle\approx x-tA(u^{0}(x)) which defines a pointwise-dependent displacement map considered below in Section D.1 . The 2nd order approximation of G_{t}(x) in t is given by \displaystyle G_{t}(x) \displaystyle\approx x-tA(u^{0}(x))+t^{2}A^{\prime}(u^{0}(x))(\nabla u^{0}(x))^{\top}A(u^{0}(x)) Here, the derivative \nabla u^{0}(x) can approximated using a multi-head warp network (see Section D.1 below) as \nabla u^{0}(x)^{\top}=(\frac{\partial}{\partial x^{1}}u^{0}(x),\dots,\frac{\partial}{\partial x^{d}}u^{0}(x)) can be approximated by \frac{\partial}{\partial x^{j}}u^{0}(x)=\frac{1}{s}\left(\begin{matrix}(Id+\varrho_{j,s})^{*}\\ -Id\end{matrix}\right)^{\top}\left(\begin{matrix}u^{0}(x)\\ u^{0}(x)\end{matrix}\right)+O(s), where \varrho_{j,s}(x)=se_{j} , s>0 is small, and e_{j}=(0,0,\dots,0,1,0,\dots,0) are the unit coordinate vectors. Appendix C Approximation of convolution integral operators by pullback networks We show that convolutions can be recovered as a special (non-adaptive) case of our modules. This capability may be useful for non-hyperbolic physics, for example to implement the divergence-free projections or diffusion. We start from a simplified model where the shift m(x;h) is independent of x , that is, m(x;h)=m(h) . For each head h\in\mathbb{R}^{k} and input function f , we denote \displaystyle f_{h}(x)=f(x+m(h))=((Id+{m(h)})^{*}f)(x) Let us add weights k(x,h) (or value matrices) that depend on variable x and integrate (or sum) over the head variable. This gives us operators, \displaystyle A_{k,m}f(x)=\int_{\mathbb{R}^{d}}k(x,h)f(x+m(h))dh. Assume now that m(h)=-h . By introducing a new variable y=x-h we obtain \displaystyle A_{k,m}f(x) \displaystyle= \displaystyle\int_{\mathbb{R}^{d}}k(x,h)f(x-h)dh \displaystyle= \displaystyle\int_{\mathbb{R}^{d}}k(x,x-y)f(y)dy. This shows that the operators A_{k,m} can be an arbitrary linear integral operator with kernel k(x,x-y) . Thus, one can approximate any convolution integral operator by pullback networks. If the weights k(x,h) are independent of x , that is, k(x,h)=k(h) the operators A_{k,m} can be arbitrary convolution operators. Appendix D A detailed description of the architecture For clarity we describe the pullback layer over continuous coordinates; in practice the pullback is implemented by differentiable interpolation on a grid with a boundary extension determined by the chosen boundary conditions. D.1 Feature spaces and notation conventions Let \Omega\subset\mathbb{R}^{d} be the spatial domain, \mathcal{F}(\Omega) a sufficiently rich space of real-valued functions on \Omega , and \mathcal{F}^{C}:=\left\{u:\Omega\to\mathbb{R}^{C}\right\} the space of fields with C channels. We write u,v,w for fields, x\in\Omega for spatial coordinates, and a=u(x)\in\mathbb{R}^{C} for pointwise channel vectors. Operators acting on fields take arguments in brackets (e.g. A[u] ); finite-dimensional maps take arguments in parentheses (e.g. g(a),\tau(x) ). Broadcast Given a local map h:\mathbb{R}^{p}\to\mathbb{R}^{q} , its broadcast (pointwise) extension is h[\,\cdot\,]:\mathcal{F}^{p}(\Omega)\to\mathcal{F}^{q}(\Omega),\qquad(h[u])(x)\;:=\;h(u(x)). Concatenation For u\in\mathcal{F}^{C_{1}}(\Omega) and v\in\mathcal{F}^{C_{2}}(\Omega) , we define channel concatenation as u\oplus v\in\mathcal{F}^{C_{1}+C_{2}}(\Omega),\qquad(u\oplus v)(x)=u(x)\oplus v(x)=[u_{1}(x),\ldots u_{C_{1}}(x),v_{1}(x),\ldots,v_{C_{2}}(x)]. Pullback Given a map \tau:\Omega\to\mathbb{R}^{d} , define the associated pullback operator \tau^{*}:\mathcal{F}^{C}(\Omega)\to\mathcal{F}^{C}(\Omega),\qquad(\tau^{*}v)(x):=v(\tau(x)). For a displacement field \varrho\in\mathcal{F}^{d}(\Omega) we write \tau=\mathrm{Id}+\varrho:\Omega\to\mathbb{R}^{d} for the map x\mapsto x+\varrho(x) . D.2 Multihead (pullback / spatial warp) Fix integers C_{\mathrm{in}},C_{\mathrm{out}}\geq 1 and a head count H\geq 1 with C_{\mathrm{out}}=HC_{h} . A multihead pullback layer is determined by • a local value projection f:\mathbb{R}^{C_{\mathrm{in}}}\to\mathbb{R}^{C_{\mathrm{out}}} of the form f(a)=Va , where V is a matrix; • a local multihead displacement map g:\mathbb{R}^{C_{\mathrm{in}}}\to(\mathbb{R}^{d})^{H} with components g^{(h)}:\mathbb{R}^{C_{\mathrm{in}}}\to\mathbb{R}^{d} . (In the code, g is a small pointwise MLP realized by 1\times 1 convolutions.) We will now define the key building block operator \mathrm{Warp}_{V,g}:\mathcal{F}^{C_{\mathrm{in}}}(\Omega)\to\mathcal{F}^{C_{\mathrm{out}}}(\Omega). For u\in\mathcal{F}^{C_{\mathrm{in}}}(\Omega) set v:=f[u]\in\mathcal{F}^{C_{\mathrm{out}}}(\Omega),\qquad[\varrho^{(1)},\dots,\varrho^{(H)}]:=g[u]\in\bigl(\mathcal{F}^{d}(\Omega)\bigr)^{H}, split v=v^{(1)}\oplus\cdots\oplus v^{(H)} with v^{(h)}\in\mathcal{F}^{C_{h}}(\Omega) , and define head warps \tau_{u}^{(h)}:=\mathrm{Id}+\varrho^{(h)} . Then \mathrm{Warp}_{V,g}[u]=\bigoplus_{h=1}^{H}(\tau_{u}^{(h)})^{*}v^{(h)}=\bigoplus_{h=1}^{H}\bigl(\mathrm{Id}+g^{(h)}[u]\bigr)^{*}(f^{(h)}[u]). For H=1 this reduces to the single-head layer u\mapsto(\mathrm{Id}+g[u])^{*}(Vu) . D.3 Residual pullback block (FlowerBlock) Fix the number of input and output channels C_{\mathrm{in}},C_{\mathrm{out}} . A FlowerBlock is an operator \mathrm{Block}:\mathcal{F}^{C_{\mathrm{in}}}(\Omega)\to\mathcal{F}^{C_{\mathrm{out}}}(\Omega), defined by \mathrm{Block}=\phi\circ\mathrm{Norm}\circ\Bigl(\mathrm{Warp}_{V,g}+\mathrm{IdProj}\Bigr). Here \mathrm{IdProj}:\mathcal{F}^{C_{\mathrm{in}}}(\Omega)\to\mathcal{F}^{C_{\mathrm{out}}}(\Omega) is a pointwise linear projection (a 1\times 1 convolution) and \phi is a pointwise nonlinearity (GELU). \mathrm{Norm} is group normalization. 2 2 2 For conditioned prediction we use FiLM modulated normalization. D.4 Multiscale (U-Net) Flowers We arrange FlowerBlocks into a U-Net with L levels. Let \Omega_{0} denote the finest-resolution domain (in practice a grid), and let \Omega_{\ell+1} be obtained from \Omega_{\ell} by downsampling by a factor of 2 in each spatial direction. Define channel widths c_{0}:=c_{\mathrm{lift}},\qquad c_{\ell}:=2^{\ell}c_{0},\quad\ell=0,\ldots,L-1. Positional augmentation and lifting Let \xi\in\mathcal{F}^{d}(\Omega_{0}) be the coordinate field (positional grid) so that \xi(x)=x . Define the augmentation operator \mathrm{Aug}:\mathcal{F}^{C_{\mathrm{in}}}(\Omega_{0})\to\mathcal{F}^{C_{\mathrm{in}}+d}(\Omega_{0}),\qquad\mathrm{Aug}[u]:=u\oplus\xi, and a pointwise lifting map \mathrm{Lift}:\mathcal{F}^{C_{\mathrm{in}}+d}(\Omega_{0})\to\mathcal{F}^{c_{0}}(\Omega_{0}). Encoder-side operators For \ell=0,\ldots,L-2 , let E_{\ell}:\mathcal{F}^{c_{\ell}}(\Omega_{\ell})\to\mathcal{F}^{c_{\ell+1}}(\Omega_{\ell+1}) denote the encoder stage E_{\ell}=\rho\circ\mathrm{DownConv}_{\ell}\circ\mathrm{Block}^{\downarrow}_{\ell}, where \mathrm{Block}^{\downarrow}_{\ell}:\mathcal{F}^{c_{\ell}}(\Omega_{\ell})\to\mathcal{F}^{c_{\ell}}(\Omega_{\ell}) is a FlowerBlock (same in/out channels), \mathrm{DownConv}_{\ell} is a stride- 2 convolution mapping c_{\ell}\mapsto c_{\ell+1} , and \rho is ReLU. Bottleneck Let \mathrm{Bot}:\mathcal{F}^{c_{L-1}}(\Omega_{L-1})\to\mathcal{F}^{c_{L-1}}(\Omega_{L-1}) be a FlowerBlock at the coarsest scale. Decoder-side operators Define decoder channel sizes \tilde{c}_{L-1}:=c_{L-1},\qquad\tilde{c}_{\ell}:=2c_{\ell}\ \ (\ell=0,\dots,L-2), so that after concatenating a skip at level \ell the decoder state has 2c_{\ell} channels. For \ell=1,\dots,L-1 , define an up stage U_{\ell}:\mathcal{F}^{\tilde{c}_{\ell}}(\Omega_{\ell})\to\mathcal{F}^{c_{\ell-1}}(\Omega_{\ell-1}) by U_{\ell}=\rho\circ\mathrm{UpConv}_{\ell}\circ\mathrm{Block}^{\uparrow}_{\ell}, where \mathrm{Block}^{\uparrow}_{\ell}:\mathcal{F}^{\tilde{c}_{\ell}}(\Omega_{\ell})\to\mathcal{F}^{c_{\ell-1}}(\Omega_{\ell}) is a FlowerBlock that reduces channels, \mathrm{UpConv}_{\ell} is a stride- 2 transposed convolution preserving channel count c_{\ell-1} while upsampling \Omega_{\ell}\to\Omega_{\ell-1} , and \rho is ReLU. Final projection Let \mathrm{Proj}:\mathcal{F}^{2c_{0}}(\Omega_{0})\to\mathcal{F}^{C_{\mathrm{out}}}(\Omega_{0}) be the pointwise projection \mathrm{Proj}=W_{2}\circ\rho\circ W_{1}, where W_{1} maps 2c_{0}\mapsto c_{0} and W_{2} maps c_{0}\mapsto C_{\mathrm{out}} (both 1\times 1 convolutions), and \rho is ReLU. Definition (Flower) The full network is an operator \Phi:\mathcal{F}^{C_{\mathrm{in}}}(\Omega_{0})\to\mathcal{F}^{C_{\mathrm{out}}}(\Omega_{0}), defined by the recursion x_{0}:=\mathrm{Lift}\bigl(\mathrm{Aug}[u]\bigr), \text{for }\ell=0,\dots,L-2:\qquad s_{\ell}:=x_{\ell},\qquad x_{\ell+1}:=E_{\ell}[x_{\ell}], d_{L-1}:=\mathrm{Bot}[x_{L-1}], \text{for }\ell=L-1,\dots,1:\qquad\hat{d}_{\ell-1}:=U_{\ell}[d_{\ell}],\qquad d_{\ell-1}:=\hat{d}_{\ell-1}\oplus s_{\ell-1}, and finally \Phi[u]=\mathrm{Proj}[d_{0}]. {\lx@inpgf@ignorespaces x_{0}} {\lx@inpgf@ignorespaces d_{0}} {\lx@inpgf@ignorespaces x_{1}} {\lx@inpgf@ignorespaces d_{1}} {\lx@inpgf@ignorespaces\ddots} {\lx@inpgf@ignorespaces\vdots} \ddots {\lx@inpgf@ignorespaces x_{L-1}} {\lx@inpgf@ignorespaces d_{L-1}} E_{0} E_{1} E_{L-2} \mathrm{Bot} U_{L-1} U_{2} U_{1} s_{0} s_{1} \oplus \oplus x_{0}:=\mathrm{Lift}(\mathrm{Aug}[u]) x_{1}:=E_{0}[x_{0}] x_{L-1}:=E_{L-2}[x_{L-2}] d_{L-1}:=\mathrm{Bot}[x_{L-1}] \hat{d}_{1}:=U_{2}[d_{2}] d_{1}:=\hat{d}_{1}\oplus s_{1} \hat{d}_{0}:=U_{1}[d_{1}] d_{0}:=\hat{d}_{0}\oplus s_{0} \Phi[u]:=\mathrm{Proj}[d_{0}] Figure 12 : A schematic illustration of the Flower U-Net architecture. Appendix E The continuum limit and kinetic theory perspective We use similar notation as in Appendix D but work with a single-scale architecture, without the U-Net. In particular, let H be the number of heads and write (moving the head index to the argument) u(x)=(u(x,1),\ldots,u(x,H)),\quad\text{with}\quad u(x,h)\in\mathbb{R}^{c}, so that u\in\mathcal{F}^{Hc}(\Omega) and u(x)\in\mathbb{R}^{Hc} . A multihead warp has two pointwise components, • a value map V:\mathbb{R}^{Hc}\to\mathbb{R}^{Hc} (a 1\times 1 linear (affine) convolution mixing channels and heads; here we assume that we are already “in the lift” and the number of channels is constant so that we can derive a meaningful limit; • a displacement map g:\mathbb{R}^{Hc}\to(\mathbb{R}^{d})^{H} with head components g^{(h)}:\mathbb{R}^{Hc}\to\mathbb{R}^{d} ; this a pointwise MLP (implemented via a convnet with 1\times 1 convolutions). For a u\in\mathcal{F}^{Hc}(\Omega) , let v:=V[u] and \varrho^{(h)}=g^{(h)}[u] . Define the warped outputs for each head as (\mathrm{Warp}_{V,g}[u])(x,h):=(\mathrm{Id}+\varrho^{(h)})^{*}v(\cdot,h)(x)=v(x+\varrho^{(h)}(x),h). Note that x\mapsto\varrho^{(h)}(x) only depends on u(x) ; non-locality enters via the pullback. We now interpret the discrete head index as a sampling of a continuous latent variable, \eta\in\Lambda compact (which will turn out to have a kinetic interpretation). A mixing or integration over heads can then be viewed as an integration against some probability measure. Through a quadrature, with weights w_{h} and samples \eta_{h} , we identify u(x,\eta_{h})\ \text{with}\ u(x,h). Letting H\to\infty , we obtain a function u\in\mathcal{F}^{c}(\Omega\times\Lambda) that can be interpreted as a kinetic field u(x,\eta) , that depends on positional variable x\in\Omega and a directional (velocity) variable \eta\in\Lambda . A Flower layer is (approximately) computing \mathcal{A}\left(\mathrm{Warp}_{V,g}[u]+P[u]\right) with P a pointwise linear map (a generalized skip connection) and \mathcal{A} a pointwise nonlinear map. 3 3 3 GroupNorm is not strictly local in space but we ignore this here as it is not a leading order effect. It anyway globally rescales the field. We interpret this through the lens of kinetic theory: the pullback x\mapsto\varrho(x,\eta) implements streaming in an \eta -dependent manner; the pointwise operations that mix channels and/or kinetic labels \eta at a fixed x implement interactions. Pointwise maps act along the kinetic variable, \eta , that is to say, on the functions \eta\mapsto u(x,\eta) for each x . Replacing layer index by time By considering a probability measure \mu on \Lambda\subset\mathbb{R}^{d} and interpreting \eta_{h} to be the quadrature nodes \eta_{h}\in\Lambda and corresponding weights w_{k}\in\mathbb{R} , h=1,\ldots,H , on the set \Lambda , we can consider sums over multiple heads as approximations of integrals over \Lambda obtained using a quadrature formula \displaystyle\int_{\Lambda}F(\eta)d\mu(\eta)\approx\sum_{h=1}^{H}w_{h}F(\eta_{h}). (14) We consider a composition of N+1 Flower blocks indexed by n=0,\ldots,N , let \Delta t:=T/N , and write t_{n}:=n\Delta t . We will interpret each layer as being “small” in the sense that we will assume that there are operators \mathcal{B}_{t}(\eta;.) and \mathcal{Q}(\eta;.) such that for each layer n , • the displacement function is given by \varrho(x,\eta)=\Delta t\cdot\mathcal{B}_{t_{n}}(\eta;u_{n}(x,.)) where \mathcal{B}_{t_{n}}(\eta;u_{n}(x,.))=\sum_{h=1}^{H}b_{t_{n}}(\eta,\eta_{h})u_{n}(x,\eta_{h}). In several physical applications, see e.g. ( 18 ) below, the function \mathcal{B}_{t_{n}}(\eta;u_{n}(x,.)) has the simple form, \mathcal{B}_{t_{n}}(\eta;u_{n}(x,.))=\eta ; • the local contribution of the layer is determined by P[u]=\Delta t\cdot\mathcal{Q}_{t_{n}}(\eta;u_{n}(x,.)), where \mathcal{Q}_{t_{n}}(\eta;u_{n}(x,.))=\sum_{h=1}^{H}q_{t_{n}}(\eta,\eta_{h})u_{n}(x,\eta_{h}). We then expand the pullback of u_{n} as u_{n}(x+\Delta t\cdot\mathcal{B}_{t_{n}}(\eta;u_{n}(x,{.}),\eta)=u_{n}(x,\eta)+\Delta t\mathcal{B}_{t_{n}}(\eta;u_{n}(x,.))\cdot\nabla_{x}u_{n}(x,\eta)+O(\Delta t^{2}). We get u_{n+1}(x,\eta)=u_{n}(x,\eta)+\Delta t\mathcal{B}_{t_{n}}(\eta;u_{n}(x,.))\cdot\nabla_{x}u_{n}(x,\eta)+\Delta t\cdot\mathcal{Q}_{t_{n}}(\eta;u_{n}(x,.)). (15) If \mathcal{A}=I+\Delta t\cdot\mathcal{A}^{\prime} we can account for the composition with \mathcal{A} by redefining \mathcal{B}_{t_{n}}:=\mathcal{A}_{t_{n}}\mathcal{B}_{t_{n}},\quad\mathcal{Q}_{t_{n}}:=\mathcal{A}_{t_{n}}\mathcal{Q}_{t_{n}}+\mathcal{A}_{t_{n}}^{\prime}. (16) With t_{n}=n\Delta t , through a finite difference approximation, assuming that as N\to\infty and \Delta t\to 0 (and \mathcal{A}\to I ), u_{n}(t,x) converges to u(t,x,\eta) , we get the following PDE \partial_{t}u(t,x,\eta)+\mathcal{B}_{t}(\eta;u(t,x,.))\cdot\nabla_{x}u(t,x,\eta)=\mathcal{Q}_{t}(\eta;u(t,x,.)), (17) which is a nonlinear kinetic equation with a self-adaptive streaming velocity. A notable special case of this equation is \partial_{t}u(t,x,\eta)+\eta\cdot\nabla_{x}u(t,x,\eta)=\mathcal{Q}_{t}(\eta;u(t,x,.)), (18) which is the familiar Boltzmann equation upon interpreting u as a distribution of particles, and which is known to yield hydrodynamics by evaluating moments.The nonlinear equation ( 18 ) can be considered as a Boltzmann-like equation where the drift (the streaming velocity) depends on the local kinetic state (e.g. a distribution over microscopic velocities in the true Boltzmann case). This gives it a great deal of flexibility. Appendix F Linear waves We consider the linear wave equation in \mathbb{R}^{d} , with a variable wave speed, c=c(x) , \partial^{2}_{t}u-c(x)^{2}\Delta u=0,\quad(t,x)\in[0,\infty)\times\mathbb{R}^{d} (19) with initial data u(0,x)=u_{0}(x) and \partial_{t}u(0,x)=u_{1}(x) . Here, we focus on the three-dimensional case, that is, d=3 on such times t that waves have not formed caustics, that is, the Riemannian geodesics associated to the travel time metric do not have conjugate points. We let G(t,x;y) be the causal Green’s function so that u(t,x)=\int_{\mathbb{R}^{d}}\partial_{t}G(t,x;y)u_{0}(y)dy+\int_{\mathbb{R}^{d}}G(t,x;y)u_{1}(y)dy. (20) The derivation is standard, but we explicitly connect it with our architecture, Flowers . We use the geometric optics formalism. It states that for (relatively) small t and away from caustics, that is, focusing of wave fronts caused by lensing effects, G(t,x;y) is concentrated on the wavefront surface t=\tau(x;y) , where \tau(x;y) is the travel time or geodesic distance from the point y to the point x . This motivates the following “WKB Ansatz”, G(t,x;y)\approx A(x;y)\delta(t-\tau(x;y)), (21) where the amplitude coefficient A encodes the so-called geometric spreading. It can be shown that this ansatz ( 21 ) is correct “up to smooth terms”. What this means is that it correctly propagates the high-frequency singularities. Inserting the ansatz ( 21 ) into the wave equation ( 19 ) yields A(1-c(x)^{2}|\nabla_{x}\tau|^{2})\delta^{\prime\prime}(t-\tau)+c(x)^{2}(2\nabla_{x}A\cdot\nabla_{x}\tau+A\Delta_{x}\tau)\delta^{\prime}(t-\tau)+\text{less singular terms}=0. To make G solve the wave equation ( 19 ) “up to less singular terms” away from x=y , we set the coefficients in front of the derivatives of the Dirac’s delta distributions \delta^{\prime\prime} and \delta^{\prime} to zero (that is., we analyze the leading order wavefront singularities). The \delta^{\prime\prime} coefficient yields the well-known eikonal equation for the travel times, c(x)^{2}|\nabla_{x}\tau(x;y)|^{2}=1,\quad\tau(y;y)=0. The \delta^{\prime} coefficient yields the transport equation which determines the geometric spreading A(x;y) , 2\nabla_{x}A(x;y)\cdot\nabla_{x}\tau(x;y)+A(x;y)\Delta_{x}\tau(x;y)=0. For simplicity, we focus on the u_{1} term (and assume that u_{0}\equiv 0 for now). Using ( 21 ), \int_{\mathbb{R}^{d}}G(t,x;y)u_{1}(y)dy\approx\int_{\mathbb{R}^{d}}A(x;y)\delta(t-\tau(x;y))u_{1}(y)dy. The delta function turns a volume integral into a surface integral over the surface \{y:t(x;y)=t\} : \int_{\mathbb{R}^{d}}G(t,x;y)u_{1}(y)dy\approx\int_{t=\tau(x;y)}\frac{A(x;y)}{|\nabla_{y}\tau(x;y)|}u_{1}(y)dS(y). The factor W_{1}(x;y):=\frac{A(x;y)}{|\nabla_{y}\tau(x;y)|} is a weight. The u_{0} term would have the same structure, with a different weight, W_{0}(x;y) say. We can now make the pullback structure of the solution explicit. Before caustics form, the wavefront \{y:\tau(x;y)=t\} can be parameterized by a “ray label” \eta , for example, the initial (tangent) direction on \mathbb{S}^{d-1} . Let x(t,y,\eta) denote position of the ray at time t that emanated at the time 0 from the point y\in\mathbb{R}^{d} to the direction \eta\in\mathbb{S}^{d-1} . The rays, that are equivalent to geodesic curves determined by the Riemannian metric c(x)^{-2}(dx_{1}^{2}+\ldots+dx_{d}^{2}) , are given by the ordinary differential equation, \displaystyle\partial_{t}^{2}x(t,y,\eta) \displaystyle=2((\nabla\log c)(x(t,y,\eta))\cdot\partial_{t}x(t,y,\eta))\,\partial_{t}x(t,y,\eta)-c(x(t,y,\eta))^{2}\,(\nabla\log c)(x(t,y,\eta)), (22) \displaystyle x(t,y,\eta)|_{t=0}=y,\ \partial_{t}x(t,y,\eta)|_{t=0}=\eta. (23) In the absence of caustics, x , \eta and t determine uniquely y such that x(t,y,\eta)=x . We introduce the map, \Gamma_{-t}(x,\eta)=y . Then, under the change of variable \eta\to\Gamma_{-t}(x,\eta) , the surface measure on the surface \{y\in\mathbb{R}^{d}:\ \tau(x;y)=t\} takes the form \frac{dS(y)}{|\nabla_{y}\tau(x;y)|}=J(t,x,\eta)d\eta, where J is the relevant Jacobian. Putting everything together, we now see that we can write ( 20 ) as an integral of pullbacks u(t,x)\approx\int_{\mathbb{S}^{d-1}}W_{0}(t,x,\eta)u_{0}(\Gamma_{-t}(x,\eta))\,J(t,x,\eta)d\eta+\int_{\mathbb{S}^{d-1}}W_{1}(t,x,\eta)u_{1}(\Gamma_{-t}(x,\eta))\,J(t,x,\eta)d\eta, signifying pullbacks of u_{0} and u_{1} for given t and \eta , or, upon introducing the “displacement”, \varrho(t,x,\eta)=\Gamma_{-t}(x,\eta)-x , u(t,x)\approx\int_{\mathbb{S}^{d-1}}W_{0}(t,x,\eta)u_{0}(x+\varrho(t,x,\eta))J(t,x,\eta)d\eta+\int_{\mathbb{S}^{d-1}}W_{1}(t,x,\eta)u_{1}(x+\varrho(t,x,\eta))J(t,x,\eta)d\eta. The displacement appears in the self-warp component of our architecture while \eta labels the heads. As the equation is linear, \Gamma_{-t} does not depend on u_{0,1} . Choosing (for a large H ) the quadrature nodes \eta_{h}\in\mathbb{S}^{d-1} and corresponding weights w_{k}\in\mathbb{R} , h=1,\ldots,K , we can approximate integrals over \mathbb{S}^{d-1} by a quadrature formula \displaystyle\int_{\mathbb{S}^{d-1}}F(\eta)d\eta\approx\sum_{h=1}^{H}w_{h}F(\eta_{h}). (24) By using the quadrature ( 24 ) on \mathbb{S}^{d-1} to approximate the above integrals, we obtain \displaystyle u(t,x) \displaystyle\approx \displaystyle\int_{\mathbb{S}^{d-1}}W_{0}(t,x,\eta)u_{0}(x+\varrho(t,x,\eta))J(t,x,\eta)d\eta+\int_{\mathbb{S}^{d-1}}W_{1}(t,x,\eta)u_{1}(x+\varrho(t,x,\eta))J(t,x,\eta)d\eta \displaystyle\approx \displaystyle\sum_{h=1}^{H}W_{0}(t,x,\eta_{h})u_{0}(x+\varrho_{t,h}(x))J(t,x,\eta_{h})w_{h}+\sum_{h=1}^{H}W_{1}(t,x,\eta_{h})u_{1}(x+\varrho_{t,h}(x))J(t,x,\eta_{h})w_{h}, where \varrho_{t,h}(x)=\Gamma_{-t}(x,\eta_{h})-x are the displacement fields corresponding to the head \eta_{h}. Appendix G Related work on neural PDE learning Early learning-based PDE solvers largely adopted CNN image-to-image backbones ( 55 ; 5 ; 46 ) , with U-Nets ( 47 ) remaining a strong baseline due to their locality and multiscale structure. A parallel line of work frames PDE prediction as learning solution operators between (discretizations of) function spaces ( 24 ; 34 ; 27 ) . Within this operator-learning view, Fourier Neural Operators (FNOs) ( 28 ; 29 ) have been particularly influential: they use spectral mixing to realize global interactions and have become a standard reference point across benchmarks. Following the success of Vision Transformers ( 11 ) , attention-based architectures have also been adapted to operator learning ( 8 ; 26 ) . Their set-based formulations facilitate irregular meshes and complex geometries ( 15 ; 52 ; 36 ; 51 ) , but attention typically trades away strong locality priors and incurs quadratic scaling in the number of tokens, which is restrictive at high spatial resolution and in 3D. Hybrid models that mix multiple formalisms also exist ( 13 ) , aiming to combine global coupling with practical efficiency. More recently, several works pursue foundation-model training for scientific dynamics: pretraining on diverse PDE families followed by task-specific finetuning ( 49 ; 16 ; 39 ; 14 ) . These efforts are enabled by standardized benchmark suites and data collections, including PDEBench ( 50 ) , PDEGym ( 16 ) , and The Well ( 45 ) . In contrast to convolution-, Fourier-, or attention-based mixing, Flowers build global interactions from sparse, state-dependent warps (pullbacks) coupled with pointwise channel mixing, providing an explicit transport primitive that complements these prior approaches.