PDE-SSM: A Spectral State Space Approach to Spatial Mixing in Diffusion Transformers

Eshed Gal, Moshe Eliasof, Siddharth Rout, Eldad Haber · 2026

Read PDE-SSM: A Spectral State Space Approach to Spatial Mixing in Diffusion Transformers by Eshed Gal, Moshe Eliasof, Siddharth Rout, Eldad Haber (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-13663.txt.

Introduction

Eshed Gal, Moshe Eliasof, and Eldad Haber propose PDE-SSM, a spectral state space approach to spatial mixing in diffusion transformers. Self-attention has quadratic cost and weak spatial inductive bias, which limits vision transformers. PDE-SSM replaces attention with a learnable convection-diffusion-reaction partial differential equation block. Solving the PDE in the Fourier domain gives global coupling at near-linear complexity. Integrated into a flow-matching model, PDE-SSM-DiT matches or exceeds state-of-the-art Diffusion Transformers at far lower compute.

Catalog record

CREATOREshed Gal, Moshe Eliasof, Siddharth Rout, Eldad Haber
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE41860
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.13663

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PDE-SSM: A Spectral State Space Approach to Spatial Mixing in Diffusion Transformers Eshed Gal, Moshe Eliasof, Siddharth Rout, Eldad Haber Abstract The success of vision transformers—especially for generative modeling—is limited by the quadratic cost and weak spatial inductive bias of self-attention. We propose PDE-SSM, a spatial state-space block that replaces attention with a learnable convection–diffusion–reaction partial differential equation. This operator encodes a strong spatial prior by modeling information flow via physically grounded dynamics rather than all-to-all token interactions. Solving the PDE in the Fourier domain yields global coupling with near-linear complexity of O ⁡ ( N ​ log ⁡ N ) O(N\log N) , delivering a principled and scalable alternative to attention. We integrate PDE-SSM into a flow-matching generative model to obtain the PDE-based Diffusion Transformer PDE-SSM-DiT. Empirically, PDE-SSM-DiT matches or exceeds the performance of state-of-the-art Diffusion Transformers while substantially reducing compute. Our results show that, analogous to 1D settings where SSMs supplant attention, multi-dimensional PDE operators provide an efficient, inductive-bias-rich foundation for next-generation vision models. 1 Introduction The Transformer architecture ( 42 ) , with its self-attention mechanism, has become the de facto standard for state-of-the-art machine learning techniques, extending its dominance from Natural Language Processing ( 41 ) to Computer Vision ( 5 ; 27 ) . In generative modeling for images, the U-Net ( 30 ) backbone with attention mechanism has become the standard for diffusion models over the years ( 12 ; 34 ; 25 ; 4 ; 29 ; 20 ) . However, after a long streak of success, the Diffusion Transformer (DiT) ( 27 ) marked a significant milestone by replacing the U-Net backbone with a Transformer-based architecture, demonstrating that pure Transformer models can achieve or surpass the performance of convolutional U-Net backbones in large-scale diffusion models. Like other transformers, DiTs utilize deep networks. However, it is worth noting that DiTs are also constructed using attention mechanisms, which makes them fundamentally scalable. Be it modern U-Nets or DiTs, these are attention-based models which essentially operate by treating an image as a collection of patches and apply full weight within the patch and self-attention to model the relationship between patches ( 5 ; 21 ; 27 ) . Despite its empirical success, self-attention has two well-known drawbacks in the spatial domain ( 38 ) , which are the focus of this paper. The limitations of self-attention in this case are as follows: first , its computational and memory requirements become increasingly challenging in 2D inputs, which have quadratically as many tokens as standard 1D inputs. Although there are a few recent ideas to attempt and reduce this cost (e.g. linear attention) ( 15 ; 31 ) , their performance generally falls short compared to standard attention-based methods. Second , self-attention is fundamentally permutation-invariant, meaning it does not inherently encode spatial locality or grid structure. Positional encodings are commonly introduced to remedy this limitation, and indeed, they allow Transformers to capture ordering and relative spatial information. Nevertheless, this mechanism is not as direct or specialized as the inductive bias present in convolutional neural networks (CNNs), which are explicitly designed around locality. Consequently, Transformer-based models often rely more heavily on data to learn robust spatial relationships. Concurrently, a distinct lineage of sequence models has emerged to address these limitations in the 1D domain. State Space Models (SSMs), particularly structured SSMs (S4) ( 11 ) and their modern successors like Mamba ( 10 ) , have achieved remarkable success on long-sequence tasks while offering compelling efficiency. These models are grounded in the theory of continuous-time systems, governed by a linear Ordinary Differential Equation (ODE): \frac{dh(t)}{dt}=Ah(t)+Bu(t). (1) By imposing structure on the state matrix A , these models can be formulated as a recurrent or convolutional system, enabling them to capture extremely long-range dependencies in linear or near-linear time ( 11 ) . Their success raises a critical and compelling question: how can the principles of SSMs be generalized from 1D time to N-D space to create a more efficient and spatially-aware foundation for vision models? Recent work has extended SSMs to graphs ( 2 ; 19 ) , replacing A with the adjacency matrix. In the context of SSMs in 2D, initial forays into this area, such as Vision Mamba (ViM) ( 46 ) , have adapted 1D SSMs to images by ”flattening” the 2D image domain into a 1D sequence, followed by applying the SSM model along forward and backward paths. While effective, this approach imposes an artificial causality on the spatial domain, effectively processing pixels/nodes in a scan order rather than respecting the true multi-dimensional structure of images. In this sense, it represents a heuristic adaptation of 1D principles, rather than a full generalization to the spatial setting. We provide an additional discussion of concurrent and complementary approaches in Appendix A . Our approach. To bridge this gap, in this work, we propose a more principled approach. Concretely, we argue that the natural continuous-space generalization of an ODE that is the core of SSMs as shown in Equation 1 , is a Partial Differential Equation (PDE). This observation has been studied for the control of PDEs in the context of chemical engineering ( 24 ; 7 ) . We extend this idea and introduce the PDE State-Space Model (PDE-SSM) , an architectural block that models the evolution of a hidden state h(t,\mathbf{x}) in time t , where \mathbf{x}\in\mathbb{R}^{d} — according to a learnable, general-form diffusion–convection–reaction equation, as follows: \frac{\partial h}{\partial t}=\underbrace{\nabla\cdot(K\nabla h)}_{\text{Diffusion}}+\underbrace{\mathbf{b}\cdot\nabla h}_{\text{Convection}}+\underbrace{rh}_{\text{Reaction}}, (2) where the diffusion tensor K , convection vector field \mathbf{b} , and reaction term r are learnable parameters. This formulation provides a flexible and physically-grounded inductive bias; information propagates across the spatial domain via structured and well-studied mechanisms: (i) diffusion smooths and aggregates local information; (ii) convection directs the flow of information; and (iii) reaction models local feature transformations. Importantly, by implementing the solution to the PDE in Equation 2 in the Fourier domain, the underlying operator admits a computational complexity of O(N\log N) , making it highly scalable relative to the O(N^{2}) cost of self-attention, while still capturing global interactions more directly than local convolutions. We integrate the PDE-SSM block into a standard transformer architecture, replacing the self-attention layers to create the PDE-based Diffusion Transformer, to obtain PDE-SSM DiT . Our contributions are as follows: 1. We introduce PDE-SSM, a novel architectural block that provides a principled generalization of State Space Models to multi-dimensional spatial data by leveraging learnable PDEs. 2. We present PDE-SSM DiT , a new backbone for generative vision models that utilizes the PDE-SSM for efficient and spatially-aware feature mixing, efficiently implemented via the Fourier transform. 3. Through extensive experiments on both low- and high-resolution image generation tasks, we show that PDE-SSM DiT is an effective generative model, achieving competitive or superior performance to the standard DiT, while being more computationally efficient, especially at higher resolutions. Figure 1 : Visualizing the PDE-SSM Convolutional Kernels. By sampling the learnable parameters \xi=(\mathcal{B}_{\gamma},\zeta) , our PDE-SSM can represent a diverse family of convolutional kernels. The examples show kernels that are (from left to right): localized, directionally blurred (anisotropic diffusion), shifted (convection), and a combination of effects. This flexibility allows our PDE-SSM model to learn a rich basis for spatial feature mixing,

Read the complete text