Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identification
Achraf El Messaoudi, Noureddine Khaous, Karim Cherifi · 2026
Read Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identification by Achraf El Messaoudi, Noureddine Khaous, Karim Cherifi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17566.txt.
Introduction
El Messaoudi and two colleagues revisit how diffusion models should be trained for nonlinear system identification with spatial outputs. Old methods fail in turbulent flow, where dynamics are many-sided and sensitive to growing rollout errors. Many diffusion models borrow target choices from image generation, usually noise or velocity prediction. Using a patch-based transformer on fields with turbulent flow as testbed, the authors find clean-state prediction lifts rollout stability. It cuts long-horizon error versus noise- and velocity-based targets, with the edge growing with per-token size.
Catalog record
| CREATOR | Achraf El Messaoudi, Noureddine Khaous, Karim Cherifi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36344 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.17566
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Target Parameterization in Diffusion Models for Nonlinear Spatiotemporal System Identification Achraf El Messaoudi, Noureddine Khaous, Karim Cherifi Abstract Machine learning is becoming increasingly important for nonlinear system identification, including dynamical systems with spatially distributed outputs. However, classical identification and forecasting approaches become markedly less reliable in turbulent-flow regimes, where the dynamics are high-dimensional, strongly nonlinear, and highly sensitive to compounding rollout errors. Diffusion-based models have recently shown improved robustness in this setting and offer probabilistic inference capabilities, but many current implementations inherit target parameterizations from image generation, most commonly noise or velocity prediction. In this work, we revisit this design choice in the context of nonlinear spatiotemporal system identification. We consider a simple, self-contained patch-based transformer that operates directly on physical fields and use turbulent flow simulation as a representative testbed. Our results show that clean-state prediction consistently improves rollout stability and reduces long-horizon error relative to velocity- and noise-based objectives, with the advantage becoming more pronounced as the per-token dimensionality increases. These findings identify target parameterization as a key modeling choice in diffusion-based identification of nonlinear systems with spatial outputs in turbulent regimes. I INTRODUCTION System identification for dynamical systems is a central problem in control, where the objective is to construct predictive models from measured data for simulation, analysis, and control design. In recent years, machine learning has become an increasingly important tool for this task, owing to its ability to capture complex dynamical behaviors directly from data [ 1 ] . Recent transformer-based approaches have started to broaden the scope of data-driven modeling for dynamical systems. In particular, sequence models have been used to infer the behavior of previously unseen systems directly from trajectory data [ 2 , 3 , 4 ] . Beyond these settings, related ideas have also been explored for parametric dynamical systems with time-varying inputs, where transformer architectures are used as black-box predictors for multi-output temporal evolution [ 5 ] . These developments are also relevant for dynamical systems with spatial outputs, where the observations are no longer low-dimensional signals but distributed fields evolving over time. In this setting, the identification problem must account not only for temporal dynamics, but also for the spatial structure of the outputs. A natural extension of machine-learning-based state-space identification is therefore to preserve the nonlinear state-space viewpoint while replacing dense input-output mappings by convolutional parameterizations [ 6 ] . For turbulent flows, however, this identification problem becomes substantially more difficult. The dynamics are high-dimensional, strongly nonlinear, and sensitive to compounding rollout errors, so that locally accurate predictors may still drift or become unstable in free-running simulation. In this regime, autoregressive conditional diffusion model (ACDM) has recently emerged as a promising approach, combining strong long-horizon prediction with posterior sampling capabilities that are attractive when multiple future flow evolutions are consistent with the same conditioning information [ 7 ] . At the same time, several modeling choices in diffusion-based forecasting are inherited from image generation [ 8 , 9 , 10 , 11 ] . One important choice concerns the prediction objective: rather than predicting the clean next state directly, diffusion models are commonly trained to predict injected noise, or closely related parameterizations such as velocity [ 8 , 10 ] . However, recent work suggests that clean-signal prediction can be substantially easier than noise-like targets, especially in high-dimensional token spaces [ 12 ] . This observation naturally motivates revisiting the learning objective itself when identifying predictive surrogates for turbulent-flow dynamics. Motivated by these considerations, we revisit prediction target choice in diffusion-based system identification for nonlinear distributed dynamics. Our goal is not to propose a highly specialized architecture, but rather to isolate how the diffusion prediction objective affects the quality of the identified model. To this end, we use a deliberately simple, self-contained patch-based transformer operating directly on physical fields, without latent autoencoders, auxiliary tokenizers, or external backbone components, and vary only the diffusion target while keeping the architecture and training budget fixed. Concretely, we compare clean-state, velocity, and noise prediction on the incompressible and transonic benchmarks introduced in [ 7 ] , with an emphasis on long-horizon rollouts. These observations suggest that, in diffusion-based system identification of nonlinear distributed dynamics, the prediction target should be regarded as a genuine modeling choice rather than a secondary implementation detail. This motivates the controlled study carried out in this paper, where we isolate the effect of target parameterization on the quality of the identified transition model. The contributions of this paper are as follows: • We revisit the choice of diffusion prediction objective ( x -, v -, or \epsilon -prediction) in diffusion-based system identification of turbulent-flow dynamics, and provide a controlled evaluation on incompressible and transonic cylinder benchmarks under matched training budgets. • We introduce a minimal, self-contained patch-transformer diffusion system identification model for next-step field prediction that operates directly in physical space and conditions on past frames and scalar parameters, enabling a clean assessment of objective-choice effects without encoders, U-Nets, or latent variables. • We show that clean-state prediction leads to more reliable identified transition models than conventional noise-like targets, yielding improved rollout stability and lower long-horizon error, and we relate this behavior to a manifold-based interpretation of physical states versus diffusion targets. • We propose a two-resolution, matched-token-count protocol that varies per-token dimensionality while keeping sequence length fixed, and show that the advantage of clean-state prediction becomes more pronounced in the large-patch regime. II BACKGROUND AND PROBLEM SETUP We consider dynamical systems with spatial outputs, for which collected input-output trajectory data are available. The spatial outputs are spatio-temporal fields \{x_{t}\}_{t=1}^{T} with x_{t}\in\mathbb{R}^{C\times H\times W} , where C denotes the number of physical channels and (H,W) the spatial resolution. Here, T denotes the number of time steps in a trajectory. In many control-oriented settings, each trajectory is associated with two types of exogenous information: a time-varying input signal u_{1:T} driving the system evolution, and possibly a fixed low-dimensional parameter vector \theta describing the operating condition or system instance. Accordingly, from a system-identification viewpoint, the objective is to identify a predictive model of the field evolution from past observations together with the available control inputs and, when relevant, static parameters. This setting arises naturally in nonlinear systems with spatially distributed outputs, where one seeks to model the evolution of measured fields under external actuation. Representative examples are discussed in [ 6 ] , including flow around an oscillating cylinder, where the transverse cylinder displacement acts as the input, and Rayleigh–Bénard convection, where the manipulated input is the temperature difference imposed between two bottom wall sections. We model the conditional next-step distribution given a fixed-length history window together with the available exogenous information: \hat{x}_{t+1}\sim p_{\phi}(\,\cdot\mid x_{t-k+1:t},\,u_{t-k+1:t},\,\theta), (1) where k denotes the context length, x_{t-k+1:t}:=(x_{t-k+1},\ldots,x_{t}) , and u_{t-k+1:t}:=(u_{t-k+1},\ldots,u_{t}) . A free-running rollout of horizon H is then obtained by recursively replacing the true past states in ( 1 ) with previously generated ones, for t=k,\ldots,k+H-1 . Here, \phi denotes the learnable parameters of the conditional generative model p_{\phi} . From a system-identification perspective, this corresponds to evaluating the identified surrogate in free-running simulation, which is the regime relevant for long-horizon prediction and downstream control-oriented use. A central difficulty in this setting is compounding error: small inaccuracies at each step can accumulate during