On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics

Mikhail Osipov · 2026

Read On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics by Mikhail Osipov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-03238.txt.

Introduction

Mikhail Osipov tests four ways to tidy the hidden space of a learned simulator, and most of them backfire. His 2026 paper studies reduced models built on an autoencoder, which squeezes a system's state so a small model can step it forward. The test case is a spread-and-react equation, with the hidden state stepped by a learned rate law.

Three smoothing penalties, applied while the squeezer trains, often made later training harder. Long forecasts suffered most, even when local smoothness scores improved. The fourth approach, forcing the decoder's first layer into a well-shaped set, steadily helped both diagnostics and forecasts. Osipov argues that a mismatch between hidden shape and dynamics costs more than rough decoding does.

Catalog record

CREATORMikhail Osipov
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47414
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.03238

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics [email protected] Mikhail Osipov Abstract We investigate geometric regularization strategies for learned latent representations in encoder–decoder reduced-order models. In a fixed experimental setting for the advection–diffusion–reaction (ADR) equation, we model latent dynamics using a neural ODE and evaluate four regularization approaches applied during autoencoder pre-training: (a) near-isometry regularization of the decoder Jacobian, (b) a stochastic decoder gain penalty based on random directional gains, (c) a second-order directional curvature penalty, and (d) Stiefel projection of the first decoder layer. Across multiple seeds, we find that (a)–(c) often produce latent representations that make subsequent latent-dynamics training with a frozen autoencoder more difficult, especially for long-horizon rollouts, even when they improve local decoder smoothness or related sensitivity proxies. In contrast, (d) consistently improves conditioning-related diagnostics of the learned latent dynamics and tends to yield better rollout performance. We discuss the hypothesis that, in this setting, the downstream impact of latent-geometry mismatch outweighs the benefits of improved decoder smoothness. 1 Introduction Autoencoder (AE) architectures are widely used in machine learning (ML) and scientific machine learning (SciML). Dimensionality reduction via an encoder–decoder construction enables the modeling of high-dimensional dynamics in a low-dimensional latent space [ 1 , 2 , 3 , 4 , 5 ] . Neural ordinary differential equations (NODE) [ 6 , 7 ] are commonly employed to learn continuous-time dynamics in latent space. More broadly, operator-learning methods have also been successfully applied to partial differential equations (PDEs) [ 8 ] . Recent works [ 9 , 10 ] provide mathematical frameworks for reduced-order modeling (ROM) via autoencoders, discuss theoretical limitations, and demonstrate computational efficiency in representative benchmarks. A fundamental concern in autoencoder-based ROM is that the latent space \mathbb{R}^{d} typically has lower dimension than the ambient space \mathbb{R}^{n} . When d<n , the encoder E:\mathbb{R}^{n}\to\mathbb{R}^{d} cannot be globally injective and therefore introduces information loss away from the data manifold. Consequently, the decoder D:\mathbb{R}^{d}\to\mathbb{R}^{n} may exhibit locally expansive behavior along certain directions of the manifold. This can result in amplification of latent-space errors after decoding, an effect that becomes particularly pronounced in long-horizon rollouts. The amplification of latent errors by the decoder can be quantified using its Lipschitz constant [ 11 ] . Let u(t) denote the true (discretized) trajectory in the ambient space and let z(t)=E(u(t)) be its latent representation. Let \hat{z}(t) be a numerical approximation in latent space and define \hat{u}(t)=D(\hat{z}(t)) . Then \|u-\hat{u}\|_{2}=\|u-D(\hat{z})\|_{2}\leq\|u-D(E(u))\|_{2}+\|D(E(u))-D(\hat{z})\|_{2}\leq\varepsilon_{\text{recon}}+L_{D}\|z-\hat{z}\|_{2}, where \varepsilon_{\text{recon}} denotes the reconstruction error and L_{D} is a local Lipschitz constant of the decoder on the relevant region of latent space. One classical approach to controlling sensitivity in representation learning is to introduce Jacobian-based regularization [ 12 , 13 , 14 , 15 , 16 ] . Some authors also discuss curvature regularization of autoencoders [ 17 ] . Another commonly used way of regularization is related to Stiefel projection [ 18 , 19 , 20 , 21 , 22 ] . In this work, we investigate several heuristics aimed at controlling the decoder’s local sensitivity and geometric properties, as well as a structured projection onto the Stiefel manifold: (a) a local isometry penalty on the decoder Jacobian, enforcing \|J_{D}^{\top}J_{D}-\mathbf{I}\|_{\text{Frobenius}}\approx 0 ; (b) a stochastic decoder gain penalty based on directional norms \|J_{D}v\| for random unit vectors v ; (c) a second-order directional curvature penalty based on variations of Jacobian–vector products; (d) a Stiefel (orthonormal-column) projection applied to the first decoder layer. We demonstrate numerically, in the setting of an advection–diffusion–reaction (ADR) system combined with a latent neural ODE, that pretraining the autoencoder with regularizers (a)–(c) often produces latent geometries for which learning stable latent dynamics becomes more difficult and long-horizon rollouts degrade. This occurs even when intrinsic geometric diagnostics indicate improved local smoothness or reduced decoder-sensitivity proxies. In contrast, the Stiefel projection (d) consistently improves conditioning-related diagnostics of the learned latent dynamics and yields slightly better (or at least not worse) rollout performance. 2 Geometry Regularizers In this section, we describe the regularization methods used in our experiments and provide brief motivation for each of them. Full formal definitions are given in Appendix A . 2.1 (a) Near-isometry Jacobian penalty A classical approach to autoencoder regularization is based on Jacobian control [ 12 ] . As one of our regularization options, we use a decoder-Jacobian near-isometry penalty, added to the loss during autoencoder pre-training: \mathcal{R}_{a}\;=\;\mathbb{E}\!\left[\left\|J_{D}^{\top}J_{D}-\mathbf{I}\right\|_{F}^{2}\right]. In practice, this term can be evaluated either exactly (via basis Jacobian–vector products) or approximately using a Hutchinson-style stochastic estimator [ 23 ] . In our experiments, we did not observe substantial differences between the exact and stochastic evaluations of this penalty. 2.2 (b) Directional gain penalty To control decoder sensitivity without enforcing full near-isometry, we also consider a weaker directional gain penalty: \mathcal{R}_{b}\;=\;\mathbb{E}_{v}\!\left[\bigl(\|J_{D}v\|-1\bigr)^{2}\right], where v is a random unit vector in latent space. This penalty encourages approximately unit gain along sampled directions, but unlike \mathcal{R}_{a} , it does not directly constrain the full spectrum of J_{D} . The required Jacobian–vector products are estimated numerically using finite differences. 2.3 (c) Flatness penalty To probe the local flatness of the decoder-induced geometry without explicitly enforcing strong isometry or gain constraints, we introduce a curvature-related penalty: \mathcal{R}_{c}\;\approx\;\mathbb{E}\!\left[\left\|\nabla^{2}D(z)\right\|^{2}\right]\;\approx\;\mathbb{E}\!\left[\frac{\left\|J_{D}(z+\varepsilon v)\,v-J_{D}(z)\,v\right\|^{2}}{\varepsilon^{2}}\right]. This term approximates a directional second-order variation of the decoder. To compute it, we use autodiff JVPs and estimate directional curvature by finite-differencing these JVPs across a shifted basepoint. 2.4 (d) Stiefel projection Projection onto the Stiefel manifold is a standard tool in optimization on matrix manifolds [ 18 , 19 , 20 , 21 , 22 ] . In our setting, after each gradient step, the weight matrix of a selected decoder layer is projected back onto the Stiefel manifold, enforcing orthonormal columns for that layer. Unlike the penalties above, this method imposes a structural constraint only on a single layer (in our case, the first convolutional layer of the decoder). As a result, it can improve the conditioning of that layer without directly enforcing near-isometry, low curvature, or controlled gain for the full decoder Jacobian. Therefore, it should be viewed as a partial architectural regularization rather than as an explicit constraint on the geometry of the decoder as a whole. 3 Numerical experiments 3.1 Dataset generation We generate full-order trajectories for a parametric advection–diffusion–reaction (ADR) problem using a dolfinx finite-element solver on the unit square. The spatial discretization uses first-order Lagrange elements on a uniform triangular mesh with 31\times 31 subdivisions, yielding 32\times 32=1024 scalar degrees of freedom per snapshot. The governing dynamics depend on a parameter vector \mu=(\mu_{1},\mu_{2},\mu_{3})\in\mathbb{R}^{3} , where \mu_{1} controls diffusion and (\mu_{2},\mu_{3}) determine the center of a Gaussian source term. The reaction coefficient is fixed, and the advection field is time-dependent. Full-order trajectories are integrated on the interval [0,T] , with T=10\pi , using backward Euler with 1000 time steps. This produces 1001 fine-grid snapshots per trajectory. For reduced-model training, we additionally construct a coarse temporal grid by retaining every fifth full-order step, resulting in 201 snapshots per trajectory. The parameter domain is split into three subsets: a structured training / validation set inside an inner parameter box, an interpolation test set formed by midpoint parameters within the same box, and an extrapolation test set sampled outside the training box but within a larger outer box.

Read the complete text