Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection

Xinwei Cao, Mengxuan Lu, Torbjørn Svendsen, Giampiero Salvi · 2026

Read Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection by Xinwei Cao, Mengxuan Lu, Torbjørn Svendsen, Giampiero Salvi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-00684.txt.

Introduction

Cao and co-authors tackle a safety task for AI in this 2026 paper: spotting inputs that fall outside the data a model was trained on. They use continuous normalizing flows, which assign each input a density score. A known flaw is that such models can give odd inputs a high score.

Their frame, called Lagrangian sub-flow, splits the input space in two. One part holds the traits that matter, and the rest gives context. Density is then judged on the key part alone. In speech tests, the same flaw shows up: odd sounds can score as likely. The team adds signals drawn from the shape of the flow path itself. These signals beat score-based checks on a real test of spotting mispronounced sounds, they report.

Catalog record

CREATORXinwei Cao, Mengxuan Lu, Torbjørn Svendsen, Giampiero Salvi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE52 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.00684

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Local Diagnostics of Continuous Normalizing Flow for Out-of-Distribution Detection Xinwei Cao, Mengxuan Lu, Torbjørn Svendsen, Giampiero Salvi Abstract We address the problem of out-of-distribution (OOD) detection for target observations embedded in a subspace of the high dimensional data space. Using continuous normalizing flows (CNFs), we propose a Lagrangian sub-flow (LSF) framework designed to isolate and estimate the density for the relevant components in the representation and using the remaining components as context. Through experimentation with models for speech synthesis, we show that CNFs, similarly to other deep generative models (DGMs), are susceptible to the “likelihood paradox”, where high likelihood is erroneously assigned to OOD samples. This is attributed to the inductive bias of DGMs that prioritize low-level structural details over high-level semantic coherence. To mitigate this phenomenon, we propose a number of geometric diagnostic signals based on the velocity field over the sub-flow trajectory. Based on these signals, we design metrics for the challenging task of zero-shot phoneme-level mispronunciation detection. Finally, we demonstrate the superiority of these metrics compared to likelihood-based methods on a real-world mispronunciation detection benchmark. 1 Introduction Deep generative models (DGMs) have demonstrated an exceptional ability to model complex data distributions. Yet, their applications in out-of-distribution (OOD) detection underperform dedicated discriminative models due to the so called “likelihood paradox” effect. As identified by Nalisnick et al. 2019 Nalisnick et al. 2019 , likelihood-based models often assign higher density to OOD samples than to in-distribution data. For example, a model trained on images from CIFAR-10 Krizhevsky 2009 , may assign higher likelihoods to images from another domain, such as SVHN Netzer et al. 2011 . In complement to it, Ren et al. Ren et al. 2019 have found that standard likelihood scores are often dominated by background statistics rather than the “semantic" features that actually define a class. Similarly, Kirichenko et al. Kirichenko et al. 2020 provided a geometric explanation, suggesting that DGMs such as normalizing flows (NF) fail because they prioritize low-level pixel statistics over high-level semantics. Efforts have been made to mitigate the “likelihood paradox” issue. For example, researchers managed to use likelihood ratios ( Ren et al. 2019 ) or input complexity Serrà et al. 2020 to compensate the bias towards background noise. Le et al. Le & Dinh 2021 gave a geometric explanation of why NF models fail to detect simple OOD samples and proposed to estimate the local intrinsic dimensions (LID) and augment the likelihood with additional criteria. The above mentioned methods treat DGMs as static probability estimators whilst ignoring the rich dynamical information embedded in the training and inference of the model itself. Xiao et al. Xiao et al. 2020 proposed to use the likelihood-ratio before and after a training step of DGMs to distinguish between In-domain (ID) and OOD samples. More recently, Diffusion models, which have dominated the field of content generation on all modalities, have been applied for OOD detection, e.g, MSMA Jolicoeur-Martineau et al. 2021 or DiffGuard Gao et al. 2023 . The former approach is based on the trained score-function evaluated at each point on the noise adding process (the diffusion path). The latter runs the data to noise path first forwards and then procedes backwards with an invertible diffusion model. OOD detection is performed by analyzing the property of the reconstructed point. Latent ODE models Rubanova et al. 2019 can be used to model the evolution of time series in the latent space, e.g., the temperature over a period of time. Morningstar et al. Morningstar et al. 2021 defined a dynamics prior with a latent ODE model to distinguish between normal and abnormal time-series. Another powerful category of DGMs, namely the continuous normalizing flow (CNF), is however left understudied for OOD detection. One of the major concerns of using CNF for OOD detection is the computational complexity since CNF usually requires multiple steps or number of function evaluations (NFE) to evaluate the likelihood. However, we believe that the theoretical analogy between CNF and fluid dynamics offers a powerful perspective for extracting diagnostic signals. Moving beyond the holistic perspective of the flow, we argue that the true diagnostic power of CNF lies in the localized, fine-grained trajectories within the high-dimensional manifold, or the sub-flow dynamics. Due to averaging over high-dimensional representations, in real-world applications, OOD signals rarely correspond to very low likelihoods. By decomposing the global flow into context-specific sub-flows, we can capture the structural incoherence that is otherwise masked by global averaging. There are many application domains of sub-flow diagnostics. In speech generation, for example, a sub-flow can represent the trajectory of a specific phoneme while the global flow represents global information such as environmental noise and speaker’s identity; In computer vision and text-to-image generation, sub-flows correspond to spatial-semantic regions and analyzing these localized flows may allow for the detection of “hallucinations" where a specific object’s internal geometry contradicts the global scene; Furthermore, detecting localized OOD patches in medical imaging may be useful to detect a disease, where the rest of the image acts as calibration for the specific patient. Finally, in bio-informatics and time-series analysis, anomalies in these sub-flows can signal mutations or transient regime shifts under a global trend. In this work, we investigate the possibilities of using CNF for OOD detection. More specifically, we focus on the task of variable scale sub-flow OOD detection within the holistic flow. We make the following contributions: • We identify the global coupling issue in modern Transformer-based CNFs and propose the Lagrangian sub-flow (LSF) framework to inspect local properties of the sub-flow while preserving global context awareness from the original flow. • We provide several stochastic tools that efficiently separate local diagnostic signals from the global evolution, for analyzing the sub-flow dynamics. • We demonstrate the usage of these tools in the domain of phoneme-level pronunciation assessment using a well-known CNF-based speech generative model. • We visualize and discuss the geometric properties of the learned vector field. 2 Background and Related Work In this section, we briefly introduce the continuous normalizing flow (CNF) which is essential for understanding the methods. Consider observations x in a D -dimensional data space \mathbb{R}^{D} distributed according to an unknown data distribution q_{\text{data}}(x) . CNF-based models learn a continuous transformation with the probability path p_{t}(x) over flow-time t\in[0,1] from a Gaussian noise source p_{0}(x)=\mathcal{N}(x;0,I_{D}) to the approximated data density p_{1}(x)\approx q_{\text{data}}(x) . This transformation corresponds to a flow \phi(x,t):\mathbb{R}^{D}\times\mathbb{R}\rightarrow\mathbb{R}^{D} which is generated by a time-dependent velocity field v(x,t):\mathbb{R}^{D}\times\mathbb{R}\rightarrow\mathbb{R}^{D} . This process can be described by a time-dependent random variable X_{t} . Each initial value x_{0}\in\mathbb{R}^{D}\sim X_{t=0} corresponds to a specific trajectory (streamline) \mathbf{x}(t)=\phi(x_{0},t) 1 1 1 Throughout this paper, we distinguish between the spatial coordinate x_{t}\in\mathbb{R}^{D} for the global Eulerian view of the flow and the time-dependent trajectory \mathbf{x}(t) for the Lagrange view of the flow. . The relation between the points along the trajectory and their velocity vectors v(\mathbf{x}(t),t) can be expressed using the following ordinary differential equation (ODE): \frac{d\mathbf{x}(t)}{dt}=v(\mathbf{x}(t),t). (1) The CNF parametrizes the velocity vector field using a neural network (neural ODE) v_{\theta}(\mathbf{x}(t),t) dependent on the parameters \theta . Given a trained CNF and a numerical ODE-solver, e.g., Euler or RK4, sampling from the data distribution can be approximated by solving the initial value problem (IVP) of Eq. 1 forwards

Read the complete text