Support-Conditioned Flow Matching Is Kernel Smoothing

Daniel Matsui Smola · 2026

Read Support-Conditioned Flow Matching Is Kernel Smoothing by Daniel Matsui Smola (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13386.txt.

Introduction

Daniel Matsui Smola proves that conditioning a generative model on a small support set amounts to kernel smoothing. Under a Gaussian optimal-transport path, the exact velocity field is a Nadaraya-Watson smoother. Its bandwidth shrinks as flow time advances, moving from broad averaging to nearest-neighbour behaviour. A single Gaussian-kernel attention head computes this field exactly. The theory predicts three failure regimes, including collapse at high dimension. Experiments confirm that learned conditioning helps precisely in those regimes.

Catalog record

CREATORDaniel Matsui Smola
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50972
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.13386

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Support-Conditioned Flow Matching Is Kernel Smoothing Daniel Matsui Smola Abstract Generative models are often conditioned on a small set of examples via cross-attention. Under the Gaussian optimal-transport path, we show that the exact velocity field induced by a finite support set is a Nadaraya–Watson kernel smoother whose bandwidth decreases with flow time, from broad averaging at early steps to nearest-neighbor at late steps. A single Gaussian-kernel attention head exactly computes this field, connecting cross-attention conditioning to classical kernel theory. The theory predicts three failure regimes: nearest-neighbor collapse of the kernel at high dimension, mismatch between the isotropic kernel and the data geometry, and insufficient support for nonparametric estimation. Experiments on Gaussian mixtures, spherical shells, and DINOv2 ImageNet features confirm that learned conditioning improves in precisely these regimes, and that IP-Adapter’s cross-attention implements approximate NW smoothing in practice. Code 1 Introduction Text alone often cannot specify the visual attributes one wants to generate in an image: a particular face, a beloved pet, art style, or object appearance. The standard solution is to condition on reference examples via cross-attention: IP-Adapter ( Ye et al. 2023 ) , Context Diffusion ( Najdenkoska et al. 2024 ) , SuTI ( Chen et al. 2023 ) , and few-shot diffusion models ( Giannone et al. 2022 ) all condition the generative process by attending to a set of reference image features. However, this mechanism is understood only empirically. One may ask: What is the mathematical structure of the velocity field that a finite reference set induces? What governs the success or failure of cross-attention conditioning? We answer these questions in the flow matching setting. Given a finite support set of samples from an unknown target distribution, the task is to learn a velocity field that generates fresh samples via ODE integration without parameter updates at test time. We show that the exact velocity field induced by such a support set under the Gaussian optimal-transport path is a Nadaraya–Watson (NW) kernel smoother ( Nadaraya 1964 ; Watson 1964 ) . This amounts to a weighted average of the support points, with Gaussian kernel weights that concentrate more heavily on nearby points as the flow progresses. A single Gaussian-kernel cross-attention head computes this field exactly (Theorem 3 ), for any support set and any flow time. Classical kernel smoothing has well-characterized failure modes, and each appears in the conditioning setting: 1. Nearest-neighbor collapse. At high dimension, concentration of measure collapses the Gaussian kernel weights to a single nearest neighbor. The exact field reduces to nearest-neighbor interpolation, and using it as an inductive bias becomes harmful. 2. Geometry mismatch. The isotropic kernel smoothes uniformly in all directions. When variation in the data is concentrated along a low-dimensional subspace, the kernel under-smoothes the informative directions while wasting bandwidth on uninformative ones, yielding high-variance velocity estimates independently of support size. 3. Support scarcity. At small support sizes, nonparametric estimation is data-starved. Learned models improve by amortizing over a meta-distribution, exploiting cross-task structure that single-task kernel estimation cannot access. We verify each failure mode experimentally on Gaussian mixtures, spherical shells, and DINOv2 ImageNet features, and show that learned conditioning improves precisely where the theory predicts the exact field is deficient. IP-Adapter’s attention weights approximate NW kernel weights in practice (§ 5 ), suggesting the framework applies beyond our controlled setting. Relation to prior work. Tsai et al. 2019 and Goel & Bartlett 2024 show that attention can implement kernel regression. We show that the OT-FM velocity field is a kernel smoother—the attention construction is one implementation of a structure already present in the velocity. Fukumizu et al. 2025 and Kunkel & Trabs 2025 derive minimax rates from the endpoint KDE connection; Zhou et al. 2026 analyzes the empirical velocity and discretization bias. We derive the full-path velocity-field decomposition, of which the endpoint KDE is a special case, and experimentally verify its predicted failure modes. Contributions. • We derive the NW velocity identity (Proposition 2 ). • We prove one cross-attention head computes this field exactly (Theorem 3 ). • We identify three failure modes from kernel theory and confirm each experimentally (§ 4 ). • We show that multi-head attention decomposes into H NW smoothers in d_{k} dimensions (Proposition 4 ). • We show IP-Adapter’s attention weights approximate NW kernel weights (§ 5 ). 2 Background and setup Flow matching learns a velocity field v(x,t) that transports samples from a noise distribution ( t=0 ) to a target data distribution ( t=1 ) via ODE integration \mathrm{d}x/\mathrm{d}t=v(x,t) . To generate, one draws noise X_{0}\sim\mathcal{N}(0,I_{d}) and integrates forward to obtain a sample at t=1 . Optimal-transport (OT) flow matching. Given a data distribution q on \mathbb{R}^{d} and base p_{0}=\mathcal{N}(0,I_{d}) , the OT conditional path ( Lipman et al. 2023 ) is X_{t}=t\,X_{1}+\sigma_{t}\,X_{0},\qquad\sigma_{t}=1-(1-\sigma_{\mathrm{min}})\,t, (1) where X_{1}\sim q , X_{0}\sim\mathcal{N}(0,I_{d}) , and 0<\sigma_{\mathrm{min}}\leq 1 . The conditional flow matching objective regresses the conditional velocity target Y_{t}=X_{1}-(1-\sigma_{\mathrm{min}})X_{0} . We define the de-scaled bandwidth h(t)=\sigma_{t}/t , which is strictly decreasing on (0,1] , sweeping from \infty (at t\to 0 ) to \sigma_{\mathrm{min}} (at t=1 ). As we will show, this quantity connects flow matching time to kernel density estimation bandwidth. Nadaraya–Watson kernel regression. For bandwidth h>0 and support set S=\{s_{i}\}_{i=1}^{m} , the Gaussian kernel weights and local mean are w_{i}(x)=\frac{\phi_{h}(x-s_{i})}{\sum_{j=1}^{m}\phi_{h}(x-s_{j})},\qquad m_{h}(x;S)=\sum_{i=1}^{m}w_{i}(x)\,s_{i}, (2) where \phi_{h}(z)=(2\pi h^{2})^{-d/2}\exp(-\|z\|^{2}/2h^{2}) . The effective sample size n_{\mathrm{eff}}=1/\sum_{i}w_{i}^{2} measures how many support points meaningfully contribute. Problem: support-conditioned generation. Given S=\{s_{i}\}_{i=1}^{m}\overset{\text{iid}}{\sim}q from an unknown q , the goal is to learn a velocity field v_{\theta}(x,t,S) that generates fresh samples from q via ODE integration, without parameter updates at test time. We call this In-Context Flow Matching (ICFM) and train with the support-conditioned loss: \mathcal{L}(\theta)=\mathbb{E}_{q\sim\Pi}\;\mathbb{E}_{S\sim q^{m}}\;\mathbb{E}_{X_{1}\sim q,\,X_{0}\sim\mathcal{N}(0,I_{d}),\,t\sim\tau}\bigl[\|v_{\theta}(X_{t},t,S)-Y_{t}\|^{2}\bigr], (3) where \Pi is a meta-distribution over data distributions and \tau=\mathrm{Uniform}(0,1] . We distinguish three velocity fields. The population field u_{t}^{q} is the marginal velocity induced by the true q . The empirical plug-in field u_{t}^{S} replaces q by \hat{q}_{S}=m^{-1}\sum_{i}\delta_{s_{i}} and is exactly computable for any support set via a single Gaussian-kernel attention head (§ 3.3 ). The learned field v_{\theta} is trained over tasks from \Pi . The empirical question is whether v_{\theta} improves over u_{t}^{S} , and if so, whether the improvement can be explained by identifiable failure modes of the plug-in field. 3 The NW velocity identity Full proofs are in Appendix A ; we state the key results with proof sketches here. 3.1 The OT path as a bandwidth sweep We show that the OT probability path through a finite support set is a kernel density estimate (KDE) whose bandwidth decreases with time. The key step is de-scaling : dividing X_{t}=tX_{1}+\sigma_{t}X_{0} by t gives \tilde{X}_{t}\coloneqq X_{t}/t=X_{1}+(\sigma_{t}/t)\,X_{0} . Since X_{0}\sim\mathcal{N}(0,I_{d}) , this is X_{1} plus Gaussian noise at standard deviation h(t)=\sigma_{t}/t . When the data distribution is the empirical measure \hat{q}_{S}=\frac{1}{m}\sum_{i}\delta_{s_{i}} (uniform point masses at the support points), the de-scaled density is a Gaussian KDE: Proposition 1 (De-scaled empirical OT path is a KDE) . Under the Gaussian OT conditional path ( 1 ), for every t\in(0,1] , the de-scaled empirical density \tilde{p}_{t}^{S}(\tilde{x})\coloneqq t^{d}\,p_{t}^{S}(t\tilde{x}) equals \tilde{p}_{t}^{S}(\tilde{x})=\frac{1}{m}\sum_{i=1}^{m}\phi_{h(t)}(\tilde{x}-s_{i}),\qquad h(t)=\frac{\sigma_{t}}{t}. (4) As t traverses (0,1] , h(t) sweeps from \infty to \sigma_{\mathrm{min}} : the OT path through a support set is a continuous multi-resolution KDE, from maximum smoothing

Read the complete text