Constructive Disintegration and Conditional Modes
Nathaël Da Costa, Marvin Pförtner, Jon Cockayne · 2025
Read Constructive Disintegration and Conditional Modes by Nathaël Da Costa, Marvin Pförtner, Jon Cockayne (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-00617.txt.
Introduction
Nathael Da Costa, Marvin Pfortner, and Jon Cockayne build tools for constructing disintegrations, the formal backbone of conditioning in probability, in a 2025 paper. Conditioning on an observation is central to Bayesian reasoning, yet the objects that define it are given only indirectly, so building them in hard cases is tricky. A common shortcut treats a sliced density as if it were the conditional one.
The authors show how to build true disintegrations, including on curved spaces, and give a simple case where the shortcut and the true answer differ wildly. They also sort out a recent notion of conditional mode, showing it matches the shortcut's peaks, not the true conditional's. Both views have uses, they argue, if chosen knowingly.
Catalog record
| CREATOR | Nathaël Da Costa, Marvin Pförtner, Jon Cockayne |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 98 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-00617
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Constructive Disintegration and Conditional Modes Nathaël Da Costa, Marvin Pförtner, Jon Cockayne Abstract Conditioning, the central operation in Bayesian statistics, is formalised by the notion of disintegration of measures. However, due to the implicit nature of their definition, constructing disintegrations is often difficult. A folklore result in machine learning conflates the construction of a disintegration with the restriction of probability density functions onto the subset of events that are consistent with a given observation. We provide a comprehensive set of mathematical tools which can be used to construct disintegrations and apply these to find densities of disintegrations on differentiable manifolds. Using our results, we provide a disturbingly simple example in which the restricted density and the disintegration density drastically disagree. Motivated by applications in approximate Bayesian inference and Bayesian inverse problems, we further study the modes of disintegrations. We show that the recently introduced notion of a “conditional mode” does not coincide in general with the modes of the conditional measure obtained through disintegration, but rather the modes of the restricted measure. We also discuss the implications of the discrepancy between the two measures in practice, advocating for the utility of both approaches depending on the modelling context. 1 Introduction The construction of conditional distributions is central to statistics and probabilistic machine learning. On discrete event spaces, these can often be constructed directly, by applying the definition of conditional probability. However, this construction breaks down in case the event to be conditioned on has probability zero, which is virtually always the case on continuous event spaces. In this case, the construction is more involved, and many texts focus solely on the setting where a Lebesgue density is available or, more generally, where the conditional is dominated by, and thus has a density with-respect-to, the base measure. However, recent works have generated a profusion of examples in which this familiar construction is no longer valid, primarily due to conditionals being defined on a submanifold of the support of the measure. Some examples include: • Bayesian deep learning in which reparameterisation invariances often lead to complex multimodalities in posterior measures (e.g. Wiese et al. 2023 , Laurent et al. 2024 ), potentially mitigated by restricting a prior to a submanifold to limit these invariances, • Directional statistics [ Mardia and Jupp 1999 ] where learning problems are defined on manifolds such as the Stiefel manifold of orthonormal frames, a submanifold of \mathbb{R}^{d\times k} , • Probabilistic numerics [ Cockayne et al. 2019 ] which utilises noise-free observations of an unknown, resulting in posteriors concentrated on submanifolds. There is therefore a growing need to be able to characterise conditional distributions on a more intrinsic, measure-theoretic level than by using densities. In modern probability theory, conditioning is formalised as disintegration . A disintegration of a measure \mu on \mathbb{X} along a map h:\mathbb{X}\to\mathbb{Y} is defined as the almost-surely 1 1 1 Specifically, h_{\star}\mu -almost surely; see Definitions 2.2 and 2.4 . unique family of distributions \{\mu^{h}\left(\,\cdot\,\nonscript\>\middle|\allowbreak\nonscript\>\mathopen{}y\right)\}_{y\in\mathbb{Y}} that satisfies the law of total probability. Existence and almost-sure uniqueness of disintegrations is provided by disintegration theorems which have been available under mild conditions since at least Dellacherie and Meyer 1978 ; see Chang and Pollard 1997 for a more recent exposition. However proofs are typically abstract and, crucially, non-constructive outside of conjugate settings, such as with Gaussian \mu and linear h . This paper provides, to our knowledge, the first general set of results for constructing disintegrations. (a) Prior density and observation fibers ( ) (b) Renormalized restricted density ( ) and disintegration density ( ) for y=1.01 . (c) Renormalized restricted densities (d) Disintegration densities Figure 1 : Conditioning a standard Gaussian prior \mu on observations made through the quadratic observation operator h(x)=\frac{x_{1}^{2}}{a^{2}}+\frac{x_{2}^{2}}{b^{2}} with a=1 and b=\frac{1}{2} . In machine learning folklore, the corresponding conditional distribution is constructed by first restricting the prior’s density \derivative{\prior}{\lebesgue} to the observation fiber h^{-1}\pqty{\set{\obs}} , which is then renormalized w.r.t. the canonical volume measure on the fiber. However, in general, the law of total probability fails to hold for this restricted measure. Our main result, Theorem 2.14 , shows that an additional corrective factor of \norm{\nabla h(x)}_{2}^{-1} needs to be multiplied into this restricted density to obtain the density of the true conditional distribution, which is defined implicitly through the notion of a disintegration . Intuitively speaking, this is due to the fact that, for the law of total probability to hold, the predictive probability mass has to be distributed between fewer fibers where the fibers lie less dense, which is the case if \norm{\nabla h(x)}_{2} is small (see plots above). The plots show a stark disagreement between the restricted and disintegration densities. Notably the modes of the measures differ maximally. See Example 2.1 for details. Transferring intuition from the density setting, a “folklore” result in machine learning conflates disintegration of a distribution with restriction of the distribution to the submanifold on which the conditional is supported. For examples of this see e.g. [ Bishop 2006 , Figure 2.18] , which describes the von Mises distribution as being obtained by “considering a two-dimensional Gaussian distribution […] and conditioning on the unit circle”. In fact this is a restriction rather than a conditional, and our novel results highlight that these can differ dramatically. In Figure 1 we provide a (disturbingly) simple example related to this distribution in which restriction and disintegration result in radically different distributions. This example takes an ambient standard Gaussian measure on \mathbb{R}^{2} ( Figure 1(a) ) and contrasts restricting ( Figure 1(c) ) and disintegrating ( Figure 1(d) ) the measure on successively larger ellipses; see Example 2.1 for more detail. The colour maps on contours in Figures 1(c) and 1(d) depict appropriate densities for the restriction or disintegration, respectively. Notably, for smaller ellipses, the mass concentrates at different poles between the two approaches. A special case of our characterisation result on \mathbb{R}^{d} is reproduced below: 1.1 . Let \mu be a Borel probability measure on \mathbb{R}^{d} that has a density \derivative{\prior}{\lebesgue} w.r.t. the Lebesgue measure \lambda on \mathbb{R}^{d} . Let h\colon\mathbb{R}^{d}\to\mathbb{R}^{n} such that \mu -almost all points x\in\mathbb{R}^{d} are C^{1} regular (see Definition A.8 ). Then there exists a version of the disintegration \mu^{h} with probability densities 2 2 2 The densities are with respect to the restriction \lambda_{h^{-1}\pqty{\set{\obs}}} of the Lebesgue measure to the observation fiber h^{-1}\pqty{\set{\obs}} (see Definition 2.13 ). \derivative{\priordisint}{\lebesgue_{\obsfib}}{(x)}\propto\derivative{\prior}{\lebesgue}{(x)}\cdot\absolutevalue{\det(\jacobian{\obsop}(\latval)\rvert_{\kernel{\jacobian{\obsop}(\latval)}^\perp})}^{-1} for all C^{1} regular values y\in\mathbb{R}^{n} , where \lambda_{h^{-1}\pqty{\set{\obs}}} is the restricted Lebesgue measure to the fiber h^{-1}\pqty{\set{\obs}} (see Definition 2.13 ). Theorem 1.1 shows that the disintegration density differs from the restriction density by a corrective factor depending on the Jacobian of h . See Theorem 2.14 for further details. In the case where h is linear, this Jacobian is constant, and we hence recover the classical proportionality formula for densities \derivative*{\priordisint}{\lebesgue_{\obsfib}}\propto\derivative*{\prior}{\lebesgue} . To demonstrate the importance of our results we focus on the problem of mode computation, i.e., computation of maximum a-posteriori (MAP) points in Bayesian computation. MAP estimation summarises a distribution with a single, representative estimate that can be interpreted informally as a “point of maximal probability”. This is particularly significant in Bayesian inference. In many practical settings sampling from a full posterior is computationally expensive and a point