A novel sequential method for building upper and lower bounds of moments of distributions Journal: Journal of The Franklin Institute

Solal Martin, Emilie Chouzenoux, Víctor Elvira · 2025

Read A novel sequential method for building upper and lower bounds of moments of distributions Journal: Journal of The Franklin Institute by Solal Martin, Emilie Chouzenoux, Víctor Elvira (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-01761.txt.

Introduction

Solal Martin, Emilie Chouzenoux, and Victor Elvira bound moments of distributions with a sequential method. Moments summarize a distribution's shape, like mean and spread. Computing them means integrals over the distribution. Many such integrals lack closed forms, so numeric methods must step in.

Their method builds upper and lower bounds step by step. Bounds certify accuracy, unlike point guesses. Sequential bounds can stop once tight enough. Certified moments serve signal processing and Bayesian learning, where downstream choices rest on those values. Sequential bounds adapt to hardness, spending steps where the integral resists and stopping early where the shape is tame and clear.

Catalog record

CREATORSolal Martin, Emilie Chouzenoux, Víctor Elvira
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE80 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-01761

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A novel sequential method for building upper and lower bounds of moments of distributions Journal: Journal of The Franklin Institute Solal Martin, Emilie Chouzenoux, Víctor Elvira Abstract Approximating integrals is a fundamental task in probability theory and statistical inference, and their applied fields of signal processing, and Bayesian learning, as soon as expectations over probability distributions must be computed efficiently and accurately. When these integrals lack closed-form expressions, numerical methods must be used, from the Newton-Cotes formulas and Gaussian quadrature, to Monte Carlo and variational approximation techniques. Despite these numerous tools, few are guaranteed to preserve majoration/minoration inequalities, while this feature is fundamental in certain applications in statistics. In this paper, we focus on the integration problem arising in the estimation of moments of scalar, unnormalized, distributions. We introduce a sequential method for constructing upper and lower bounds on the sought integral. Our approach leverages the majorization-minimization framework to iteratively refine these bounds, in an enveloped principle. The method has proven convergence, and controlled accuracy, under mild conditions. We demonstrate its effectiveness through a detailed numerical example of the estimation of a Monte-Carlo sampler variance in a Bayesian inference problem. 1 Introduction and Problem Statement 1.1 Introduction The approximation of intractable integrals has long been for long a fundamental problem in mathematical sciences, leading to extensive research and the development of various methods [ 30 , 23 , 22 ] . Classical approaches include Newton-Cotes formulas, such as the trapezoidal and Simpson’s rules, which approximate the integrand using polynomial interpolation over equally spaced points [ 11 ] . While these methods are easy to implement, they can have low accuracy for highly oscillatory functions due to the Runge’s phenomenon [ 12 ] . A more advanced approach relies on Gaussian quadrature, which optimizes deterministics integration points and weights to achieve higher accuracy with fewer function evaluations [ 6 , 14 ] . MATLAB’s integrator [ 33 ] , for example, extends this idea using the Gauss-Kronrod quadrature rule, which enhances accuracy by adding additional nodes to an existing Gaussian quadrature formula. For integrals over infinite domains or with singularities, adaptive quadrature methods dynamically refine the integration mesh based on local error estimates, improving efficiency and precision [ 20 ] . When dealing with multi-dimensional settings, traditional quadrature methods rapidly become impractical, and Monte Carlo methods provide an alternative approach by estimating integrals through random sampling [ 30 , 28 ] . Importance sampling methods, an important family of Monte Carlo algorithms, improve efficiency by favoring regions that contribute most to the integral’s value [ 30 , 16 ] . Another powerful approach in high-dimensional integration problems is sparse grid integration, which reduces the curse of dimensionality by combining lower-order quadrature rules adaptively [ 25 ] . Each of these methods has its advantages and limitations, making their selection highly dependent on the problem at hand. A more detailed review of these numerical integration techniques can be found in standard numerical analysis texts, such as the reference book [ 18 ] , for instance. An important class of integration problem is the computation a unidimensional moment integral of a given target distribution. Beyond that, the assessment of moments methods often require the evaluation of the variance of the computed estimator, to evaluate the method’s efficiency. One widely used diagnostic in this context is the Effective Sample Size (ESS), which provides a quantitative measure of how well the weighted samples represent the target distribution. ESS is relevant in importance sampling and sequential Monte Carlo methods, where weight degeneracy can significantly affect estimator quality [ 24 , 15 ] . Other techniques include variance reduction strategies such as control variates, stratified sampling, and resampling methods, which aim to improve estimator stability and convergence [ 29 , 13 ] . The variance of the estimator is often expressed as an integral as hard as the original problem. Even more the lower and upper bounds of the variance, also involve intractable integrals. Numerical approximation can be used. However, most state-of-the-art integration methods come with significant approximation errors, and, do not guarantee to preserve inequalities. This paper tackles this challenge by providing an original sequential approach to compute tight bounds for moment integrals. 1.2 Problem Statement Let f:\mathbb{R}\mapsto\mathbb{R} and \phi:\mathbb{R}\mapsto\mathbb{R} , differentiable. We consider in this paper the following moment integral: \mathcal{I}\equiv\int_{-\infty}^{+\infty}f(x)\pi(x)dx,\quad\text{with}\quad\pi(x)\equiv\exp(-\phi(x)). (1) No normalization assumption is made on \pi , that is Z\equiv\int_{-\infty}^{+\infty}\pi(x)dx might be different to one. For most choices of f and \phi , \mathcal{I} does not have a closed-form expression and should be approximated by a numerical method. The goal of this paper is to provide an efficient iterative strategy for computing tight lower and upper bounds of \mathcal{I} , that we define as \underline{\mathcal{I}}\leq\mathcal{I}\leq\overline{\mathcal{I}}. (2) For the ease of presentation, we will consider throughout the paper, that function f is a monomial function, i.e., (\forall x\in\mathbb{R})\quad f(x)=x^{k},\,\text{for some}\;k\in\mathbb{N}. (3) If k=0 , the problem amounts to computing the normalization constant of \pi , since \mathcal{I}=\mathcal{Z} . Extension of the method to a polynomial test function f , will be discussed in the Remark 1 , in the end of Section 4 . Throughout the paper, we assume that the problem is well-posed, that is f and \pi are such that \mathcal{I}\in\mathbb{R} . 1.3 Paper Contributions and Outline This paper presents a new sequential method to compute the bounds (\underline{\mathcal{I}},\overline{\mathcal{I}}) in ( 2 ) with controlled precision. Our approach first stands on the construction of piecewise Gaussian upper and lower approximations for \pi , by relying on the powerful principle of majoration-minimization [ 9 , 10 , 32 ] . We then design a recursive method, to compute upper and lower envelopes, from the set of piecewise approximations. Our approach hence yields the construction of two sequences of upper and lower bounds for the integral of interest ( 1 ), with increasing, and controlled, accuracy, and both proved to converge to the exact integral value \mathcal{I} . Although integral \mathcal{I} could be computed, in special cases of \pi , using known results such as the moments of truncated Gaussian distributions, our method applies more generally to any unnormalized distribution satisfying the assumptions detailed in Section 3 . We then specialize our method to the computation of bounds on the variance of an importance sampling estimator. Finally, we illustrate its practical effectiveness of our method, and compare it to the state-of-the-art envelope-based integration technique from [ 18 , Sec.5.4] , in an application of statistical inference, namely a Bayesian classification problem. The outline of the paper is as follows. In Section 2 , we introduce our construction of upper and lower Gaussian approximations for \pi , under reasonable assumptions on \phi . We deduce a preliminary formula of rough lower and upper bounds \underline{\mathcal{I}} and \overline{\mathcal{I}} for \mathcal{I} . Then, in Section 3 , we propose an iterative algorithm which aims at producing (\underline{\mathcal{I}}^{(n)})_{n\in\mathbb{N}} and (\overline{\mathcal{I}}^{(n)})_{n\in\mathbb{N}} , that are sequences of refined lower and upper bounds for the integral of interest, \mathcal{I} . We prove in Section 4 that both sequences converge to \mathcal{I} as n approaches infinity, and we provide a quantitative control of the error along iterations. In Section 5 , we formulate the problem of variance estimation of a Monte-Carlo

Read the complete text