Relationship between Hölder Divergence and Functional Density Power Divergence: Intersection and Generalization

Masahiro Kobayashi · 2025

Read Relationship between Hölder Divergence and Functional Density Power Divergence: Intersection and Generalization by Masahiro Kobayashi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-17008.txt.

Introduction

Masahiro Kobayashi maps how two families of distance measures between data patterns relate, in a 2025 paper. Such measures, called divergences, grade how far one spread of data sits from another, and robust ones guard against stray points. Both families grow from a power-based root and cover famous older measures as special cases. One family adds free shape functions through a classic inequality trick.

Kobayashi proves the two families overlap only inside an older, narrower family from a 2001 paper. He then builds a wider family that covers both, proving its key trait with a blend of the two older proofs. New ranking rules fall out of the wider frame, linking scores across the families. The work gives statisticians one roof over tools that had stood apart.

Catalog record

CREATORMasahiro Kobayashi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-17008

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Relationship between Hölder Divergence and Functional Density Power Divergence: Intersection and Generalization Masahiro Kobayashi Abstract In this study, we discuss the relationship between two families of density-power-based divergences with functional degrees of freedom—the Hölder divergence and the functional density power divergence (FDPD)—based on their intersection and generalization. These divergence families include the density power divergence and the γ \gamma -divergence as special cases. First, we prove that the intersection of the Hölder divergence and the FDPD is limited to a general divergence family introduced by Jones et al. (Biometrika, 2001). Subsequently, motivated by the fact that Hölder’s inequality is used in the proofs of nonnegativity for both the Hölder divergence and the FDPD, we define a generalized divergence family, referred to as the ξ \xi -Hölder divergence. The nonnegativity of the ξ \xi -Hölder divergence is established through a combination of the inequalities used to prove the nonnegativity of the Hölder divergence and the FDPD. Furthermore, we derive an inequality between the composite scoring rules corresponding to different FDPDs based on the ξ \xi -Hölder divergence. Finally, we prove that imposing the mathematical structure of the Hölder score on a composite scoring rule results in the ξ \xi -Hölder divergence. 1 Introduction Bregman div. NB-DPD Hölder div. FDPD DPD \gamma -div. BDPD BHD This paper Figure 1: Relationship among the DPD, the \gamma -divergence, and their generalized divergence families. In robust inference, the density power divergence (DPD), also known as \beta -divergence, is widely used [ 2 ] . The DPD has a power parameter \gamma\geq 0 that controls the trade-off between model efficiency and robustness against outliers. The \gamma -divergence [ 6 , 10 ] is another density-power-based divergence. Similarly to the DPD, the \gamma -divergence provides a trade-off between efficiency and robustness. The \gamma -divergence can reduce the latent bias to zero, even when the proportion of outliers in the data is large [ 6 ] . These divergences are defined to ensure that the true distribution can be replaced by the empirical distribution, expressed as a sum of Dirac delta functions. Such divergences are called non-kernel [ 9 ] or decomposable [ 3 ] , and this property is important for practical statistical inference. Motivated by the same consideration, the divergence induced by a composite scoring rule [ 12 ] has been proposed and will be explained in Section 2 . Furthermore, several generalized divergences that include the DPD and the \gamma -divergence as special cases have been developed in the class of non-kernel divergences. Examples of two- or three-parameter divergence families include the JHHB divergence family [ 10 ] , the Bregman–Hölder divergence (BHD) [ 12 ] , and the bridge density power divergence (BDPD) [ 14 , 7 ] . The DPD and the \gamma -divergence can be generalized into three classes of divergences with functional degrees of freedom, within which the two- and three-parameter divergence families are positioned as special subclasses (Fig. 1 ). The first class is the (functional or non-separable) Bregman divergence [ 5 ] , which is defined using a convex functional. This divergence can be represented using proper scoring rules, and the estimation problem can be reduced to M-estimation [ 8 ] . Recently, we have proposed a norm-based Bregman density power divergence (NB-DPD), which is subclass of Bregman divergences characterized by density-power-based formulations [ 13 ] . The second class is the Hölder divergence [ 12 ] , which is characterized by invariance under affine transformations. The third class is the functional density power divergence (FDPD) [ 16 ] , which is defined based on the mathematical structure shared by the DPD and the \gamma -divergence. Specifically, the FDPD is defined by applying a functional transformation to each term of the DPD. Ray et al. [ 16 ] clarified the necessary and sufficient conditions for the function such that the FDPD is valid as a divergence. Investigating the intersection of different divergence families causes a deeper understanding and characterization of their structural properties. Kanamori and Fujisawa [ 12 ] derived the BHD as the intersection of the Bregman divergence and the Hölder divergence. Recently, we demonstrated that the intersection of the Bregman divergence and the FDPD is the BDPD [ 13 ] . However, the relationship between the Hölder divergence and the FDPD has not been discussed. In this study, we discuss the relationship between the Hölder divergence and the FDPD from two perspectives. First, we prove that the intersection of the Hölder divergence and the FDPD is limited to the JHHB divergence family (Fig. 1 ). Furthermore, we derive the function \eta corresponding to the Hölder divergence when the JHHB divergence family is expressed in the form of a Hölder divergence. Subsequently, we define a \xi -Hölder divergence that includes both the Hölder divergence and the FDPD as special cases. Furthermore, we derive an inequality between composite scoring rules corresponding to different FDPDs based on the \xi -Hölder divergence. Finally, we prove that the \xi -Hölder divergence can be derived by imposing the mathematical structure of the Hölder score on a composite scoring rule. The proofs of the theorems omitted in the main text are presented in the appendix. 2 Notation and definitions In this section, we introduce certain mathematical definitions. The set of nonnegative real numbers is denoted by \mathbb{R}_{+} . Let X\subseteq\mathbb{R}^{d} be a subset of the Euclidean space, and define a measure space (X,\mathcal{B},\nu) . The set of nonnegative functions is denoted by \mathcal{L}_{0}=\{f:X\to\mathbb{R}\mid f\text{ is measurable on }(X,\mathcal{B},\nu),f\geq 0,f\neq 0\} . For f\in\mathcal{L}_{0} , we write the integral with respect to the Lebesgue measure \nu as \langle f\rangle=\int_{X}f(x)d\nu(x) . Suppose that \mathcal{F}\subseteq\mathcal{L}_{0} , and define the set of probability distributions as \mathcal{P}=\{p\in\mathcal{F}\mid\langle p\rangle=1\} . We assume \mathcal{F}_{\gamma}=\{f\in\mathcal{L}_{0}|\langle f^{1+\gamma}\rangle<\infty\} for a fixed \gamma\geq 0 . Here, we define a specific form of composite scoring rule discussed in [ 12 ] . Definition 1 (Composite scoring rule and divergence [ 12 ] ) Let U,V:\mathbb{R}_{+}\to\mathbb{R} , and T:\mathbb{R}^{2}\to\mathbb{R} be given functions. The composite scoring rule S:\mathcal{F}\times\mathcal{F}\to\mathbb{R} is defined as \displaystyle S(g,f)=T\left(\langle gU(f)\rangle,\langle V(f)\rangle\right). The composite scoring rule S is strictly proper if and only if the following conditions hold: \displaystyle\forall g,f\in\mathcal{F},\;S(g,f)\geq S(g,g), \displaystyle\forall q,p\in\mathcal{P},\;S(q,p)=S(q,q),\;\text{if and only if}\;q=p\;\text{almost everywhere}. A divergence is defined as the difference between composite scoring rules. From the properties of composite scoring rule, the following properties of the divergence hold: \displaystyle\forall g,f\in\mathcal{F},\;D(g,f)=S(g,f)-S(g,g)\geq 0, \displaystyle\forall q,p\in\mathcal{P},\;D(q,p)=0\;\text{if and only if}\;q=p\;\text{almost everywhere}. Definition 2 (Equivalence of composite scoring rules [ 12 ] ) Two composite scoring rules \tilde{S}(g,f) and S(g,f) are equivalent if and only if there exists a strictly increasing function \tau:\mathbb{R}\to\mathbb{R} such that \tilde{S}(g,f)=\tau(S(g,f)),\;\mathrm{for\;all}\;g,f\in\mathcal{F} . This equivalence also holds for probability distributions q,p\in\mathcal{P} . The estimator is preferably designed to transform consistently when the data is transformed. An affine transformation of a data is expressed by \bm{x}\mapsto\bm{\Sigma}^{-1}(\bm{x}-\bm{\mu}) , where \bm{\Sigma}\in\mathbb{R}^{d\times d} is an invertible matrix and \bm{\mu}\in\mathbb{R}^{d} is a d -dimensional vector. The corresponding probability distribution transforms as q(\bm{x})\mapsto q_{\bm{\Sigma},\bm{\mu}}(\bm{x})=|\det\bm{\Sigma}|q(\bm{\Sigma}\bm{x}+\bm{\mu}) . When the statistical model \hat{p} estimated from the original data distribution q is equal to the model \widehat{p_{\bm{\Sigma},\bm{\mu}}} estimated from the affinely transformed distribution, i.e., when \hat{p}=\widehat{p_{\bm{\Sigma},\bm{\mu}}} holds, the estimator is called an affine invariant estimator [ 12 ] . A divergence that

Read the complete text