A Local Characterization of f f -Divergences Yielding PSD Mutual-Information Matrices
Zachary Robertson · 2026
Read A Local Characterization of f f -Divergences Yielding PSD Mutual-Information Matrices by Zachary Robertson (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-08929.txt.
Introduction
Zachary Robertson asks when a table of shared facts between traits is a sound table in a strict math sense. Each cell in the table holds a shared fact score for a pair of traits. The score can be built in more than one way, based on a curve that sets how gaps count. A sound table here means one with no bad eigen traits, called PSD for short.
He gives a local test that tells which curves yield sound tables. The test rests on how the curve acts near the point where two traits match in full. Such tables show up in learn tools that use shared fact as a link gauge. A reader gets a crisp rule for when that gauge is safe to use as a base.
Catalog record
| CREATOR | Zachary Robertson |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-08929
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Local Characterization of f f -Divergences Yielding PSD Mutual-Information Matrices Zachary Robertson Abstract We study when the variable-indexed matrix of pairwise f f -mutual informations M i j ( f ) = I f ( X i , X j ) M^{(f)}_{ij}=I_{f}(X_{i};X_{j}) is positive semidefinite (PSD). Let f : ( 0 , ∞ ) → ℝ f:(0,\infty)\to\mathbb{R} be convex with f ( 1 ) = 0 f(1)=0 , finite in a neighborhood of 1 1 , and with f ( 0 ) < ∞ f(0)<\infty so that diagonal terms are finite. We give a sharp local characterization around independence: there exists δ = δ ( f ) > 0 \delta=\delta(f)>0 such that for every n n and every finite-alphabet family ( X 1 , … , X n ) (X_{1},\ldots,X_{n}) whose pairwise joint-to-product ratios lie in ( 1 − δ , 1 + δ ) (1-\delta,1+\delta) , the matrix M ( f ) M^{(f)} is PSD if and only if f f is analytic at 1 1 with a convergent expansion f ( t ) = ∑ m = 2 ∞ a m ( t − 1 ) m f(t)=\sum_{m=2}^{\infty}a_{m}(t-1)^{m} and a m ≥ 0 a_{m}\geq 0 on a neighborhood of 1 1 . Consequently, any negative Taylor coefficient yields an explicit finite-alphabet counterexample under arbitrarily weak dependence, and non-analytic convex divergences (e.g. total variation) are excluded. This PSD requirement is distinct from Hilbertian/metric properties of divergences between distributions (e.g. JS \sqrt{\mathrm{JS}} ): we study PSD of the variable-indexed mutual-information matrix. The proof combines a replica embedding that turns monomial terms into Gram matrices with a replica-forcing reduction to positive-definite dot-product kernels, enabling an application of the Schoenberg–Berg–Christensen–Ressel classification. I Introduction Positive semidefinite (PSD) mutual-information matrices enable kernel representations of multivariate dependence. Given n random variables X_{1},\ldots,X_{n} , the mutual-information matrix has entries M_{ij}=I(X_{i};X_{j}) . When M\succeq 0 , the variables admit useful feature/kernels for downstream tasks in factor analysis, testing, and machine learning [ 1 , 2 , 3 ] . In this work we study PSD of the variable-indexed matrix of pairwise mutual informations, which is conceptually different from metric/Hilbertian results for divergences between distributions . Jakobsen [ 4 ] shows MI matrices can be indefinite for n\geq 4 (Shannon), and other work shows some divergences (e.g. \sqrt{\text{JS}} ) yield metrics on distributions [ 5 , 6 ] . In this paper, we characterize the local behavior of f -divergences that yield PSD f -mutual-information matrices. An f -divergence between distributions P and Q is D_{f}(P\|Q)=\sum_{x}q(x)\,f\!\left(\frac{p(x)}{q(x)}\right), with f convex and f(1)=0 [ 7 , 8 ] . We define I_{f}(X;Y)=D_{f}\big(P_{XY}\,\|\,P_{X}\otimes P_{Y}\big), and set M^{(f)}_{ij}:=I_{f}(X_{i};X_{j}) whenever these values are finite (in particular, the diagonal is finite under mild conditions such as f(0)<\infty ). Our main theorem is uniform in n and assumes only pairwise weak dependence (no global joint constraints): it identifies exactly when M^{(f)} is PSD for all finite alphabets in a neighborhood of independence. Our main result characterizes the PSD property locally around independence: f -MI matrices remain PSD for all numbers of variables n and all finite alphabets if and only if f has a local power series with nonnegative coefficients. Equivalently, the local Taylor coefficients of any PSD-generating f at t=1 lies in the cone spanned by \{(t-1)^{m}:m\geq 2\} [ 9 , 10 ] . II Main Results The admissible local Taylor coefficients at t=1 form a closed convex cone. Our main theorem shows this cone consists precisely of the nonnegative mixtures of powers (t-1)^{m} for m\geq 2 . Theorem II.1 (PSD-generating f : local characterization) . Let f:(0,\infty)\to\mathbb{R} be convex with f(1)=0 , and assume f admits a finite boundary value f(0):=\lim_{t\downarrow 0}f(t)<\infty (so diagonal terms I_{f}(X_{i};X_{i}) are finite in our constructions). Assume further that f is finite on a neighborhood (1-\varepsilon,1+\varepsilon) of t=1 . The following are equivalent: 1. There exists a dependence radius \delta=\delta(f)\in(0,\varepsilon) such that for every n\in\mathbb{N} and every finite-alphabet family of discrete random variables (X_{1},\dots,X_{n}) that is \delta -pairwise-weakly-dependent (Definition III.1 ), the matrix M^{(f)}_{ij}\;:=\;I_{f}(X_{i};X_{j}) is positive semidefinite. Here \delta is uniform in n , and the assumption is purely pairwise (no conditions on higher-order marginals). 2. f is absolutely monotone at t=1 : there exist coefficients a_{m}\geq 0 and an interval |t-1|<\eta such that f(t)\;=\;\sum_{m=2}^{\infty}a_{m}(t-1)^{m} for all t with |t-1|<\eta . Theorem II.1 has three immediate consequences. A local obstruction suffices: a single negative Taylor coefficient of f at 1 yields an explicit finite-alphabet counterexample under arbitrarily weak dependence. Non-analytic divergences are excluded: convex but non-analytic f at 1 (e.g. total variation) cannot generate PSD f -MI matrices even locally around independence. Distinct from distribution metrics: Hilbertian/metric properties of a divergence between distributions (e.g. \sqrt{\mathrm{JS}} ) do not imply PSD of the variable-indexed matrix [I_{f}(X_{i};X_{j})] . The necessity direction probes only a neighborhood of t=1 : our counterexample constructions ensure that all arguments of f appearing in the induced kernel H_{a} lie in (1-\varepsilon,1+\varepsilon) . Consequently, a single negative Taylor coefficient at t=1 yields a finite-alphabet counterexample under arbitrarily weak dependence. Conversely, if f is absolutely monotone at t=1 , then for sufficiently small \delta the expansion f(t)=\sum_{m\geq 2}a_{m}(t-1)^{m} is valid on (1-\delta,1+\delta) . Lemma IV.2 then reduces PSD of M^{(f)} to PSD of the monomial cases f_{m}(t)=(t-1)^{m} , which hold by the replica embedding of Proposition IV.1 . A simple global sufficient condition is that f(t)=\sum_{m=2}^{\infty}a_{m}(t-1)^{m} with a_{m}\geq 0 holds for all t>0 ; then PSD holds without a small-dependence restriction (by Proposition IV.1 and Lemma IV.2 ). In summary, Theorem II.1 shows that even local positive semidefiniteness near independence imposes severe restrictions on f : outside of nonnegative mixtures of (t-1)^{m} in a neighborhood of t=1 , PSD fails for weakly dependent variables. This explains why Shannon mutual information fails, and why in practice many twice-differentiable divergences appear PSD near independence; their leading term is the \chi^{2} -divergence. II-A Counterexamples when f is not absolutely monotone When f fails absolute monotonicity at 1 , we can construct explicit finite-alphabet families whose f -MI matrix is indefinite while remaining arbitrarily close to independence (in the sense of Definition III.1 ). The construction proceeds by reducing the matrix PSD requirement to positive definiteness of a scalar dot-product kernel and then amplifying any negative direction via conditional replicas. Three components drive this reduction: Latent family and three-point mixture We work with a biased latent-variable family that yields a scalar kernel representation I_{f}(Y_{i};Y_{j})=:H_{a}(\rho_{ij}) , where \rho_{ij} is a covariance-like parameter determined by the loadings. This reduces necessity to positive definiteness of the dot-product kernel z\mapsto H_{a}(z) on small Gram sets, enabling an application of the Schoenberg–Berg–Christensen–Ressel characterization. Replica block forcing: from f -MI to scalar PD kernels This family does not give us a Gram matrix because of the diagonal deviations. For a fixed finite family \{u_{i}\}_{i=1}^{n} , we introduce a technique to address this. Given a single draw of the latents (U_{1},\ldots,U_{k}) , we form R conditionally independent copies Y_{i}^{(1)},\ldots,Y_{i}^{(R)} of each Y_{i} —that is, independent draws from \Pr(Y_{i}=\cdot\mid U_{1},\ldots,U_{k}) . Form the n\times n kernel matrix \displaystyle K_{0} \displaystyle=[H_{0}(\langle u_{i},u_{j}\rangle)]_{i,j}, \displaystyle\Delta_{0} \displaystyle=\mathrm{diag}(d_{0}-H_{0}(\|u_{1}\|^{2}),\ldots,d_{0}-H_{0}(\|u_{n}\|^{2})). The f -MI matrix over these R conditional replicas, with J_{R} the all-ones matrix, takes the form B_{R}=J_{R}\otimes