Edgeworth Expansion for Semi-hard Triplet Loss

Masanari Kimura · 2025

Read Edgeworth Expansion for Semi-hard Triplet Loss by Masanari Kimura (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-12893.txt.

Introduction

Masanari Kimura refines the statistics of a workhorse loss used to train embedding models, in this 2025 paper. The semi-hard triplet loss pulls similar items together and pushes unlike ones apart by a margin. Its average behavior hides skew that can sway training.

Using Edgeworth expansions, a sharpened version of the central limit theorem, Kimura derives correction terms tied to the data's third cumulant. The math shows how the margin and the skew of distance gaps interact. Those formulas guide margin choices that keep training stable. The analysis speaks to face and image retrieval systems alike.

Catalog record

CREATORMasanari Kimura
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE18 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-12893

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Edgeworth Expansion for Semi-hard Triplet Loss Masanari Kimura Abstract We develop a higher-order asymptotic analysis for the semi-hard triplet loss using the Edgeworth expansion. It is known that this loss function enforces that embeddings of similar samples are close while those of dissimilar samples are separated by a specified margin. By refining the classical central limit theorem, our approach quantifies the impact of the margin parameter and the skewness of the underlying data distribution on the loss behavior. In particular, we derive explicit Edgeworth expansions that reveal first-order corrections in terms of the third cumulant, thereby characterizing non-Gaussian effects present in the distribution of distance differences between anchor-positive and anchor-negative pairs. Our findings provide detailed insight into the sensitivity of the semi-hard triplet loss to its parameters and offer guidance for choosing the margin to ensure training stability. Journal: Statistics \& Probability Letters 1 Introduction Metric learning, which seeks to learn a meaningful embedding of data into a lower-dimensional space, has numerous applications, such as face recognition [ 1 , 2 , 3 ] , image retrieval [ 4 , 5 , 6 ] , and clustering [ 7 , 8 , 9 ] . A central concept in metric learning is the triplet loss [ 9 , 10 , 11 , 12 ] , which simultaneously minimizes the distance between embeddings of similar samples (anchor-positive pairs) and maximizes the distance between dissimilar samples by enforcing a margin. Although simple in structure, the performance of the triplet loss heavily depends on the choice of the margin parameter and the manner in which negative examples are selected. In practice, the semi-hard negative mining strategy [ 13 , 14 ] is often employed to select informative triplets. Specifically, semi-hard negatives are those for which the embedding distance is greater than that of the positive but still within the margin \alpha . This mining strategy effectively excludes trivial negatives, which do not contribute significantly to training, and overly hard negatives that can destabilize the optimization process. Despite widespread empirical success, there is limited theoretical analysis on how the margin parameter \alpha and the underlying data distribution impact the learning dynamics and performance of the resulting embeddings. This paper addresses this theoretical gap by conducting a rigorous higher-order asymptotic analysis of the semi-hard triplet loss via the Edgeworth expansion [ 15 , 16 ] . We explicitly derive expansions to show how the margin parameter and underlying data distribution affect loss behavior. The resulting insights provide theoretical understanding of semi-hard triplet loss, and also offer practical guidance for selecting the margin parameter to enhance training stability and embedding quality. 2 Preliminary Let \mathcal{X} be the input space and let f\colon\mathcal{X}\to\mathbb{R}^{k} be a trainable embedding function into k -dimensional space. We assume that we have i.i.d. draws of (X_{a},X_{p},X_{n}) from some distribution \mathcal{D} over triplets, where X_{a} is an anchor sample, X_{p} is a positive sample (same label as X_{a} ) and X_{n} is a negative sample (different label from X_{a} ). Let d\colon\mathbb{R}^{k}\times\mathbb{R}^{k}\to\mathbb{R}_{+} be a differentiable distance function. The standard triplet loss with margin \alpha>0 is defined as \mathcal{L}_{\text{triplet}}\coloneqq\mathbb{E}\left[\max\left(\alpha-\Delta,0\right)\right], (1) where \Delta\coloneqq\Delta(X_{a},X_{p},X_{n})\coloneqq d(f(X_{a}),f(X_{n}))-d(f(X_{a}),f(X_{p})) . Semi-hard negative mining seeks negative samples X_{n} that are not too easy and not too hard. That is, those X_{n} satisfy d(f(X_{a}),f(X_{p}))<d(f(X_{a}),f(X_{n}))<d(f(X_{a}),f(X_{p}))+\alpha. (2) Equivalently, the semi-hard condition is 0<\Delta(X_{a},X_{p},X_{n})<\alpha . Define the event E_{\text{sh}}\coloneqq\left\{(X_{a},X_{p},X_{n})\ \colon\ 0<\Delta(X_{a},X_{p},X_{n})<\alpha\right\}. (3) In practice, we only compute or backpropagate the loss for triplets in E_{\text{sh}} . Formally, one can write the semi-hard triplet loss as \mathcal{L}_{\text{semi}}\coloneqq\mathbb{E}\left[1_{E_{\text{sh}}}\cdot\max(\alpha-\Delta,0)\right]. (4) Note that if \Delta\geq\alpha the loss is zero anyway, and if \Delta\leq 0 (an easy case) it also contributes zero. Hence, focusing on semi-hard triples means focusing on \Delta\in(0,\alpha) . Suppose that \Delta is a well-defined random variable with finite moments. Let \mu_{\Delta}\coloneqq\mathbb{E}[\Delta] , \sigma^{2}_{\Delta}\coloneqq\mathrm{Var}(\Delta) , and \bar{\Delta}_{N} be the sample mean of i.i.d. copies \{\Delta\}^{N}_{i=1} , where N is the batch size. The central limit theorem (CLT) implies that \sqrt{N}(\bar{\Delta}_{N}-\mu_{\Delta})\leadsto\mathcal{N}(0,\sigma^{2}_{\Delta}) . However, higher-order asymptotic expansions enable us to quantify how \bar{\Delta}_{N} behaves beyond the leading Gaussian approximation. 3 Main Results We provide statements capturing the role of the margin \alpha , the underlying distribution of distances, and how the semi-hard event constrains \Delta . Lemma 3.1 . Assume \Delta has finite moments up to order s\geq 4 . Denote by \kappa_{j} the j -th cumulant of \Delta . Let Z_{N}\coloneqq\sqrt{N}(\bar{\Delta}_{N}-\mu_{\Delta})/\sigma_{\Delta} be the standard average of i.i.d copies of \Delta . Then, there is an Edgeworth expansion for the distribution function F_{Z_{N}}(z) of the form \displaystyle F_{Z_{N}}(z)=\Phi(z)-\phi(z)\frac{\gamma_{3}}{6\sqrt{N}}(z^{2}-1)+O(N^{-1}), where \Phi is the standard normal c.d.f., \phi is the standard normal p.d.f., and \gamma_{3}=\kappa_{3}/\kappa_{2}^{3/2} is the skewness. Although Lemma 3.1 is classical, it lays the foundation for the expansions needed to analyze the triplet loss. Define the semi-hard probability as P_{\text{sh}}\coloneqq\mathbb{P}(0<\Delta<\alpha)=\mathbb{E}\left[1_{\{0<\Delta<\alpha\}}\right]. (5) Since \Delta is a real-valued random variable, the behavior of P_{\text{sh}} as \alpha varies is crucial to the learning dynamics. For \alpha near \mu_{\Delta} , one can write P_{\text{sh}}(\alpha)\coloneqq\int^{\alpha}_{0}f_{\Delta}(t)dt\approx F_{\Delta}(\alpha)-F_{\Delta}(0) , where F_{\Delta} is the c.d.f. of \Delta . Then, applying the Edgeworth expansion in Lemma 3.1 yields the following proposition. Proposition 3.2 . If \alpha and 0 lie in regions where the Edgeworth expansion applies, then P_{\text{sh}}(\alpha)\approx\left[\Phi(\zeta_{\alpha})-\Phi(\zeta_{0})\right]-\frac{\gamma_{3}}{6\sqrt{N}}\left[\phi(\zeta_{\alpha})\tilde{P}_{1,\alpha}-\phi(\zeta_{0})\tilde{P}_{1,0}\right], (6) where \zeta_{x}=(x-\mu_{\Delta})/\sigma_{\Delta} , and \tilde{P}_{1,\alpha} , \tilde{P}_{1,0} are polynomial terms in \zeta_{\alpha} , \zeta_{0} . Proposition 3.2 implies that, for large N , P_{\text{sh}}(\alpha) depends on how \alpha compares with the mean \mu_{\Delta} . The margin \alpha effectively shifts the region (0,\alpha) and the leading correction to the normal approximation is on the order of 1/\sqrt{N} times a function of the skewness \gamma_{3} . We now combine the expansions to analyze the expected semi-hard triplet loss. \mathcal{L}_{\text{semi}}(\alpha)=\mathbb{E}\left[1_{\{0<\Delta<\alpha\}}\cdot\max(\alpha-\Delta,0)\right]=\int^{\alpha}_{0}(\alpha-t)f_{\Delta}(t)dt. (7) Here, f_{\Delta}(t) is the p.d.f. of \Delta . Theorem 3.3 . Under the assumptions in Lemma 3.1 , the semi-hard triplet loss in Eq. ( 7 ) admits the Edgeworth expansion, \displaystyle\mathcal{L}_{\text{semi}}(\alpha)=\mathcal{L}^{(0)}_{\alpha}+\frac{1}{\sqrt{N}}\mathcal{L}^{(1)}_{\alpha}+O(N^{-1}), with \displaystyle\mathcal{L}^{(0)}_{\alpha} \displaystyle=(\alpha-\mu_{\Delta})\left[\Phi_{\zeta_{\alpha}}-\Phi_{\zeta_{0}}\right]+\sigma_{\Delta}\left[\phi_{\zeta_{\alpha}}-\phi_{\zeta_{0}}\right], \displaystyle\mathcal{L}^{(1)}_{\alpha} \displaystyle=\frac{\gamma_{3}}{6}\left\{(\alpha-\mu_{\Delta})\left[\phi_{\zeta_{\alpha}}\zeta_{\alpha}^{2}-\phi_{\zeta_{0}}\zeta_{0}^{2}-\phi_{\zeta_{\alpha}}+\phi_{\zeta_{0}}\right]-\sigma_{\Delta}\left[\zeta_{\alpha}^{3}\phi_{\zeta_{\alpha}}-\zeta_{0}^{3}\phi_{\zeta_{0}}\right]\right\}, where f_{\mathcal{N}}(t) is the density of \mathcal{N}(\mu_{\Delta},\sigma_{\Delta}^{2}) , \Phi_{\zeta_{t}}=\Phi(\zeta_{t}) , \phi_{\zeta_{t}}=\phi(\zeta_{t}) and \zeta_{t}=(t-\mu_{\Delta})/\sigma_{\Delta} . Proof. Introduce the change of variable \zeta_{t}=(t-\mu_{\Delta})/\sigma_{\Delta} so that t=\mu_{\Delta}+\sigma_{\Delta}\zeta_{t} and dt=\sigma_{\Delta}d\zeta_{t} . Then, the loss becomes \displaystyle\mathcal{L}_{\text{semi}}(\alpha) \displaystyle=\int^{t=\alpha}_{t=0}(\alpha-t)f_{\Delta}(t)dt \displaystyle=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\left[\alpha-(\mu_{\Delta}+\sigma_{\Delta}\zeta_{t})\right]f_{\Delta}(\mu_{\Delta}+\sigma_{\Delta}\zeta_{t})\sigma_{\Delta}d\zeta_{t} \displaystyle=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\left[\alpha-\mu_{\Delta}-\sigma_{\Delta}\zeta_{t}\right]\phi(\zeta_{t})\left[1-\frac{\gamma_{3}}{6\sqrt{N}}(\zeta_{t}^{3}-3\zeta_{t})+O(N^{-1})\right]d\zeta_{t} \displaystyle=I_{0}-\frac{\gamma_{3}}{6\sqrt{N}}I_{1}+O(N^{-1}), where \displaystyle I_{0} \displaystyle\coloneqq\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\left[\alpha-\mu_{\Delta}-\sigma_{\Delta}\zeta_{t}\right]\phi(\zeta_{t})d\zeta_{t}, \displaystyle I_{1} \displaystyle\coloneqq\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\left[\alpha-\mu_{\Delta}-\sigma_{\Delta}\zeta_{t}\right](\zeta_{t}^{3}-3\zeta_{t})\phi(\zeta_{t})d\zeta_{t}. First, we consider to evaluate I_{0} as follows. \displaystyle I_{0} \displaystyle=(\alpha-\mu_{\Delta})\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\phi(\zeta_{t})d\zeta_{t}-\sigma_{\Delta}\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}\phi(\zeta_{t})d\zeta_{t} \displaystyle=(\alpha-\mu_{\Delta})\left[\Phi(\zeta_{\alpha})-\Phi(\zeta_{0})\right]+\sigma_{\Delta}\left[\phi(\zeta_{\alpha})-\phi(\zeta_{0})\right]. We next evaluate I_{1} . Let I_{1}=(\alpha-\mu_{\Delta})J_{1}-\sigma_{\Delta}J_{2} , with \displaystyle J_{1} \displaystyle=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}(\zeta_{t}^{3}-3\zeta_{t})\phi(\zeta_{t})d\zeta_{t}, \displaystyle J_{2} \displaystyle=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}(\zeta_{t}^{3}-3\zeta_{t})\phi(\zeta_{t})d\zeta_{t}=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}^{4}-3\zeta_{t}^{2}\phi(\zeta_{t})d\zeta_{t}. Here, \displaystyle J_{1} \displaystyle=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}^{3}\phi(\zeta_{t})d\zeta_{t}-3\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}\phi(\zeta_{t})d\zeta_{t} \displaystyle=\Big[-\phi(\zeta_{t})(\zeta_{t}^{2}+2)\Big]^{\zeta_{\alpha}}_{\zeta_{0}}-3\Big[-\phi(\zeta_{t})\Big]^{\zeta_{\alpha}}_{\zeta_{0}} \displaystyle=\left\{-\phi(\zeta_{\alpha})(\zeta_{\alpha}^{2}+2)+\phi(\zeta_{0})(\zeta_{0}^{2}+2)\right\}+3\left[\phi(\zeta_{\alpha})-\phi(\zeta_{0})\right] \displaystyle=-\phi(\zeta_{\alpha})\zeta_{\alpha}^{2}+\phi(\zeta_{0})\zeta_{0}^{2}+\phi(\zeta_{\alpha})-\phi(\zeta_{0}), and \displaystyle J_{2} \displaystyle=\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}^{4}\phi(\zeta_{t})d\zeta_{t}-3\int^{\zeta_{t}=\zeta_{\alpha}}_{\zeta_{t}=\zeta_{0}}\zeta_{t}^{2}\phi(\zeta_{t})d\zeta_{t} \displaystyle=\Big[-\zeta_{\alpha}^{3}\phi(\zeta_{\alpha})-3\zeta_{\alpha}\phi(\zeta_{\alpha})+3\Phi(\zeta_{\alpha})-\Big(-\zeta_{0}^{3}\phi(\zeta_{0})-3\zeta_{0}\phi(\zeta_{0})+3\Phi(\zeta_{0})\Big)\Big] \displaystyle\quad\quad\quad-3\Big\{\Big[-\zeta_{\alpha}\phi(\zeta_{\alpha})+\Phi(\zeta_{\alpha})\Big]-\Big[-\zeta_{0}\phi(\zeta_{0})+\Phi(\zeta_{0})\Big]\Big\} \displaystyle=\Big[-\zeta_{\alpha}^{3}\phi(\zeta_{\alpha})-3\zeta_{\alpha}\phi(\zeta_{\alpha})+3\Phi(\zeta_{\alpha})\zeta_{0}^{3}\phi(\zeta_{0})+3\zeta_{0}\phi(\zeta_{0})-3\Phi(\zeta_{0})\Big] \displaystyle\quad\quad\quad+3\Big\{\zeta_{\alpha}\phi(\zeta_{\alpha})-\Phi(\zeta_{\alpha})-\zeta_{0}\phi(\zeta_{0})+\Phi(\zeta_{0})\Big\}=-\zeta_{\alpha}^{3}\phi_{\zeta_{\alpha}}+\zeta_{0}^{3}\phi_{\zeta_{0}}. Thus, \displaystyle I_{1} \displaystyle=(\alpha-\mu_{\Delta})\Big[-\phi_{\zeta_{\alpha}}\zeta_{\alpha}^{2}+\phi_{\zeta_{0}}\zeta_{0}^{2}+\phi_{\zeta_{\alpha}}-\phi_{\zeta_{0}}\Big]-\sigma_{\Delta}\Big[-\zeta_{\alpha}^{3}\phi_{\zeta_{\alpha}}+\zeta_{0}^{3}\phi_{\zeta_{0}}\Big], and we obtain the final expansion. ∎ Corollary 3.4 . Under the assumptions of Theorem 3.3 , if the distance difference \Delta is symmetrically distributed about its mean, then the first-order Edgeworth correction vanishes. Corollary 3.5 (Margin Sensitivity of the Semi-hard Triplet Loss) . Under the assumptions of Theorem 3.3 , note that the semi-hard triplet loss satisfies \frac{d\mathcal{L}_{\text{semi}}(\alpha)}{d\alpha}=\int_{0}^{\alpha}f_{\Delta}(t)\,dt=P_{\text{sh}}(\alpha)=\mathbb{P}(0<\Delta<\alpha). (8) Employing the Edgeworth expansion from Proposition 3.2 , we have \frac{d\mathcal{L}_{\text{semi}}(\alpha)}{d\alpha}\approx\left[\Phi\Big(\frac{\alpha-\mu_{\Delta}}{\sigma_{\Delta}}\Big)-\Phi\Big(\frac{-\mu_{\Delta}}{\sigma_{\Delta}}\Big)\right]-\frac{\gamma_{3}}{6\sqrt{N}}\left[\phi\Big(\frac{\alpha-\mu_{\Delta}}{\sigma_{\Delta}}\Big)\tilde{P}_{1,\alpha}-\phi\Big(\frac{-\mu_{\Delta}}{\sigma_{\Delta}}\Big)\tilde{P}_{1,0}\right]+O(N^{-1}). (9) In particular, when the margin is set equal to the mean, the sensitivity becomes \left.\frac{d\mathcal{L}_{\text{semi}}(\alpha)}{d\alpha}\right|_{\alpha=\mu_{\Delta}}\approx\frac{1}{2}-\Phi\Big(\frac{-\mu_{\Delta}}{\sigma_{\Delta}}\Big)-\frac{\gamma_{3}}{6\sqrt{N}}\left[\phi(0)\tilde{P}_{1,\mu_{\Delta}}-\phi\Big(\frac{-\mu_{\Delta}}{\sigma_{\Delta}}\Big)\tilde{P}_{1,0}\right]+O(N^{-1}). (10) This expression quantifies how small variations in \alpha impact the loss via the probability mass in the semi-hard region. Consequently, it provides a direct criterion for tuning \alpha to balance training stability and effective mining of informative triplets. Finally, following statements provide the uniform error bound for

Read the complete text