Signature Maximum Mean Discrepancy Two-Sample Statistical Tests
Andrew Alden, Blanka Horvath, Zacharia Issa · 2025
Read Signature Maximum Mean Discrepancy Two-Sample Statistical Tests by Andrew Alden, Blanka Horvath, Zacharia Issa (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-01718.txt.
Introduction
Andrew Alden, Blanka Horvath, and Zacharia Issa explore, in a 2025 paper, a statistical test that decides whether two batches of paths come from the same random process. The test builds on maximum mean discrepancy, a kernel method for comparing distributions, extended to whole paths through the signature kernel.
The authors explain the signature version in accessible terms and give checkable examples of its use. They also expose a weakness: with limited data, the test can miss real differences, a type 2 error that falsely declares a match. Practical remedies are then presented to cut that risk. The work targets finance and other fields awash in path data.
Catalog record
| CREATOR | Andrew Alden, Blanka Horvath, Zacharia Issa |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 74 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-01718
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Signature Maximum Mean Discrepancy Two-Sample Statistical Tests Andrew Alden, Blanka Horvath, Zacharia Issa Abstract Maximum Mean Discrepancy (MMD) is a widely used concept in machine learning research which has gained popularity in recent years as a highly effective tool for comparing (finite-dimensional) distributions. Since it is designed as a kernel-based method, the MMD can be extended to path space valued distributions using the signature kernel. The resulting signature MMD (sig-MMD) can be used to define a metric between distributions on path space. Similarly to the original use case of the MMD as a test statistic within a two-sample testing framework, the sig-MMD can be applied to determine if two sets of paths are drawn from the same stochastic process. This work is dedicated to understanding the possibilities and challenges associated with applying the sig-MMD as a statistical tool in practice. We introduce and explain the sig-MMD, and provide easily accessible and verifiable examples for its practical use. We present examples that can lead to Type 2 errors in the hypothesis test, falsely indicating that samples have been drawn from the same underlying process (which generally occurs in a limited data setting). We then present techniques to mitigate the occurrence of this type of error. 1 Introduction The seminal paper A Kernel Two-Sample Test [ 21 ] lays the foundation for establishing a versatile and robust framework for analysing and comparing distributions. This framework has been assessed and benchmarked against established classical two-sample tests in the finite-dimensional setting (such as Kolmogorov-Smirnov test) where it demonstrated superior or comparable performance [ 13 , 20 , 21 ] . It has also been shown to be well-suited for machine learning (ML) applications [ 3 , 11 , 29 , 34 , 50 ] . This framework is therefore particularly well-known for its applications to analysing finite-dimensional distributions in a ML context. The test statistic which underpins this framework is based on computing the largest average difference between distributions, over functions in the unit ball of a Reproducing Kernel Hilbert Space (RKHS). This concept has been named the kernel Maximum Mean Discrepancy (MMD) [ 21 ] . On an appropriate (rich enough) space, the kernel MMD determines a metric between probability distributions. Using a signature kernel ( k_{\text{Sig}} ) (cf [ 9 , 13 , 31 , 40 ] ), the aforementioned MMD can be established for measures on path space defined over the set \mathcal{X}_{1-\text{var}} . Similar to the originally proposed framework of the MMD (cf [ 21 ] ), its path-valued counterpart (sig-MMD) can also be applied as a test statistic in a two-sample hypothesis test to determine if two collections of time series are drawn from different stochastic processes. If \mathbf{X} and \mathbf{Y} are two stochastic processes with laws \mathbb{P}_{\mathbf{X}}=\mathbb{P}\circ\mathbf{X}^{-1} and \mathbb{P}_{\mathbf{Y}}=\mathbb{P}\circ\mathbf{Y}^{-1} respectively, the two-sample test is described by the following null hypothesis ( H_{0} ) and alternative hypothesis ( H_{1} ): H_{0}\colon\mathbb{P}_{\mathbf{X}}=\mathbb{P}_{\mathbf{Y}}~\text{against}~H_{1}\colon\mathbb{P}_{\mathbf{X}}\neq\mathbb{P}_{\mathbf{Y}}. The sig-MMD has facilitated the use of distribution regression on path space [ 3 , 28 , 41 ] and the training of generative models for time series [ 8 , 29 ] . Since the path signature is infinite dimensional, the extension of these concepts to path space using the (full) signature within numerical algorithms is non-trivial. Recent literature focuses on the numerical computation of the signature kernel. These include (i) truncating the signature [ 31 ] , (ii) using a kernel trick by means of an associated partial differential equation (PDE) to work with the full signature [ 40 , 41 ] , and (iii) using random Fourier features [ 48 , 39 ] . This work aims to demonstrate how to use the sig-MMD in practice with specific focus on effectively navigating the challenges that may emerge in this context. We give practical insights for setting up the associated ML algorithms that can save the interested user considerable time when implementing these tools. We also suggest hyperparameter settings and discuss common pitfalls. We provide simple and easy-to-verify examples together with explanatory discussions which help build intuition surrounding these tools. As mentioned in [ 33 ] , in practice, the numerical approximation of the signature kernel is based on various hyperparameters. We study the statistical power of the two-sample hypothesis test as a function of the following subset of hyperparameters: • Static Kernel Parameters \Theta_{k} : Instead of working directly with the original paths, the paths can be embedded in an infinite dimensional feature space using another kernel [ 31 , 40 ] . • Truncation Parameter \mathbb{N}_{\infty} : Since the signature of a path is an infinite dimensional object, it is not possible to work with the full signature. To work directly with the signature, one must truncate the signature by using the first M\in\mathbb{N} terms of the signature. However, when using kernel methods, a kernel trick can be used to work with the full signature. In this case, computing the signature kernel amounts to solving a PDE [ 41 ] . Therefore, when using the signature, one works with either an imperfect representation of the signature through its truncated version or the full signature by way of the kernel trick. • Preprocessing Parameters \Theta_{\text{preprocess}} : Preprocessing the paths by using lead-lag transformations [ 12 , 18 , 27 , 46 ] , adding a time-component [ 12 , 46 ] , and by standardising dimensions 1 1 1 If the path is multi-dimensional, this involves standardising the individual dimensions. This could be done by subtracting the mean to have each dimension centred around 0. [ 29 ] can be performed. • Normalisation Parameters \Theta_{\text{normalisation}} : Normalising the signature kernel results in a more robust statistic [ 13 ] . Moreover, by rescaling inner products, a generalised form of the signature kernel is obtained through which further signature MMDs can be constructed [ 9 , 10 ] . • Time Discretisation Parameters \Theta_{\text{discrete}} : In practice, computations are performed on discrete data. The discretisation of the time interval can sometimes be controlled. Finer grids provide better approximations of the stochastic process [ 14 ] whilst also increasing the computational burden. In addition, if working with the full signature through the PDE approach, the PDE is solved on a discrete grid which itself is controlled through a hyperparameter. The following sections are aimed at quantifying and interpreting the behaviour of the sig-MMD test statistic in the context of the two-sample hypothesis test. This behaviour is mostly characterised by the above parameters and therefore a discussion of the role of these parameters in improving performance on the hypothesis test is provided. The variance of the sig-MMD plays a significant role in characterising this behaviour. This variance is closely linked with the number of sample paths used. Although this can sometimes be directly addressed through sampling more samples from the processes and/or collecting additional data if the paths correspond to a physical system, one cannot disregard the computational bottlenecks associated with the sig-MMD. Therefore, increasing the sample size is generally not a feasible solution. We provide insights into stabilising the errors associated with the sig-MMD in the two-sample hypothesis testing framework in a high variance setting. Specifically, we focus on the following: 1. Hypothesis Testing Errors Associated with