Statistical Validation of Computer Models: Global and Subdomain Hypothesis Testing

Chaoan Li, Xianyang Zhang, Rui Tuo · 2026

Read Statistical Validation of Computer Models: Global and Subdomain Hypothesis Testing by Chaoan Li, Xianyang Zhang, Rui Tuo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17144.txt.

Introduction

Li, Zhang, and Tuo develop a frequentist scheme for validating computer models overall and in parts. Their FMMT estimates the gap between model and physical process with kernel ridge regression, then tests weighted Fourier coefficients by frequency. Theory gives normal limits for the coefficients, hence direct p-values, and tests on simulations and a shear-layer test show high power, tight error control, and keen local response.

Catalog record

CREATORChaoan Li, Xianyang Zhang, Rui Tuo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE55673
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.17144

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Statistical Validation of Computer Models: Global and Subdomain Hypothesis Testing Chaoan Li, Xianyang Zhang, Rui Tuo Abstract Computer simulations play an important role in scientific discovery and engineering innovation. Reliable computer models enable virtual experimentation that reduces the need for costly and time-consuming physical testing. However, the credibility of such models hinges on rigorous statistical validation against real-world data. This paper develops a formal frequentist framework for both global and subdomain validation of computer models. We propose the Fourier Maximum Modulus Test (FMMT), which leverages kernel ridge regression (KRR) to estimate the discrepancy between the computer model and the physical process, followed by a frequency-domain test based on weighted generalized Fourier coefficients. The theoretical analysis establishes the asymptotic normality of these coefficients, allowing for closed-form p-values. Simulation studies and a shear-layer experiment demonstrate that FMMT achieves high power, accurate Type I error control, and strong sensitivity to localized discrepancies. Keywords: Function comparison, Uncertainty quantification, Computer experiments 1 Introduction Computer simulations are central to modern scientific discovery and engineering design, enabling virtual experimentation that reduces reliance on costly physical testing. Statistical validation—rigorously comparing simulation outputs with real-world data—is essential for establishing model credibility and identifying limitations before deployment. A computer model is by nature an approximation of the physical system it represents. Statistical validation asks whether this approximation is consistent with observations of the physical process: formally, whether any systematic discrepancy between the model output y^{s}(x) and the true response function f(x) is detectable over the input space \Omega . Validation methods span both frequentist and Bayesian frameworks. Frequentist approaches include Hotelling’s T^{2} test ( 1 ) , residual-based diagnostics ( 10 ) , engineering frameworks for computational fluid dynamics ( 26 , 25 , 24 ) , and likelihood- or norm-based metrics ( 11 , 12 ) ; these quantify aggregate error but lack formal function-level hypothesis tests. Bayesian approaches model outputs as stochastic processes ( 33 ) and incorporate calibration via explicit discrepancy terms ( 16 , 17 , 45 ) , but do not target formal testing with controlled error rates. This work fills this gap by formulating the null hypothesis as a functional statement: the computer model exhibits no statistically detectable structural discrepancy from the physical process over the entire input domain—going beyond conventional tests on finite-dimensional output vectors. Additionally, we propose subdomain validation on user-specified subregions: a model may perform adequately globally while failing locally, and subdomain testing identifies such deficiencies to guide targeted refinement. Our framework naturally accommodates calibration parameters. When the simulator takes the form y^{s}(x,\theta) with \theta estimated from data, the calibrated estimator satisfies \hat{\theta}-\theta^{*}=O_{p}(n^{-1/2}) . Since calibration occurs at the parametric \sqrt{n} rate, the plug-in error is asymptotically negligible relative to the nonparametric KRR estimation error, and the asymptotic results in Theorem 4.1 remain valid for the calibrated simulator y^{s}(x,\hat{\theta}) . We propose the Fourier maximum modulus test (FMMT) for global and subdomain validation. FMMT uses KRR to estimate the discrepancy, constructs test statistics from generalized Fourier coefficients of the weighted estimated discrepancy, and derives p -values from their joint asymptotic normality. The remainder of this paper is organized as follows. Section 2 formulates the testing problems and reviews related literature; Section 3 presents FMMT; Section 4 provides theoretical justifications; Section 5 reports simulation studies; Section 6 presents a shear-layer application; and Section 7 concludes. 2 Problem Formulation Let \Omega\subset\mathbb{R}^{d} be convex and compact. Let y^{s}:\Omega\rightarrow\mathbb{R} be the computer model output, or a surrogate model ( 34 , 6 ) when the simulator is expensive; we treat y^{s} as known. Note that y^{s} may also represent a calibrated model y^{s}(x,\hat{\theta}) as discussed in Section 1 . The physical data (x_{i},y_{i}) , i=1,\ldots,n , satisfy x_{i}\stackrel{{\scriptstyle\text{i.i.d.}}}{{\sim}}p(x) on \Omega . The input-output relationship of the field data follows the nonparametric regression model y_{i}=f(x_{i})+e_{i}, where f is a continuous function called the physical process and is the objective function that the computer model is intended to reproduce; e_{i} ’s are i.i.d. random errors with zero mean and variance \sigma^{2} . This work focuses on statistical validation of computer models, with the goal of determining whether the computer response function differs from the physical process in a detectable manner. We formulate the validation process as a hypothesis testing problem: H_{0}:f(x)=y^{s}(x)\text{ for all }x\in\Omega\leftrightarrow H_{1}:f(x)\neq y^{s}(x)\text{ for some }x\in\Omega. (1) We call this process the global validation . In addition, we introduce the subdomain validation , which aims at identifying the subregions where the computer response and the physical process differ. We partition \Omega into N disjoint subdomains as \Omega=\bigcup_{i=1}^{N}\Omega_{i} . Each \Omega_{i} should have a positive volume. Then the corresponding subdomain validation with respect to \Omega_{i} is formulated as H_{0i}:f(x)=y^{s}(x)\text{ for all }x\in\Omega_{i}\leftrightarrow H_{1i}:f(x)\neq y^{s}(x)\text{ for some }x\in\Omega_{i}. (2) Now the subdomain validation is formulated as the multiple testing problem for the null hypotheses H_{0i} ’s. 2.1 Literature review on function comparisons The problem of testing equality of two regression functions has been studied extensively; see 5 for a survey. Smoothing-based approaches ( 7 , 19 , 9 ) laid the groundwork, extended by 20 to unequal design points. Residual-based methods ( 23 , 28 ) achieved optimal \sqrt{n} detection rates via resampling calibration. Characteristic-function tests ( 27 , 2 ) offer simplified calibration. These methods address the two-sample problem, where neither f_{1} nor f_{2} is known a priori; their settings differ from direct testing of regression function equality over arbitrary subdomains. A distinct literature addresses the one-sample goodness-of-fit setting, where the null model is fully specified. 4 proposed an order-selection test whose critical value depends only on the target level \alpha , avoiding smoothing-parameter tuning, but whose calibration assumes uniform fixed design points; the procedure does not extend to multivariate inputs or subdomain inference. The present work falls in this one-sample category: y^{s} is treated as fully specified, and the objective is to test for functional discrepancy over the input domain or over prescribed subdomains. Recent work has extended to local testing: density-based ( 3 ) , regression-based ( 18 ) , and distance-based ( 48 ) approaches provide strong theoretical guarantees in related settings. Applied methods include Gaussian process diagnostics for wind turbine power curves ( 30 ) and subdomain quantile-curve tests for surface finishing ( 14 ) , though these lack theoretical guarantees. 3 Proposed Methodology The method rests on Theorem 4.1 , which establishes asymptotic normality of weighted generalized Fourier coefficients of KRR; the nonparametric regression step is introduced in Section 3.1 . 3.1 Nonparametric regression We propose to use KRR to reconstruct f , defined as \hat{f}_{n}:=\operatorname*{argmin}_{v\in\mathcal{H}}\frac{1}{n}\sum_{i=1}^{n}(y_{i}-v(x_{i}))^{2}+\lambda_{n}\|v\|^{2}_{\mathcal{H}}, (3) where \mathcal{H} denotes a reproducing kernel Hilbert space, \lambda_{n} is the smoothing parameter. In this work, we choose the reproducing kernel to be an isotropic Matérn kernel ( 37 , 31 ) . K(x_{1},x_{2};\nu,\theta)=\frac{2^{1-\nu}}{\Gamma(\nu)}\left(\frac{\sqrt{2\nu}\|x_{1}-x_{2}\|}{\theta}\right)^{\nu}K_{\nu}\left(\frac{\sqrt{2\nu}\|x_{1}-x_{2}\|}{\theta}\right), (4) where \nu>0 is the smoothness parameter, \theta>0 is the scale parameter, K_{\nu} denotes the modified Bessel function of the second kind, and \|\cdot\| denotes the Euclidean norm. In view of the representer theorem ( 35 , 44 ) , \hat{f}_{n} in eq. 3 admits the explicit expression \hat{f}_{n}(x)=K(x,X)(K(X,X)+\lambda_{n}nI)^{-1}Y, with K(A,B):=(K(a_{i},b_{j};\nu,\theta))_{ij} for any sets of input points A=(a_{1},\ldots,a_{m})^{T} and B=(b_{1},\ldots,b_{l})^{T} , and Y=(y_{1},\ldots,y_{n})^{T} . To use Theorem 4.1 , \lambda_{n} should be chosen to satisfy certain bounds in its

Read the complete text