Certification of Machine Learning Models via Directional Sharpness

Gefei Tan, Adrià Gascón, Sarah Meiklejohn, Mariana Raykova · 2026

Read Certification of Machine Learning Models via Directional Sharpness by Gefei Tan, Adrià Gascón, Sarah Meiklejohn, Mariana Raykova (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25004.txt.

Introduction

Gefei Tan, Adrià Gascón, Sarah Meiklejohn, and Mariana Raykova propose directional sharpness as a certificate of machine learning model quality. Generalisation itself cannot be certified directly: it depends on unseen data. Proxies disappoint. Test accuracy misleads when training was perturbed, deliberately or by accident. Sharpness, a curvature measure with an empirical link to generalisation, is costly to compute and can fail when training deviates from the prescribed procedure. Directional sharpness is designed to stay reliable and cheap under such changes.

Evidence from experiments and proofs supports two claims. It tracks generalisation more strongly than existing metrics. It also flags weak models more reliably. An auditor holding the training data can evaluate it quickly. Zero knowledge proofs extend the reach, certifying quality without revealing the training data at all. Tan and colleagues position the metric for real auditing pipelines.

Catalog record

CREATORGefei Tan, Adrià Gascón, Sarah Meiklejohn, Mariana Raykova
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE81 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.25004

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Certification of Machine Learning Models via Directional Sharpness Gefei Tan, Adrià Gascón, Sarah Meiklejohn, Mariana Raykova Abstract In machine learning, model certification has been identified as an important method for gaining assurance about a model’s trustworthiness and quality. A model’s quality is largely determined by its ability to generalize, i.e., to perform well on data beyond what it was trained on. It is not possible to certify generalization directly, however, as it depends on unknown data and is not directly measurable. Proxies such as test accuracy can be misleading when the training process is perturbed (intentionally or accidentally), and metrics such as sharpness —which has an empirically supported link to generalization—are computationally expensive and can also serve as unreliable signals when training deviates from a prescribed procedure. In this work, we propose directional sharpness , a metric designed to efficiently and reliably indicate generalization despite potential training deviations. We provide empirical and analytical evidence that directional sharpness (1) correlates more strongly with generalization than existing metrics and (2) identifies models with poor generalization more reliably than existing metrics. Furthermore, directional sharpness is efficiently computable in model auditing settings, where the verifier has access to training data, and via zero-knowledge proofs that certify quality without revealing training data. 1 Introduction There are many properties of machine learning (ML) models that model providers and external stakeholders may wish to verify before those models are deployed. Arguably, the most universal property for such model certification is the quality of the model, in terms of its performance on downstream tasks. A natural and widely accepted notion of model quality is generalization [ 2 , 40 ] : the ability of the model to perform well on new, unseen data. In practice, model quality is typically estimated using a holdout test set, and a model is said to “generalize well” if it achieves high test accuracy. Test accuracy is not a reliable metric, however, as test sets can contain errors [ 64 ] , fail to detect overfitting [ 19 , 31 , 35 ] , or miss real-world distribution shifts [ 68 , 48 , 8 , 78 ] . In adversarial settings, research has shown that a model can achieve high accuracy on test data while being vulnerable to adversarial inputs [ 32 ] or containing backdoors that activate only on specific triggers [ 34 , 54 , 28 ] . In statistical learning theory, generalization is captured by the gap between the empirical risk on the training set and population risk on fresh samples from the same (unknown) data distribution. Ideally, a certificate of generalization would show that this gap is small. However, the true data distribution is unknown, and thus generalization cannot be measured directly. Researchers have instead focused on generalization bounds: upper bounds on this gap computable from the training set and model [ 2 , 83 , 58 , 59 , 40 , 62 ] . While theoretically appealing, existing bounds are often numerically vacuous or too loose to track actual performance for modern, over-parameterized neural networks, rendering them unsuitable for certification. These gaps have motivated empirically grounded generalization metrics : quantities computable from the learned model and training data that reliably predict generalization. Such metrics can serve as a principled proxy for generalization without relying on a specific holdout test set. Recently, the sharpness of the loss landscape has emerged as a compelling generalization metric, with empirical and theoretical evidence that models in wide “flat” basins generalize better than those in narrow “sharp” minima [ 45 , 43 , 67 , 36 , 14 ] . One particularly effective approach measures the sensitivity of training loss to small parameter perturbations [ 25 , 50 , 45 , 67 , 21 ] . The seminal work of Foret et al. [ 25 ] introduces sharpness-aware minimization (SAM), an optimizer that explicitly minimizes the model’s sharpness. SAM has been shown to significantly improve generalization [ 25 , 50 , 14 ] , robustness to noisy labels [ 6 ] , and adversarial robustness [ 90 ] across various benchmarks and model architectures. SAM training is also associated with improved resistance to membership inference attacks [ 56 ] , backdoor defense [ 100 ] , and improvements in machine unlearning [ 80 , 22 ] . These results provide strong evidence that perturbation-based sharpness is a promising proxy for model generalization. Despite this explanatory power, existing sharpness metrics have not been explored in the context of certification, likely due to two fundamental barriers. First, these metrics have been validated only via correlation with test accuracy on models trained honestly on clean data. Given that test accuracy itself can fail to detect poor generalization, it remains unclear whether or not sharpness can reliably predict generalization under (intentional or unintentional) training deviations. Second, existing metrics incur prohibitive computational cost even with direct access to the model and training data, let alone the overhead required in a setting where an external stakeholder would like to verify (without access to the training data) a cryptographic proof computed by the model provider. This motivates our goal: to design a generalization metric that is strongly predictive of generalization, remains informative under training deviations, and is efficient enough to support privacy-preserving verification. Our results. In this paper, we introduce directional sharpness , a generalization metric designed specifically for model certification. Unlike static sharpness, which measures the loss landscape geometry at a single point, directional sharpness measures dynamic stability : the extent to which a model remains in a flat region under continuous, stochastic perturbation. We show that directional sharpness (1) outperforms existing metrics in terms of empirical correlation with generalization; (2) remains predictive of generalization even under training deviations, making it more sensitive in detecting poor generalization than existing measures; and (3) is significantly more efficient to compute. Directional sharpness thus provides an efficient and robust metric for evaluating generalization when the verifier has access to the training data. We also show how to extend this metric to the setting where the verifier does not have access to the training dataset, via zero-knowledge proofs for directional sharpness. 1.1 Technical Overview We now outline the key ideas behind directional sharpness. We first explain the shift from static to dynamic measurement, then discuss its theoretical grounding, and finally present empirical validation showing the metric is efficient and enables practical certification. From static sharpness to dynamic stability. Our starting point is perturbation-based sharpness, which shows strong empirical correlation with generalization in honestly trained models [ 43 ] . However, certification demands more than correlation in benign settings: a metric must detect low-quality models that are adversarially chosen to evade detection. We identify a fundamental limitation of existing sharpness metrics: they are static , and evaluate a single, averaged perturbation around the final model to measure sharpness. This one-shot view can miss two failure modes that matter for certification: (i) incoherent per-example geometry , where loss landscapes are sharp only on data subsets but appear flat when averaged over the full dataset, and (ii) unstable flat minima , where a model sits in an apparently flat, narrow pocket adjacent to sharp regions that a single perturbation may not reliably detect. Both failure

Read the complete text