On Uncertainty Calibration for Equivariant Functions

Edward Berman [email protected], Jacob Ginesin [email protected], Marco Pacini [email protected], Robin Walters [email protected] · 2025

Read On Uncertainty Calibration for Equivariant Functions by Edward Berman [email protected], Jacob Ginesin [email protected], Marco Pacini [email protected], Robin Walters [email protected] (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-21691.txt.

Introduction

Edward Berman and three co-authors ask whether symmetry-respecting models know what they don't know. Equivariant networks shine where data are scarce, from robot hands to molecules to galaxy shapes. Honest uncertainty estimates guard against their overconfidence, yet the link between built-in symmetry and calibration had never been charted.

Their theory brackets the calibration error of equivariant models from both sides under several symmetry conditions, for classification and regression alike. The bounds expose a failure mode: when the model's assumed symmetry mismatches the data's true symmetry, confidence systematically skews, and the size of the group tunes the effect. Experiments on real and simulated data confirm the predicted patterns, including how random and systematic uncertainties interact with the mismatch, giving practitioners a map of when to trust an equivariant model's self-assessment.

Catalog record

CREATOREdward Berman [email protected], Jacob Ginesin [email protected], Marco Pacini [email protected], Robin Walters [email protected]
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE116 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-21691

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On Uncertainty Calibration for Equivariant Functions Edward Berman [email protected], Jacob Ginesin [email protected], Marco Pacini [email protected], Robin Walters [email protected] Abstract Data-sparse settings such as robotic manipulation, molecular physics, and galaxy morphology classification are some of the hardest domains for deep learning. For these problems, equivariant networks can help improve modeling across undersampled parts of the input space, and uncertainty estimation can guard against overconfidence. However, until now, the relationships between equivariance and model confidence, and more generally equivariance and model calibration, has yet to be studied. Since traditional classification and regression error terms show up in the definitions of calibration error, it is natural to suspect that previous work can be used to help understand the relationship between equivariance and calibration error. In this work, we present a theory relating equivariance to uncertainty estimation. By proving lower and upper bounds on uncertainty calibration errors (ECE and ENCE) under various equivariance conditions, we elucidate the generalization limits of equivariant models and illustrate how symmetry mismatch can result in miscalibration in both classification and regression. We complement our theoretical framework with numerical experiments that clarify the relationship between equivariance and uncertainty using a variety of real and simulated datasets, and we comment on trends with symmetry mismatch, group size, and aleatoric and epistemic uncertainties. 1 Introduction Equivariant neural networks are a class of neural networks that encode group symmetries into the structure of the network architecture so that the symmetries do not need to be learned from data. Understanding both model calibration and confidence is particularly useful in the data-sparse settings where equivariant neural networks tend to thrive, such as pick-and-place robotics tasks ( Kalashnikov et al. 2018 ; Wang et al. 2022b ; Wang et al. 2022a ; Fu et al. 2023 ; Huang et al. 2023 ; Huang et al. 2024b ; Huang et al. 2024a ; Wang 2025 ) , galaxy morphology classification ( Pandya et al. 2023 ; Pandya et al. 2025a ) , and molecular physics ( Zou et al. 2023 ; Ramakrishnan et al. 2014 ) . While equivariance has proved effective in these domains, it does have some drawbacks, including diminishing benefits at scale as in the case of extremely large datasets or aggressive augmentation ( Wang et al. 2023b ; Klee et al. 2023 ; Gruver et al. 2023 ; Brehmer et al. 2024 ; Abramson et al. 2024 ) , provable degradation on model performance in cases of symmetry mismatch ( Wang et al. 2024 ) , more complex architectures, and higher compute costs. Despite these drawbacks, a surprising result of Wang et al. 2023a is that equivariant neural networks can still be effective even in cases of mismatch between the model and the data symmetry. This finding motivated the work of Wang et al. 2024 , which explored how equivariance can affect model accuracy , both positively and negatively. However, it is not yet understood how equivariance impacts model calibration , loosely defined as the disagreement between a model’s accuracy and predicted confidence . To help understand the tradeoffs associated with equivariance, we seek to quantify the impact of equivariance on different metrics, namely expected calibration error for classification and expected normalized calibration error for regression. The only works that directly examine the link between equivariance and calibration are Sun et al. 2023 and Cherif et al. 2024 , although these are purely experimental and provide only a few illustrative cases where equivariance influences model calibration. Despite this, because different notions of calibration error involve expressions corresponding to classification or regression errors themselves, previous results on the generalization limits of equivariant models can be applied to the study of calibration error. Understanding this relationship allows us to address several unanswered questions on the subject of calibration and confidence of equivariant models. Namely, when does equivariance help a model predict its own confidence? How do notions of symmetry mismatch between model and data affect model calibration? What is the general relationship between equivariance and uncertainty estimation? Beyond raw performance, answering these questions is crucial for developing reliable symmetry-aware models in safety-critical and high-stakes applications. The purpose of this work is to address both the lack of a theory relating equivariance to uncertainty estimation, and the scarcity of experiments exploring this relationship in practice. To accomplish this, we extend the error bounds given by Wang et al. 2024 to a broader class of calibration losses. In this way, we can quantify the effect of equivariance not just on accuracy, but also on calibration. In particular, we show that calibration error is related to typical classification and regression errors over the fibers of each confidence prediction. These errors have known bounds for equivariant functions, which we use to provide lower and upper bounds on various calibration error metrics. We explore these threads in the context of correct, incorrect, and extrinsic ( Wang et al. 2024 ) equivariance (the equivariance taxonomy hereafter), where correct means the ground truth function has the same symmetry as the equivariant model, incorrect means the ground truth function disagrees with the symmetry in the model, and extrinsic means the model symmetry transforms in-distribution data to out-of-distribution data. We also coin a metric, the aleatoric bleed, that prescribes how well a model is able to predict ground truth uncertainties themselves and lower bound it for equivariant models. This study illustrates that the effect of equivariance on model calibration is dependant on where we are in the equivariance taxonomy, supporting the previous line of work in Wang et al. 2023a ; Wang et al. 2024 . We supplement our bounds with experiments on a wide variety of real and simulated datasets that indicate how models can suffer from increased calibration error in instances of symmetry mismatch. We summarize our contributions as follows: 1. For the classification task , we provide bounds on calibration error and we tighten the bound in the limiting case of an invariance ( Section 4 ). 2. For the regression task , we generalize expected normalized calibration error beyond scalar values for mean and variance predictions. We derive its upper bound on certain equivariant models ( Section 5 ) and derive its lower bound in the special case of scalar-valued mean and variance predictions. 3. Additionally, we coin another metric, the aleatoric bleed , in order to study miscalibration in terms of aleatoric and epistemic uncertainty ( Section 6 ). 4. We provide illustrative examples and run numerical experiments on diverse real and simulated datasets. We find model calibration and aleatoric bleed suffer in cases of symmetry mismatch, and show our bounds provide a useful way of assessing relative model calibration errors. 2 Related Work Equivariant Learning. Our work is closest thematically to Petrache & Trivedi 2023 and Wang et al. 2024 , which both establish bounds on function generalization under various assumptions of symmetry mismatch, but neither study calibration errors specifically. We also consider the equivaraince taxonomy established in Wang et al. 2024 , but depict its ramifications on uncertainty quantification. Throughout this paper, we assume access to universal G -equivariant models, guaranteed by prior work ( Maron et al. 2019 ; Yarotsky 2022 ) . Yet, universality alone does not ensure reliable uncertainty estimates, and therefore our focus

Read the complete text