What should an AI assessor optimise for?

Daniel Romero-Alvarado, Fernando Martínez-Plumed, José Hernández-Orallo · 2025

Read What should an AI assessor optimise for? by Daniel Romero-Alvarado, Fernando Martínez-Plumed, José Hernández-Orallo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-00365.txt.

Introduction

Daniel Romero-Alvarado, Fernando Martínez-Plumed, and José Hernández-Orallo asked in 2025 a sly question about AI assessors, the models hired to predict how other AI systems will score. Should an assessor train on the exact target metric, or on a different one and convert afterward?

Across twenty tabular regression and classification problems, the intuitive answer lost. Training for a more informative metric was not generally better. Some sideways choices shone: logistic loss helped minimize plain and squared errors in regression, and the logarithmic score helped maximize quadratic and spherical scores in classification. Certain monotone remappings of predictions proved promising. The authors say the mapping step is a design lever, not an afterthought. Assessor builders should test it rather than assume it.

Catalog record

CREATORDaniel Romero-Alvarado, Fernando Martínez-Plumed, José Hernández-Orallo
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-00365

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

What should an AI assessor optimise for? Daniel Romero-Alvarado, Fernando Martínez-Plumed, José Hernández-Orallo Abstract An AI assessor is an external, ideally independent system that predicts an indicator, e.g., a loss value, of another AI system. Assessors can leverage information from the test results of many other AI systems and have the flexibility of being trained on any loss function or scoring rule: from squared error to toxicity metrics. Here we address the question: is it always optimal to train the assessor for the target metric? Or could it be better to train for a different metric and then map predictions back to the target metric? Using twenty regression and classification problems with tabular data, we experimentally explore this question for, respectively, regression losses and classification scores with monotonic and nonmonotonic mappings and find that, contrary to intuition, optimising for more informative metrics is not generally better. Surprisingly, some monotonic transformations are promising. For example, the logistic loss is useful for minimising absolute or quadratic errors in regression, and the logarithmic score helps maximise quadratic or spherical scores in classification. 1 Introduction AI models and systems are evaluated with very different metrics, depending on the purpose of application. For instance, metrics as diverse as the BLEU score ( Papineni et al., 2002 ) for translation, ‘Bold’ toxicity score ( Dhamala et al., 2021 ) for text generation, the area under the ROC curve ( Fawcett, 2006 ) for classification, asymmetric loss ( Elliott et al., 2005 ) for sales prediction ( Gogolev & Ozhegov, 2023 ) or any reward function ( Eschmann, 2021 ) for reinforcement learning, are commonly used. Models can be built or trained to minimise some loss, and then repurposed for a situation where another metric matters more. The most characteristic example today of this process is represented by ‘foundation models’ ( Bommasani et al., 2021 ) , such as language models. Even if the model can produce uncertainty estimates about the next token, and these are well calibrated, the metric of interest may be toxicity, for example. Since the model does not estimate toxicity, we need some external way to do this. Figure 1: For an energy consumption model M_{1} , we want to anticipate the squared error ( L^{+}_{2} ) for each new example using an external predictor, called assessor . Recommendations to customers are only made when the assessor predicts low L^{+}_{2} in the energy consumption estimate. We will explore assessors that optimise for the target loss function (squared loss L^{+}_{2} , top) but also assessors that use a proxy loss function (logistic loss L^{+}_{L} , bottom) followed by a transformation ( f ). Can the proxy assessor be better? One solution to this challenge is the development of assessor models ( Hernández-Orallo et al., 2022 ) . An assessor is a predictive model designed to estimate how well another system, called the base or subject system s , will perform on a given example or problem instance i for a specific validity metric before it is actually deployed. An assessor can estimate the conditional distribution \hat{p}(v|s,i) or simply (pointwise) map \langle s,i\rangle\mapsto v . Assessors are related to verifiers ( Li et al., 2023 ) but are anticipatory : rather than simply checking outcomes post-execution, they predict the outcomes in advance (i.e., given a new example i , they can predict the value v of the metric that s is expected to achieve). For instance, consider s a self-driving car and i a specific journey. An assessor could predict the safety outcome v of s for i . Assessors are used to anticipate any metric of quality, safety, bias or, in general, validity for any kind of subject system, from RL agents to language models. Assessors can be used to monitor or forecast system performance ( Schellaert et al., 2024a ) , to optimise configurations ( Zhao et al., 2024 ) , to do anticipatory reject ( Zhou et al., 2022 ) , or to delegate by routing ( Hu et al., 2024 ; Lu et al., 2023 ; Ding et al., 2024 ) . Assessors are usually trained on test data, capitalising on vast information from results of many systems and examples ( Burnell et al., 2023 ) . It may seem natural that the assessor is trained to optimise for the metric we are interested in. For instance, if the subject system s estimates daily energy consumption of households and the metric value v is given by the squared error ( L^{+}_{2} ) between actual and estimated consumption values, then one would expect that the assessor should be trained to predict the squared error that the system will incur for each household. However, in this paper we challenge the general assumption that training an assessor to optimise directly for a specific metric L necessarily results in the best optimisation outcome for L . In this example, what if optimising for logistic loss ( L^{+}_{L} ) were better? This situation is illustrated in Figure 1 . To start exploring this question, in this paper we will consider the base model is solving a regression problem and we will use generic regression metrics, such as absolute error, squared error and logistic error. We will consider signed and unsigned (absolute) versions of these three metrics, and explore whether optimising for a proxy metric is better than optimising for the target metric. From our experimental analysis we observe some results that may be explained by the distribution of errors (residuals) in the test data of the base subject systems. However, some other results are more surprising, such as the logistic error being the best in all situations. Give the effectiveness of some monotonic transformations for regression, we also explore the case of classification, focusing on monotonic transformations, and find a similar phenomenon. These findings suggest that learning an assessor for one central metric might suffice to optimise a family of monotonically-related metrics. 2 Background This work situates itself within a broad spectrum of research on error analysis and the exploration of alternative loss functions for training predictive models. However, the use of assessors resituates this question at the meta-level, as a second-order regression problem, an area that, to our knowledge, has not been explored yet. 2.1 Error analysis in regression In regression problems, the choice and optimisation of loss functions is critical to model performance. There is an extensive literature on traditional error measures ( Hyndman & Koehler, 2006 ; Botchkarev, 2018 ; Botchkarev, 2019 ; Chicco et al., 2021 ) such as Mean Squared Error (MSE), Absolute Error, and more robust variants such as Huber Loss ( Owen, 2007 ) , which falls somewhat in between squared and absolute error, or Tukey’s biweight loss ( Beaton & Tukey, 1974 ; Belagiannis et al., 2015 ) , which caps quadratic loss beyond a given point. Optimisation of these loss functions leads to different kinds of bias. For instance, quadratic error leads to estimators that are unbiased for the mean while absolute error leads to estimators that are unbiased for the median. Beyond their use in performance evaluation, the analysis of errors and residuals also serves a diagnostic purpose, helping to identify model inadequacies or violations of

Read the complete text