MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising

Quanyu Dai, Jiaren Xiao, Zhaocheng Du, Jieming Zhu, Chengxiao Luo, Xiao-Ming Wu, Zhenhua Dong · 2025

Read MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising by Quanyu Dai, Jiaren Xiao, Zhaocheng Du, Jieming Zhu, Chengxiao Luo, Xiao-Ming Wu, Zhenhua Dong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-00334.txt.

Introduction

Quanyu Dai and co-authors present a network that adjusts ad model scores so they match real click and buy rates. Their 2025 work targets online advertising, where raw model scores often drift from true odds. Past fix methods can miss bent relations, ignore context, and vary by data slice.

The new model uses a flexible curve that always rises, so rank order is kept. Two added penalties guard order between bins and balance across data fields. Context features feed in through a flexible design. Tests on public and industry data show better matched scores than rival methods. The authors claim gains in both raw accuracy of odds and evenness across ad fields.

Catalog record

CREATORQuanyu Dai, Jiaren Xiao, Zhaocheng Du, Jieming Zhu, Chengxiao Luo, Xiao-Ming Wu, Zhenhua Dong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-00334

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising Quanyu Dai, Jiaren Xiao, Zhaocheng Du, Jieming Zhu, Chengxiao Luo, Xiao-Ming Wu, Zhenhua Dong Abstract . In online advertising, uncertainty calibration aims to adjust a ranking model’s probability predictions to better approximate the true likelihood of an event, e.g., a click or a conversion. However, existing calibration approaches may lack the ability to effectively model complex nonlinear relations, consider context features, and achieve balanced performance across different data subsets. To tackle these challenges, we introduce a novel model called Monotonic Calibration Networks, featuring three key designs: a monotonic calibration function (MCF), an order-preserving regularizer, and a field-balance regularizer. The nonlinear MCF is capable of naturally modeling and universally approximating the intricate relations between uncalibrated predictions and the posterior probabilities, thus being much more expressive than existing methods. MCF can also integrate context features using a flexible model architecture, thereby achieving context awareness. The order-preserving and field-balance regularizers promote the monotonic relationship between adjacent bins and the balanced calibration performance on data subsets, respectively. Experimental results on both public and industrial datasets demonstrate the superior performance of our method in generating well-calibrated probability predictions. Conference: Proceedings of the ACM Web Conference 2025; April 28-May 2, 2025; Sydney, NSW, Australia Proceedings of the ACM Web Conference 2025 (WWW ’25), April 28-May 2, 2025, Sydney, NSW, Australia DOI: 10.1145/3696410.3714802 ISBN: 979-8-4007-1274-6/25/04 CCS: Information systems Computational advertising 1. Introduction In recent years, machine learning models have been extensively used to assist or even replace humans in making complex decisions in practical scenarios. Numerous applications, such as medical diagnosis ( Caruana et al., 2015 ) , autonomous driving ( Bojarski et al., 2016 ) , and online advertising ( Huang et al., 2022 ) , have significant implications for people’s safety or companies’ economic incomes. Therefore, the performance of these models is of paramount importance. Apart from classification accuracy or ranking performance, it is also crucial for the predicted probability to accurately reflect the true likelihood of an event ( Guo et al., 2017 ) . However, modern neural networks often struggle to produce accurate probability estimate, despite excelling at classification or ranking tasks. This limitation, known as miscalibration, is affected by their model architectures and training (e.g., model capacity, normalization, and regularization) ( Guo et al., 2017 ; Thulasidasan et al., 2019 ) , significantly hindering their applications in real-world scenarios. In this paper, we focus on the scenario of online advertising. Typically, ECPM (Effective Cost Per Mille) serves as a key metric for measuring the revenue earned by the platform for every 1,000 ad impressions. In Pay-Per-Click advertising, ECPM is estimated as the product of the predicted Click-Through Rate (CTR) and the bidding price, i.e., CTR\times bid\times 1000 , which is used for ad ranking. This requires a model to output the predicted CTR that precisely reflects the probability of a user clicking on a given advertisement, as it directly influences bidding results and, consequently, the platform’s revenue. However, as highlighted in ( Deng et al., 2021 ; Pan et al., 2020 ) , the widely used deep ranking models heavily suffer from miscalibration, meaning the predicted CTR does not accurately represent the true probability. To tackle this challenge, uncertainty calibration has been widely studied ( Deng et al., 2021 ; Pan et al., 2020 ) . The goal is to train a well-calibrated predictor that can generate predictions accurately reflecting the actual probability of an event ( Gawlikowski et al., 2023 ) . In this paper, we focus on the post-hoc calibration paradigm ( Zadrozny and Elkan, 2001 ; Deng et al., 2021 ) due to its flexibility and widespread adoption in practice. Post-hoc methods fix the base predictor and learn a new calibration function to transform the predictions from the base predictor into calibrated probabilities. As a result, they can be conveniently used as a model-agnostic plug-and-play module placed on top of the base predictor in real recommender systems. Post-hoc methods mainly contain binning-based methods ( Zadrozny and Elkan, 2001 ; Zadrozny and Elkan, 2002 ) , scaling-based methods ( Platt et al., 1999 ; Kweon et al., 2022 ) , and hybrid methods ( Deng et al., 2021 ; Pan et al., 2020 ) . Binning-based methods, such as Histogram Binning ( Zadrozny and Elkan, 2001 ) and Isotonic Regression ( Zadrozny and Elkan, 2002 ) , first divide data samples into multiple bins and then directly use the empirical posterior probability as the calibrated probability for each bin. Consequently, their calibration function is a piecewise constant function (see Fig. 1 a), making the samples within a bin lose their order information . To alleviate this, scaling-based methods employ parametric functions for calibration. While preserving the order information among samples, they still face limitations in expressiveness due to their strong assumptions on the distributions of the class-conditional predicted scores, such as the Gaussian distribution in Platt Scaling ( Platt et al., 1999 ) . Moreover, hybrid methods integrate binning-based and scaling-based methods into a unified solution to fully leverage their advantages. Representative methods, including SIR ( Deng et al., 2021 ) , NeuCalib ( Pan et al., 2020 ) , and AdaCalib ( Wei et al., 2022 ) , typically learn a piecewise linear calibration function (see Fig. 1 (b) and (c)). Therefore, they are incapable of learning a perfect calibration function when dealing with complex nonlinear relationships between uncalibrated scores and the true data distribution, which is often encountered in real-world applications. We defer to Section 4.1 for a more detailed and intuitive analysis. In addition to limited expressiveness, existing post-hoc methods also struggle to adaptively capture varying miscalibration issues across contexts, and fail to achieve a balanced performance across different data fields. Specifically, NeuCalib introduces an additional module to capture context information, which heavily relies on an additional dataset. On the other hand, AdaCalib learns independent calibration functions for different data subsets, but it suffers from the data-efficiency issue. In summary, existing methods fall short in developing an effective calibration function due to several key issues: their limited expressiveness, lack of context-awareness, and failure to consider field-balance calibration performance . In this paper, we propose a novel hybrid approach, M onotonic C alibration Net work ( MCNet ), designed to address the aforementioned challenges in uncertainty calibration. Like other hybrid methods, MCNet comprises a binning phase for dividing samples into multiple bins and a scaling phase for learning the calibration function for each bin. The success of MCNet hinges on three key designs, including a monotonic calibration function (MCF), an order-preserving regularizer, and a field-balance regularizer. Firstly, the MCF serves as a powerful approximator for capturing the intricate relationship between uncalibrated scores and the true data distribution (see Fig. 1 (d)). It is constructed using monotonic neural networks, ensuring the monotonically increasing property by enforcing the positivity of its derivative. Additionally, MCF can effectively model uncalibrated scores and context features with a flexible model architecture, thus achieving context-awareness efficiently. Secondly, the order-preserving regularizer is intended to promote monotonicity between different bins by penalizing calibration functions of two adjacent bins that violate the relative order at the split point of the two bins. The proposed calibration function, along

Read the complete text