Scaled Supervision is an Implicit Lipschitz Regularizer

Zhongyu Ouyang, Chunhui Zhang, Yaning Jia, Soroush Vosoughi · 2025

Read Scaled Supervision is an Implicit Lipschitz Regularizer by Zhongyu Ouyang, Chunhui Zhang, Yaning Jia, Soroush Vosoughi (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-14813.txt.

Introduction

Zhongyu Ouyang, Chunhui Zhang, Yaning Jia, and Soroush Vosoughi study stability in click prediction for social platforms in a 2025 study. Recommenders judge success by click-through rate, a binary guess about whether a user will tap an item. Real feeds change fast, though, and models chase short-term noise, overfitting passing fads instead of durable patterns of interaction.

The authors argue that scaling the supervision signal acts quietly as a smoothness control on the model, in the style of a Lipschitz regularizer. Steadier models should track genuine user and item dynamics rather than jitter. The insight links a training detail to robustness in fast-moving online environments.

Catalog record

CREATORZhongyu Ouyang, Chunhui Zhang, Yaning Jia, Soroush Vosoughi
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE70 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-14813

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Scaled Supervision is an Implicit Lipschitz Regularizer Zhongyu Ouyang, Chunhui Zhang, Yaning Jia, Soroush Vosoughi Abstract In modern social media, recommender systems (RecSys) rely on the click-through rate (CTR) as the standard metric to evaluate user engagement. CTR prediction is traditionally framed as a binary classification task to predict whether a user will interact with a given item. However, this approach overlooks the complexity of real-world social modeling, where user, item, and their interactive features change dynamically in fast-paced online environments. This dynamic nature often leads to model instability, reflected in overfitting short-term fluctuations rather than higher-level interactive patterns. While overfitting calls for more scaled and refined supervisions, current solutions often rely on binary labels that overly simplify fine-grained user preferences through the thresholding process, which significantly reduces the richness of the supervision. Therefore, we aim to alleviate the overfitting problem by increasing the supervision bandwidth in CTR training. Specifically, (i) theoretically, we formulate the impact of fine-grained preferences on model stability as a Lipschitz constrain; (ii) empirically, we discover that scaling the supervision bandwidth can act as an implicit Lipschitz regularizer, stably optimizing existing CTR models to achieve better generalizability. Extensive experiments show that this scaled supervision significantly and consistently improves the optimization process and the performance of existing CTR models, even without the need for additional hyperparameter tuning 1 1 1 https://github.com/zyouyang/ImpLipReg˙CTR.git . Dartmouth College, Hanover, NH 03755 {zhongyu.ouyang.gr, chunhui.zhang.gr, yaning.jia.gr, soroush.vosoughi}@dartmouth.edu Introduction In modern social media, recommender systems are ubiquitous in online applications and have redefined the user experience in product recommendation on e-commerce platforms ( Wang et al. 2021 ; Schafer, Konstan, and Riedl 1999 ) , personalized modelling ( Gomez-Uribe and Hunt 2015 ; Van den Oord, Dieleman, and Schrauwen 2013 ; Wu et al. 2022 ) , and friend recommendation on social media platforms ( Ma et al. 2008 ; Jamali and Ester 2010 ; Fan et al. 2019 ; Wen et al. 2022 ; Ouyang et al. 2024a ; Ouyang et al. 2024b ) . As an important type of RecSys, click-through rate (CTR) models predict users’ interaction probability, which help developers improve user engagement and recommendation accuracy in social media. CTR models specifically incorporate contextual features (e.g., user’s demographic information, item descriptions, or transaction timestamps between a user-item interaction) when making interaction prediction. The problem is generally formulated as a binary classification task. While the above CTR model training pipeline is commonly adopted, it often overlooks the underlying dynamics in the embeddings learned from the recommendation environment. Interaction prediction is delicately influenced by various factors, including recent user behavior, trends, and contextual information, all of which can shift rapidly over time. This dynamic nature poses a significant challenge for CTR models - they may struggle to keep pace with the continual changes as the binary classification formulation oversimplifies the structure of preferences. As a result, the models could overfit short-term fluctuations in the data, failing to capture general patterns that are representative of long-term interactions and impairing its stability to input noises. The Lipschitz constant is commonly adopted to assess the stability of a model. A smaller Lipschitz constant indicates that the model’s output is less prone to change drastically in response to minor variations in the input, indicating better stability. Model stability is essential for practical CTR prediction, given the fast-changing nature of inputs in RecSys. Since there are massive studies ( Shi et al. 2022 ; Wang et al. 2024 ) that acknowledge accurately estimating the Lipschitz constant for a deep model is challenging due to the complexity and diversity of the feature construction layers involved, directly estimating a CTR model’s Lipschitz constant is impractical. Alternatively, we study the theoretical impact of the granularity of preference supervisions on model stability. Our theorem suggests that training with scaled supervision via more fine-grained preferences can be formulated as an implicit Lipschitz regularizer for CTR models to achieve better stability and generalizability. In the context of CTR prediction, we propose replacing binary labels with fine-grained preference feedback to guide the training of a CTR model. With more scaled supervision, the model’s output becomes smoother and more consistent, resulting in less overfitting short-term fluctuations hidden in the data with improved model generalization and stability in learning high-level interaction patterns. In this work, we leverage the continuous nature of ratings in user feedback to increase the bandwidth of supervision signals beyond current Boolean labels in CTR prediction with explicit feedback. By modeling ratings categorically, we enable the learning of high-level interaction patterns across distinct rating levels in a more segmented and independent manner. These categorical ratings are subsequently transformed into binary labels, aligning with the original CTR prediction task while preserving the enriched supervision benefits. Our contributions can be summarized as follows: • We theoretically identify the impact of the granularity of preference supervisions on a deep model’s stability: more refined supervisions help reduce a model’s Lipschitz constant, resulting in more stable and smoother model output. • Inspired by the theorem, we identify the limitations of binary labeling in capturing the full spectrum of user preferences in modern social media recommender systems, and then propose to use more fine-grained preferences to increase the bandwidth of supervisions for more enhanced model stability and generalizability. • Extensive experiments based on our open-release implementation and datasets are conducted to demonstrate that our approach significantly improves CTR prediction performance and promotes better learning dynamics in optimizing model parameters. Preliminary Lipschitz Constant in Deep Models A function f:\mathbb{R}^{n}\rightarrow\mathbb{R}^{m} is said to be Lipschitz continuous on an input set \mathcal{X}\subseteq\mathbb{R}^{n} if there exists a bound K\geq 0 such that for all \mathbf{x},\mathbf{y}\in\mathcal{X} , f satisfies: \left\lVert{f(\mathbf{x})-f(\mathbf{y})}\right\rVert\leq K\left\lVert\mathbf{x}-\mathbf{y}\right\rVert,\quad\forall\mathbf{x},\mathbf{y}\in\mathcal{X}. (1) The smallest possible K in Equation ( 1 ) is defined as the Lipschitz constant of f , denoted as {\rm Lip}(f) : {\rm Lip}(f)=\sup\limits_{\mathbf{x},\mathbf{y}\in\mathcal{X},\mathbf{x}\neq\mathbf{y}}\frac{\left\lVert f(\mathbf{x})-f(\mathbf{y})\right\rVert}{\left\lVert\mathbf{x}-\mathbf{y}\right\rVert}. (2) In this context, f is known as a K -Lipschitz function, where the Lipschitz constant quantifies the maximum change in the function’s output resulting from a unit-norm perturbation in its input. This constant is a crucial indicator of a deep model’s stability wrt the inputs. However, determining the exact Lipschitz constant is computationally challenging. As highlighted in ( Virmaux and Scaman 2018 ) , computing the exact Lipschitz constant for deep models has been proven to be NP-hard. Figure 1: (a) and (b) demonstrate the existing and our modified paradigm of training CTR models with explicit feedback respectively. The input includes user, item IDs and the contextual information of the interaction in between. Our paradigm enlarges the supervision bandwidth from explicit preferences: we slightly modify existing CTR models’ prediction layers to recover the explicit preferences for more refined supervision. These logits are then normalized via the Softmax function to the probability that matches the CTR prediction task. CTR Prediction in Modern RecSys General CTR Prediction Paradigm Click-through rate (CTR) prediction is a task focused on estimating the probability that a user will engage with a specific item. This prediction takes the user and item IDs, features related to the user, the item, and the broader context in which the interaction occurs as the input, and the expected output is a probability likelihood ranging from zero to one. Formally, we denote the IDs of user i and

Read the complete text