Infer As You Train: A Symmetric Paradigm of Masked Generative for Click-Through Rate Prediction DOI: 10.1145/1122445.1122456 Conference: Woodstock ’18: ACM Symposium on Neural Gaze Detection; June 03–05, 2018; Woodstock, NY Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05, 2018, Woo

Moyu Zhang, Yujun Jin, Yun Chen, Jinxin Hu, Yu Zhang, Xiaoyi Zeng · 2025

Read Infer As You Train: A Symmetric Paradigm of Masked Generative for Click-Through Rate Prediction DOI: 10.1145/1122445.1122456 Conference: Woodstock ’18: ACM Symposium on Neural Gaze Detection; June 03–05, 2018; Woodstock, NY Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05, 2018, Woo by Moyu Zhang, Yujun Jin, Yun Chen, Jinxin Hu, Yu Zhang, Xiaoyi Zeng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-14403.txt.

Introduction

Moyu Zhang, Yujun Jin, Yun Chen, and three co-authors make a click-rate model use its build skill at serve time, not just in training. Click models for ads and feeds more and more train in a build-it style, to learn how traits hang together. But at serve time they fall back to a plain yes-no score and waste that skill. The team's frame, SGCTR, makes training and serving match.

After masked build-style training, the model works on each live sample the same way. It reworks the sample's noisy traits step by step, then scores it. That clean-up blunts bad traits and lifts the hit rate. Wide tests show the matched design beats the lopsided build-style tools that came before it.

Catalog record

CREATORMoyu Zhang, Yujun Jin, Yun Chen, Jinxin Hu, Yu Zhang, Xiaoyi Zeng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-14403

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Infer As You Train: A Symmetric Paradigm of Masked Generative for Click-Through Rate Prediction DOI: 10.1145/1122445.1122456 Conference: Woodstock ’18: ACM Symposium on Neural Gaze Detection; June 03–05, 2018; Woodstock, NY Woodstock ’18: ACM Symposium on Neural Gaze Detection, June 03–05, 2018, Woodstock, NY Price: 15.00 ISBN: 978-1-4503-XXXX-X/18/06 Moyu Zhang, Yujun Jin, Yun Chen, Jinxin Hu, Yu Zhang, Xiaoyi Zeng Abstract . Generative models are increasingly being explored in click-through rate (CTR) prediction field to overcome the limitations of the conventional discriminative paradigm, which rely on a simple binary classification objective. However, existing generative models typically confine the generative paradigm to the training phase, primarily for representation learning. During online inference, they revert to a standard discriminative paradigm, failing to leverage their powerful generative capabilities to further improve prediction accuracy. This fundamental asymmetry between the training and inference phases prevents the generative paradigm from realizing its full potential. To address this limitation, we propose the S ymmetric Masked G enerative Paradigm for CTR prediction (SGCTR), a novel framework that establishes symmetry between the training and inference phases. Specifically, after acquiring generative capabilities by learning feature dependencies during training, SGCTR applies the generative capabilities during online inference to iteratively redefine the features of input samples, which mitigates the impact of noisy features and enhances prediction accuracy. Extensive experiments validate the superiority of SGCTR, demonstrating that applying the generative paradigm symmetrically across both training and inference significantly unlocks its power in CTR prediction. 1. Introduction With the recent development of deep learning, the performance of click-through rate (CTR) prediction models in the recommender system domain has also seen rapid advancements. These models, by utilizing methods such as factorization machines (FM) ( fm ; deepfm ) and the attention mechanism ( hstu ) , can effectively capture complex interactions among input features, thereby improving the accuracy of predictions. However, conventional CTR prediction models also face persistent challenges, such as training instability and representation collapse arising from data sparsity, which ultimately limits their generalization ability. Addressing the above challenges has therefore become a key focus of CTR prediction research filed. Inspired by the recent success of Large Language Models (LLMs) in the natural language processing domain, generative paradigms have gained widespread attention in recommender systems. Recommender research often treats user behavior sequences as sentences and employs auto-regressive models to predict the next item that the target user will click ( onerec ; cobra ) . While this approach moves beyond simple binary classification by introducing a generative paradigm, it fundamentally neglects crucial cross-features and item attribute features, leading to significant performance degradation ( dgenctr ) . To address this issue, subsequent research has shifted its focus from modeling sequences to modeling the distribution of features in samples. By employing techniques such as dropout and discrete diffusion processes ( genctr ; dgenctr ) , these CTR models learn to capture complex feature dependencies. This, in turn, effectively mitigates representation collapse and enhances the modeling of feature interactions. Despite their progress, existing generative CTR models suffer from a critical problem in their application: they employ asymmetric paradigms for training and inference. During training, the generative objectives are used to enhance the model’s understanding of data distributions and to learn robust feature representations. During online inference, however, these models revert to a conventional discriminative approach, performing predictions based on potentially noisy input samples. This asymmetry means that the powerful generative capabilities acquired during training are entirely discarded at inference time, preventing these generative models from realizing their full potential in conducting predictions. Therefore, to better leverage the potential of generative models, this paper proposes the S ymmetric Masked G enerative Paradigm for CTR prediction (SGCTR) which fully utilizes the model’s feature generation capabilities by iteratively generating features for input samples during online inference. This allows for further improvement in prediction performance by denoising the input features, building upon the strong generalization capabilities of the network parameters acquired during training. Specifically, during training, SGCTR employs a discrete diffusion process to predict masked features. This enables the model to learn the dependencies among features within a sample, allowing it to master plausible feature combinations. The core innovation of SGCTR lies in the inference phase, where we propose an iterative refinement strategy. In each iteration, the model first predicts the features for all masked features. Simultaneously, the prediction score for each feature is used as a confidence weight to redefine corresponding original input feature. Features with lower confidence are then re-masked for the next iteration. This process is repeated for a fixed number of steps, with the masking ratio gradually decreasing until a complete feature set is generated. At this point, all feature representations have been updated using their prediction scores, thereby leveraging the generative capabilities to denoise the model input and ultimately improve prediction performance. The contributions of our paper can be summarized as follows: • As far as we know, SGCTR is the first work to propose a symmetric generative paradigm for the training and inference phases. • SGCTR enhances prediction accuracy by employing an iterative refinement strategy during inference, leveraging the generative capabilities acquired during training to denoise the input. • Evaluations using offline datasets and online A/B testing have demonstrated the effectiveness of SGCTR. 2. Preliminaries The objective of Click-Through Rate (CTR) prediction is to forecast the likelihood that a user will click on a specific item. Given a set of input features \boldsymbol{X} and the label space y\in\left\{0,1\right\} , the CTR prediction task aims to devise a unified ranking formula \mathcal{F}:\boldsymbol{X}\rightarrow y , to concurrently provide accurate and personalized prediction scores, indicating whether the target user will click the target item. The feature set \boldsymbol{X} can be constructed as [f^{1},f^{2},...,f^{N}] , where N represents the number of the feature fields. Mathematically, the CTR prediction task estimates the probability that the target user will interact with the target item, as illustrated below: (1) \displaystyle P(y|\boldsymbol{X})=\mathcal{F}(f^{1},f^{2},...,f^{N}) 3. Method While prior work has significantly improved performance by introducing generative paradigms to overcome the limitations of discriminative models, these models often merely utilize the parameters learned during training for scoring in the inference phase, failing to fully leverage the generative capabilities acquired during training ( genctr ; dgenctr ) . This paper aims to address this deficiency by proposing a novel symmetric paradigm that aligns the training and inference phases, directly applying the generative capabilities acquired during training to the online inference process. Our approach is divided into two phases: training and online inference. As our main contribution lies in the inference phase, we primarily adopt DGenCTR ( dgenctr ) during training to equip the model with fundamental feature generation capabilities. In the online inference phase, we introduce a novel iterative refinement strategy to redefine the model input by leveraging its generative capabilities. 3.1. Training Phase During the training phase, we employ a discrete diffusion process to learn the correlations among features. Specifically, we apply a binary mask M\in\{0,1\} to each feature field in an input sample to either preserve the feature or replace it with a mask token, representing introduced noise. The noise level is controlled by a forward process executed over discrete time steps t . In each training step, a time-step t is sampled, and the proportion

Read the complete text