Deferred is Better: A Framework for Multi-Granularity Deferred Interaction of Heterogeneous Features DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Personalization

Yi Xu, Moyu Zhang, Chaofan Fan, Jinxin Hu, Yu Zhang, Xiaoyi Zeng · 2026

Read Deferred is Better: A Framework for Multi-Granularity Deferred Interaction of Heterogeneous Features DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Personalization by Yi Xu, Moyu Zhang, Chaofan Fan, Jinxin Hu, Yu Zhang, Xiaoyi Zeng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-12586.txt.

Introduction

Yi Xu and co-authors tackle a basic flaw in click-through rate models: features differ wildly in sparsity and content. Item IDs are sparse, while prices are dense, yet most models mix all features at once. Early entry of weak features can add noise and hide strong signals. Their MGDIN groups features by information density at several scales. A masking scheme then holds sparse groups back in early layers. Deeper layers admit them once a firm base is set.

Catalog record

CREATORYi Xu, Moyu Zhang, Chaofan Fan, Jinxin Hu, Yu Zhang, Xiaoyi Zeng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23493
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.12586

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Deferred is Better: A Framework for Multi-Granularity Deferred Interaction of Heterogeneous Features DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Personalization Yi Xu, Moyu Zhang, Chaofan Fan, Jinxin Hu, Yu Zhang, Xiaoyi Zeng Abstract . Click-through rate (CTR) prediction models estimates the probability of a user-item click by modeling interactions across a vast feature space. A fundamental yet often overlooked challenge is the inherent heterogeneity of these features: their sparsity and information content vary dramatically. For instance, categorical features like item IDs are extremely sparse, whereas numerical features like item price are relatively dense. Prevailing CTR models have largely ignored this heterogeneity, employing a uniform feature interaction strategy that inputs all features into the interaction layers simultaneously. This approach is suboptimal, as the premature introduction of low-information features can inject significant noise and mask the signals from information-rich features, which leads to model collapse and hinders the learning of robust representations. To address the above challenge, we propose a M ulti- G ranularity Information-Aware D eferred I nteraction N etwork (MGDIN), which adaptively defers the introduction of features into the feature interaction process. MGDIN’s core mechanism operates in two stages: First, it employs a multi-granularity feature grouping strategy to partition the raw features into distinct groups with more homogeneous information density in different granularities, thereby mitigating the effects of extreme individual feature sparsity and enabling the model to capture feature interactions from diverse perspectives. Second, a delayed interaction mechanism is implemented through a hierarchical masking strategy, which governs when and how each group participates by masking low-information groups in the early layers and progressively unmasking them as the network deepens. This deferred introduction allows the model to establish a robust understanding based on high-information features before gradually incorporating sparser information from other groups. Extensive offline and online experiments validate the superiority of MGDIN. 1. Introduction Click-through rate (CTR) prediction is a cornerstone task in recommendation systems and a critical determinant of user experience ( 27 ; 9 ; 29 ) . Its core challenge lies in effectively modeling the complex and subtle interactions across a vast and diverse feature space to accurately estimate the probability of a user clicking on a candidate item ( 10 ; 25 ; 26 ) . The advent of deep learning has spurred the development of feature interaction architectures in the CTR field, such as Factorization Machines ( 12 ; 4 ; 21 ; 7 ) , Cross-Networks ( 18 ; 19 ; 8 ) , and Attention mechanisms ( 16 ; 13 ; 6 ) . These models aim to automatically learn high-order feature combinations, moving beyond manual feature engineering to generate expressive and high-quality hidden representations that enhance models’ predictive performance ( 14 ) . However, despite the remarkable achievements of existing deep CTR prediction models, they are almost universally built upon a uniform feature interaction strategy, i.e., "one-size-fits-all". This approach inputs all features into the feature interaction layers simultaneously, ignoring a fundamental yet often overlooked challenge: feature heterogeneity. Unlike in fields such as computer vision (CV) or natural language processing (NLP), the features in recommender systems are inherently heterogeneous, exhibiting significant disparities in statistical properties such as sparsity, cardinality, and information content ( 23 ; 22 ; 5 ; 3 ; 24 ; 15 ) . For example, the extreme sparsity of an Item ID feature is fundamentally different from the dense nature of a numerical feature like user age and item price, or a low-cardinality categorical feature like device type. While numerous recent studies seek to mitigate the issue of item ID sparsity by augmenting them with multimodal semantics, they frequently neglect the heterogeneous information density across other features, which in turn creates performance bottlenecks. The premature introduction of noisy and undertrained feature representations from extremely sparse features can inject significant noise into the feature interaction process. This interaction noise often overwhelms the robust signals from more informative and dense features, leading to suboptimal representations or even catastrophic representation collapse, ultimately hindering model convergence and degrading final prediction performance. To overcome the above limitations, we challenge the fundamental premise of synchronous feature interaction and propose a new guiding principle: deferred is better. Our core insight is that features should not participate uniformly at all stages of the interaction process. Instead, we argue that a more robust model can be built by first establishing a stable foundation using high-information features, and only then gradually incorporating sparser, more granular, and potentially noisier features. Acting on this principle, this paper introduces a M ulti- G ranularity Information-Aware D eferred I nteraction N etwork (MGDIN), which adaptively defers the incorporation of each feature group into the interaction process. Specifically, MGDIN’s core mechanism operates through two distinct phases: 1) Multi-Granularity Feature Grouping. To address the uneven information content of features, we first employ a multi-granularity grouping strategy, which partitions the raw features into distinct groups with more homogeneous information density. The goal is twofold: to maximize the collective information content within each group and to mitigate the disruptive impact of extremely sparse individual features on subsequent model interactions. 2) Information-Aware Deferred Interaction. Building on the above feature groups, we then implement a deferred interaction mechanism via a hierarchical masking strategy. This strategy governs the timing of each group’s participation based on its information content. In the early layers, low-information groups are masked to prevent representation collapse arising from premature and noisy interactions. As the network deepens, these masks are progressively removed, allowing the groups to participate in the feature interaction process. This deferred introduction operation enables the model to first establish a robust understanding from high-information features before gradually and safely integrating the sparser information from other feature groups. Our contributions are summarized as follows: • To the best of our knowledge, we are the first work to propose a deferred feature interaction framework to address the challenge of feature heterogeneity in CTR prediction. • By employing a multi-granularity strategy and deferred interaction, MGDIN mitigates the issues arising from feature heterogeneity, namely, uneven information distribution and representation collapse due to sparse features, enabling the model to learn a more robust feature interaction paradigm. • Comprehensive evaluations, encompassing both offline datasets and online A/B tests, not only confirm the superiority of MGDIN over state-of-the-art baselines but also underscore the efficacy of its core deferred interaction principle. 2. Preliminaries Given a user u and a candidate item i with a set of features \mathcal{X} under a specific context, the input features encompass user profiles (e.g., age, gender), item attributes (e.g., item ID, category ID, brand ID), and contextual information (e.g., geographic location, device). To address the extreme sparsity of item IDs, we leverage recent advancements by replacing the original item IDs with dense semantic IDs generated from both multimodal content and user behavior. This foundational step substantially mitigates heterogeneity at the input level. The input of the model can be represented as \mathcal{X}=[x_{1},\dots,x_{n}] , where n is the number of features. The objective of CTR prediction is to estimate the probability \hat{y}\in[0,1] that the user u will click the item i under \mathcal{X} , which can be formulated as follows,

Read the complete text