Beyond Dense Connectivity: Explicit Sparsity for Scalable Recommendation Conference: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne, VIC, Australia Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia ISBN: 979-8-4007-2599-9/2026/07 DOI: 10.1145/3805712.3809630 CCS: Information systems Recommender systems CCS: Information systems Learning to rank
Yantao Yu, Sen Qiao, Lei Shen, Bing Wang, Xiaoyi Zeng · 2026
Read Beyond Dense Connectivity: Explicit Sparsity for Scalable Recommendation Conference: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne, VIC, Australia Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia ISBN: 979-8-4007-2599-9/2026/07 DOI: 10.1145/3805712.3809630 CCS: Information systems Recommender systems CCS: Information systems Learning to rank by Yantao Yu, Sen Qiao, Lei Shen, Bing Wang, Xiaoyi Zeng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-08011.txt.
Introduction
Yu and four colleagues argue that dense connectivity wastes power in recommenders fed by sparse, high-dimensional inputs. Scaling dense nets like deep MLPs often yields diminishing returns or outright degradation. Their study of industrial click-through-rate models reveals implicit connection sparsity: most learned weights drift toward zero while a small fraction stays prominent. The dense design itself becomes the bottleneck, forcing the model to process vast low-utility links instead of valid signals. SSR prunes connections deliberately so pattern modeling focuses on signals that matter.
Catalog record
| CREATOR | Yantao Yu, Sen Qiao, Lei Shen, Bing Wang, Xiaoyi Zeng |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55159 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.08011
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Dense Connectivity: Explicit Sparsity for Scalable Recommendation Conference: Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 20–24, 2026; Melbourne, VIC, Australia Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26), July 20–24, 2026, Melbourne, VIC, Australia ISBN: 979-8-4007-2599-9/2026/07 DOI: 10.1145/3805712.3809630 CCS: Information systems Recommender systems CCS: Information systems Learning to rank Yantao Yu, Sen Qiao, Lei Shen, Bing Wang, Xiaoyi Zeng Abstract . Recent progress in scaling large models has motivated recommender systems to increase model depth and capacity to better leverage massive behavioral data. However, recommendation inputs are high-dimensional and extremely sparse, and simply scaling dense backbones (e.g., deep MLPs) often yields diminishing returns or even performance degradation. Our analysis of industrial CTR models reveals a phenomenon of implicit connection sparsity: most learned connection weights tend towards zero, while only a small fraction remain prominent. This indicates a structural mismatch between dense connectivity and sparse recommendation data; by compelling the model to process vast low-utility connections instead of valid signals, the dense architecture itself becomes the primary bottleneck to effective pattern modeling. We propose SSR (Explicit S parsity for S calable R ecommendation), a framework that incorporates sparsity explicitly into the architecture. SSR employs a multi-view "filter-then-fuse" mechanism, decomposing inputs into parallel views for dimension-level sparse filtering followed by dense fusion. Specifically, we realize the sparsity via two strategies: a Static Random Filter that achieves efficient structural sparsity via fixed dimension subsets, and Iterative Competitive Sparse (ICS), a differentiable dynamic mechanism that employs bio-inspired competition to adaptively retain high-response dimensions. Experiments on three public datasets and a billion-scale industrial dataset from AliExpress (a global e-commerce platform) show that SSR outperforms state-of-the-art baselines under similar budgets. Crucially, SSR exhibits superior scalability, delivering continuous performance gains where dense models saturate. The code is available at https://github.com/Atticus666/SSRNet . † † cc-license: by 1. Introduction Deep learning recommender systems (DLRS) are the core ranking engines in many online services. Inspired by the success of LLMs ( Kaplan et al. 2020 ; Hoffmann et al. 2022 ) , we investigate whether recommender models exhibit similar scaling properties, where performance improves as model capacity and data size grow together. In practice, mainstream industrial CTR backbones such as Wide&Deep ( Cheng et al. 2016 ) and DLRM ( Naumov et al. 2019 ) remain relatively shallow, often 3–4 layers. Attempts to simply scale up these dense MLP-based architectures frequently lead to diminishing returns or even performance degradation, as reported in prior studies ( Rendle et al. 2020 ; Liu et al. 2020 ) . This implies that naive scaling of dense architectures is suboptimal. Figure 1 . Sparsity analysis of the hidden layer in online CTR backbone. (Left) 92% of weights are suppressed to near-zero (<10^{-3}) . (Right) 80% of weight power concentrates in the top 4% of dimensions. Sparsity analysis in dense CTR backbones. A fundamental issue is the mismatch between dense connectivity and sparse recommendation data. Unlike images or text with natural spatial or sequential locality, recommendation inputs consist of hundreds of heterogeneous feature fields where only a small subset is relevant for any given sample ( Zhang et al. 2021 ; Pi et al. 2020 ) . As we show in Figure 1 , learned weights in a production CTR model exhibit extreme implicit sparsity: 92% of connections are suppressed to near-zero, and 80% of weight mass concentrates in just 4% of dimensions. This confirms that the dense architecture itself becomes the bottleneck to effective scaling, and we provide a detailed analysis and theoretical grounding in Section 2. Building on this insight, we propose SSR (Explicit S parsity for S calable R ecommendation), a framework tailored for scaling on sparse recommendation data. SSR introduces a paradigm shift from implicit weight suppression to explicit signal filtering, building on a simple principle: first filter, then fuse. It performs explicit dimension-level sparse filtering before dense nonlinear fusion. We realize the sparsity via two complementary strategies: a Static Random Filter (SSR-S) that achieves efficient structural sparsity via fixed dimension subsets at zero FLOP cost, and Iterative Competitive Sparse (ICS/SSR-D), a differentiable dynamic mechanism that introduces sparsity to adaptively filter dimensions based on sample context. Unlike existing methods that rely on soft attention ( Huang et al. 2019 ) or post-hoc pruning ( Liu et al. 2020 ) , which maintain a fully connected graph and thus fail to block noise at scale, SSR enforces explicit sparsity, preventing noise propagation at the source and thus providing a cleaner gradient flow for scaling. The main contributions of this paper are summarized as follows: • We analyze the problem of scaling dense MLPs on sparse data, highlighting that implicit weight suppression fails to block noise, and provide evidence of strong sparse connection in Figure 1 . • We propose SSR, shifting the paradigm from implicit weight suppression to explicit signal filtering. It realizes explicit sparsity to isolate noise before dense interaction, ensuring expanded capacity is dedicated to valid signals. • We introduce two strategies to realize the explicit sparsity: a Static Random Filter for efficient structural sparsity, and ICS, a differentiable dynamic filtering mechanism that enables input-adaptive sparsification to capture complex dependencies. • Experiments on three public datasets and a billion-scale industrial dataset from AliExpress demonstrate that SSR achieves better accuracy under comparable compute budgets and exhibits more stable improvements when scaling size. 2. Motivation and Theoretical Foundation Before presenting the SSR framework, we provide the analytical and theoretical motivation underlying our design choices. 2.1. Why Dense MLPs Fail for Recommendation Recommendation inputs differ fundamentally from language or vision data. They are high-dimensional yet extremely sparse ( Zhang et al. 2021 ; Wang et al. 2025 ; Kasalickỳ et al. 2025 ) , each instance typically activates only a small subset of informative dimensions in a large feature space ( Pi et al. 2020 ) . Unlike images or text, where inputs exhibit natural spatial or sequential locality that architectures like CNNs and Transformers. Recommendation inputs consist of hundreds of heterogeneous feature fields such as user profiles, item attributes, contextual signals, and behavioral sequences, concatenated into a flat vector with no inherent adjacency or ordering among dimensions. For a specific impression or purchase, only a few contextual signals and historical preferences are truly relevant, while the vast majority are weakly relevant for that specific sample ( Yu et al. 2019 ; Lu et al. 2021 ) . This sparsity pattern means the effective response of the model (e.g., weight mass) is concentrated on a small fraction of the input dimensions. In contrast, a fully connected layer enforces globally dense connectivity by coupling each output neuron with all input dimensions. This forces the model to process vast low-utility connections, which dilutes valid signals and burdens the optimizer with suppressing noise rather than learning complex patterns ( Song et al. 2019 ) . We argue that this constitutes a misalignment of inductive bias : the dense connectivity assumes all dimension pairs are equally likely to interact, whereas the data exhibits highly concentrated, subset-based interactions. To support this analysis, we visualize the learned weights of the fully connected layer in an online industrial CTR model (Figure