Request-Only Optimization for Recommendation Systems DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence CCS: Computi
Liang Guo, Wei Li, Lucy Liao, Huihui Cheng, Rui Zhang, Yu Shi, Yueming Wang, Yanzun Huang, Keke Zhai, Pengchao Wang, Timothy Shi, Xuan Cao, Shengzhi Wang, Renqin Cai, Zhaojie Gong, Omkar Vichare, Rui Jian, Leon Gao † , Shiyan Deng, Xingyu Liu, Xiong Zhang, Fu Li † , Wenlei Xie † , Bin Wen † , Rui Li · 2025
Read Request-Only Optimization for Recommendation Systems DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence CCS: Computi by Liang Guo, Wei Li, Lucy Liao, Huihui Cheng, Rui Zhang, Yu Shi, Yueming Wang, Yanzun Huang, Keke Zhai, Pengchao Wang, Timothy Shi, Xuan Cao, Shengzhi Wang, Renqin Cai, Zhaojie Gong, Omkar Vichare, Rui Jian, Leon Gao † , Shiyan Deng, Xingyu Liu, Xiong Zhang, Fu Li † , Wenlei Xie † , Bin Wen † , Rui Li (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-05640.txt.
Introduction
Liang Guo and a large team of co-authors present a 2025 paper on a new way to train the deep learning models that power large recommendation services. These models rank among the biggest machine learning systems anywhere, trained on huge data stores to serve billions of users daily. Their scale strains both storage and training.
The new approach, called request-only optimization, treats one user request, rather than each single impression, as the unit of training data. Shared features are stored and processed once instead of many times. The team rebuilt the data pipeline and the model shape around this unit. They report gains in storage, training speed, and model quality at once.
Catalog record
| CREATOR | Liang Guo, Wei Li, Lucy Liao, Huihui Cheng, Rui Zhang, Yu Shi, Yueming Wang, Yanzun Huang, Keke Zhai, Pengchao Wang, Timothy Shi, Xuan Cao, Shengzhi Wang, Renqin Cai, Zhaojie Gong, Omkar Vichare, Rui Jian, Leon Gao † , Shiyan Deng, Xingyu Liu, Xiong Zhang, Fu Li † , Wenlei Xie † , Bin Wen † , Rui Li |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-05640
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Request-Only Optimization for Recommendation Systems DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Computing methodologies Artificial intelligence CCS: Computing methodologies Neural networks CCS: Information systems Recommender systems Liang Guo, Wei Li, Lucy Liao, Huihui Cheng, Rui Zhang, Yu Shi, Yueming Wang, Yanzun Huang, Keke Zhai, Pengchao Wang, Timothy Shi, Xuan Cao, Shengzhi Wang, Renqin Cai, Zhaojie Gong, Omkar Vichare, Rui Jian, Leon Gao † , Shiyan Deng, Xingyu Liu, Xiong Zhang, Fu Li † , Wenlei Xie † , Bin Wen † , Rui Li, Lu Fang, Xing Liu, Jiaqi Zhai † Abstract . Recommendation systems represent one of the largest machine learning applications on the planet – industry-scale recommendation models are trained with petabytes of data and serve billions of users every day. To utilize the rich user signals in the long user history, these models have been scaled up to unprecedented complexity, up to trillions of floating-point operations (TFLOPs) per example. This scale, coupled with the huge amount of training data, necessitates new storage and training algorithms to efficiently improve the quality of these complex recommendation systems. In this paper, we present a Request-Only Optimizations (ROO) training and modeling paradigm. ROO simultaneously improves the storage and training efficiency as well as the model quality of recommendation systems. We holistically approach this challenge through co-designing data (i.e., request-only data), infrastructure (i.e., request-only based data processing pipeline), and model architectures. Our ROO training and modeling paradigm treats a user request as a unit of the training data. Compared with the established practice of treating a user impression as a unit, this new design directly removes redundant features in data logging, saving data storage. Second, by de-duplicating computations and communications across multiple impressions in a request, this new paradigm enables highly scaled-up neural network architectures to better capture user interest signals, such as Generative Recommenders (GRs) and other request-only friendly architectures. Our proposed ROO training and modeling paradigm has been deployed to three major recommendation products, each with billions of active users. ROO data format allows for increasing the training sample volume by 43% to 150% across these three products. ROO training optimization yields a substantial increase in training throughput – up to 570% for the retrieval and early-stage ranking models, and between 32% and 100% for the late-stage ranking models. Combined with ROO-based neural architectures like Hierarchical Sequential Transduction Units (HSTU), ROO scales model FLOPs by 7x utilizing the same amount of training compute, enabling significant offline and online metric wins. Our studies offer practical and scalable design solutions for engineers seeking to build efficient and effective large-scale recommendation systems. 2 2 footnotetext: † Work done at Meta. 1. Introduction Deep Learning Recommendation Models (DLRMs) ( Naumov et al. 2019 ; Covington et al. 2016 ) are the backbone of recommendation systems serving billions of users daily. The scale at which these models operate creates substantial computational and infrastructure challenges across the entire pipeline – from data generation and pre-processing to GPU training and inference ( Jouppi et al. 2017 ; Gupta et al. 2020b ; Gangidi et al. 2024 ) . These challenges necessitate new solutions that reduce costs while maintaining or improving model quality. Figure 1 . A traditional training data pipeline incurs excessive feature duplication at the source of data generation, i.e. impression-level event-feature joiner (b), wasting significant computational resources throughout the system. A request-level joiner (this work) replaces the traditional joiner and captures all impressions belonging to the same request in one single sample (c). Figure 2 . Number of items (i.e., number of impression-level samples) within requests across three major recommendation products on a billion-user-scale platform. A fundamental inefficiency in current recommendation systems lies in their data collection and processing pipelines. As illustrated in Figure 1 , traditional systems follow a two-phase approach: an impression phase where multiple content items are sent in response to a user request, followed by a feedback phase where user actions are collected and joined with features to create training samples. The critical issue emerges at the impression-level event-feature joiner (Figure 1 .b), where extensive feature duplication occurs. For each user request that generates multiple impressions, identical user features are redundantly stored across all impression-level training samples. Figure 2 reveals that industrial recommendation products typically generate 4-7 impression samples per request. This redundancy cascades throughout the system – consuming storage, network bandwidth, and GPU computation resources. This inefficiency is particularly pronounced for modern recommendation models that leverage long user history sequences. These sparse user features constitute the majority of features in training samples, as demonstrated in previous work ( Naumov et al. 2019 ) . The growing adoption of sequential modeling techniques further exacerbates this challenge, as these approaches rely heavily on the very user sequence features being redundantly processed ( Zhai et al. 2024 ; Liu et al. 2024 ; Pancha et al. 2022 ; Kang and McAuley 2018 ) . While request-level optimizations has been recently applied to model inference ( You et al. 2024 ) to exploit the single-user, multiple-candidates characteristic of recommendation model inference, extending this concept to training presents significant challenges. Existing implementations often act as wrappers around impression-level data at inference time, increasing system complexity. Potential training-time optimization is complicated by the widespread use of impression-based data formats throughout client and server infrastructures, and the incremental arrival of impression-based events during the feedback phase. Our work presents a novel training data format, Request-Only Optimization (ROO), that captures all impression-level samples corresponding to the same request in a single training sample. This data format distinctly separates request-only (RO) data from non-request-only, or impression-level (NRO) data. This clean separation enables user sequence tensors to be processed exactly once across all impressions from the same request or app session, eliminating redundant computation of user-side features. Building on this foundation, we develop ROO training methodologies and ROO-friendly model architectures that capitalize on this data structure. By processing user-side information at the request level rather than the impression level, we reduce the computational workload from B_{NRO} to B_{RO} examples (where B_{NRO} represents the number of impression items in a batch and B_{RO} represents the number of requests). With an average of 4-7 (Figure 2 ) impressions per request ( B_{NRO}/B_{RO} ) and user-side features dominating the feature space, our approach yields theoretical speedups of 300-600%. Our ROO training paradigm delivers substantial benefits for retrieval and early-stage ranking models, which utilize minimal cross-features. These models demonstrate up to 6x improvements in training throughput with minimal code changes. For late-stage ranking models, we leverage ROO amortization to apply user feature compression, resulting in significant quality gains without increased computational costs. Most importantly, the dramatic reduction in GPU training costs has enabled the adoption of previously prohibitive modeling technologies such as Generative Recommenders (GRs) ( Zhai et al. 2024 ) . This represents a fundamental shift in what becomes computationally feasible in production recommendation systems. Overall, our paper makes the following key contributions: • A new Request-Only Optimization (ROO) training data format that eliminates pervasive feature duplication at the source (Section 2 ), increasing training sample volumes by 43% to 150% across different recommendation products utilizing