Save, Revisit, Retain: A Scalable Framework for Enhancing User Retention in Large-Scale Recommender Systems
Weijie Jiang, Armando Ordorica, Jaewon Yang, Olafur Gudmundsson, Yucheng Tu, Huizhong Duan · 2025
Read Save, Revisit, Retain: A Scalable Framework for Enhancing User Retention in Large-Scale Recommender Systems by Weijie Jiang, Armando Ordorica, Jaewon Yang, Olafur Gudmundsson, Yucheng Tu, Huizhong Duan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-18013.txt.
Introduction
Weijie Jiang and five co-authors model why users return to content they saved earlier, a revisit behavior that signals retention on large platforms such as Pinterest. Linking a later revisit to the save that caused it is noisy, and revisits can come days or weeks later at huge scale.
Their framework defines a cleaner attribution from saves to revisits and aggregates events at scale so ranking can favor items with high retention value. Deployed on a major recommendations surface serving hundreds of millions of users, it raised active users measurably with no added computing cost.
Catalog record
| CREATOR | Weijie Jiang, Armando Ordorica, Jaewon Yang, Olafur Gudmundsson, Yucheng Tu, Huizhong Duan |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-18013
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Save, Revisit, Retain: A Scalable Framework for Enhancing User Retention in Large-Scale Recommender Systems Weijie Jiang, Armando Ordorica, Jaewon Yang, Olafur Gudmundsson, Yucheng Tu, Huizhong Duan Abstract User retention is a critical objective for online platforms like Pinterest, as it strengthens user loyalty and drives growth through repeated engagement. A key indicator of retention is revisitation, i.e., when users return to view previously saved content, a behavior often sparked by personalized recommendations and user satisfaction. However, modeling and optimizing revisitation poses significant challenges. One core difficulty is accurate attribution: it is often unclear which specific user actions or content exposures trigger a revisit, since many confounding factors (e.g., content quality, user interface, notifications, or even changing user intent) can influence return behavior. Additionally, the scale and timing of revisitations introduce further complexity; users may revisit content days or even weeks after their initial interaction, requiring the system to maintain and associate extensive historical records across millions of users and sessions. These complexities render existing methods insufficient for robustly capturing and optimizing long-term revisitation. To address these gaps, we introduce a novel, lightweight, and interpretable framework for modeling revisitation behavior and optimizing long-term user retention in Pinterest’s search-based recommendation context. By defining a surrogate attribution process that links saves to subsequent revisitations, we reduce noise in the causal relationship between user actions and return visits. Our scalable event aggregation pipeline enables large-scale analysis of user revisitation patterns and enhances the ranking system’s ability to surface items with high retention value. Deployed on Pinterest’s Related Pins surface to serve 500+ million users, the framework led to a significant lift of 0.1% in active users without additional computational costs. Our data analysis reveals novel insights, such as the impact of content topics on revisitation rates; for example, users are more likely to revisit aesthetically pleasing topics. Introduction User retention is an important business objective for online platforms such as Pinterest, as it fosters stronger relationships with users, builds loyalty, and ultimately drives both growth and revenue. A key behavior indicating strong user retention is revisitation ( 25 ) , which occurs when a user returns to view specific content that they previously accessed and saved. This behavior often signals user satisfaction, typically driven by personalized content and recommendations. Therefore, identifying and leveraging previously saved content that encourages users to return can serve as a valuable signal for fostering long-term user retention. Modeling and optimizing revisitation behavior presents several challenges. The first is attribution: it is difficult to determine what exactly triggers a user to revisit. Many confounding factors can influence revisitation, such as content quality, user experience, and platform notifications or reminders ( 1 ; 25 ) . For instance, if a user returns with a completely different intent than in their previous session, it would be inappropriate to attribute that revisit to their earlier activities. The second challenge concerns scale. Since revisitations can occur within a day, a week, or even a month ( 2 ) , tracking them effectively requires maintaining extensive records of user activity over indefinite periods. This creates significant data processing and scaling challenges ( 13 ) . To date, there has been little concrete research on whether it is reasonable to disregard revisitations after a certain period (e.g., after X days). Because of these challenges, many existing methods prioritize immediate engagement metrics (e.g., clicks, saves) over longer-term retention ( 9 ; 21 ; 6 ; 8 ; 12 ) . In some cases, effective modeling also increases the time users spend ( 10 ) . Reinforcement learning (RL) offers a framework for optimizing long-term rewards. Although RL methods have improved intra-session engagement ( 3 ; 5 ; 17 ) , applying RL to optimize longer-term outcomes such as revisitation (events that may occur days after exposure) remains highly challenging. Existing RL approaches often lack the scalability and robustness to handle multi-day sequences that may encompass thousands of recommended items. To the best of our knowledge, there are currently no RL-based approaches that, at Pinterest’s scale, explicitly target long-term revisitation while jointly addressing attribution and scalability. Figure 1: Revisitation Attribution Illustration In this study, we propose a novel lightweight and interpretable framework for modeling user revisitation behavior and optimizing for long-term user retention in the recommendation context of search. Research shows that well‑chosen user behaviors can serve as effective surrogate rewards for predicting and optimizing long‑term engagement in recommender systems ( 23 ) . Therefore, to address the first challenge of attribution, we establish a surrogate for the causal effect of saving an item on subsequent revisitation events to reduce ambiguity and noise in revisitation signals. Figure 1 illustrated two types of revisitation attribution process. • Same day revisitation: A user searched for “recipe” on day 0 and saved a particular recipe item to their profile. Later that day, they revisited the saved recipe item in their profile. In this case, we attribute revisitation to the saved recipe item. We do not attribute other items the user viewed on other days to the saved recipe item. • Cross-day revisitation: The user saved a shoe item via search on day 1 and viewed some other items in subsequent days. We do not attribute revisitation to the shoe item until the user revisited it in own profile on day 6. To leverage the constructed revisitation attributions and address the second challenge of scale, we first conducts a comprehensive analysis of user revisitation patterns and then translate the findings into a scalable data pipeline. The pipeline aggregates event data over a seven‑day window across sessions and presentation surfaces by linking search‑surface save events to profile‑surface revisit events. This aggregation allows us to capture the most important revisitation signals for saved items, which serve as an auxiliary prediction task for the ranking model. In this way, the recommender system prioritizes items with high predicted probabilities of saving and revisit in the top positions, thereby driving downstream return visits and promoting long-term retention. This framework was shown to be effective in significantly increasing active users by 0.1% through both extensive offline experiments and a large-scale online A/B test with 24 million users on Pinterest’s Related Pins surface ( 16 ) over two months. In summary, our contributions include: • Large-scale analysis : We provide the first large-scale analysis of user revisitation patterns on online platform with hundreds of millions of users. • Methodology & metrics improvements : We propose a novel, lightweight, and scalable framework for modeling user revisitation behavior without additional cost, which is demonstrated to improve long-term user retention via a large-scale online experiment of 24 millions of users. • Intepretability & insights : The proposed framework offers greater interpretability than the SOTA methods, and it is able to show the reasoning behind the metrics improvement and the long-term and short-term revisitation patterns on topics. • Deployment : The proposed method has been deployed on Pinterest’s Related Pins surface ( 16 ) , efficiently serving 500+ millions of users without incurring additional computational costs.. Related Work User retention is critical objective for online platforms as sustained engagement typically drives more growth and revenue than short-term interactions. However, typical recommender systems in online platforms mainly focused on immediate engagement metrics such as