SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Rank aggregation
Renzhe Zhou, Songyang Li, Feiran Zhu, Chenglei Dai, Yi Zhang, Yi Wang, Jingwei Zhuo · 2026
Read SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Rank aggregation by Renzhe Zhou, Songyang Li, Feiran Zhu, Chenglei Dai, Yi Zhang, Yi Wang, Jingwei Zhuo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-19585.txt.
Introduction
Zhou and co-authors present SaFRO, a fusion framework that ranks short videos by user satisfaction, not just clicks. Search fusion blends many prediction signals into one score. Optimising for instant engagement can miss long-term satisfaction. SaFRO first builds a reward model from query-level behaviour proxies. A dual-relative policy optimisation step then learns the fusion policy by group comparisons. A task-relation module adapts weights to context. On Kuaishou search, offline tests and large A/B tests beat prior baselines on ranking and retention.
Catalog record
| CREATOR | Renzhe Zhou, Songyang Li, Feiran Zhu, Chenglei Dai, Yi Zhang, Yi Wang, Jingwei Zhuo |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40255 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.19585
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SaFRO: Satisfaction-Aware Fusion via Dual-Relative Policy Optimization for Short-Video Search DOI: XXXXXXX.XXXXXXX ISBN: 978-1-4503-XXXX-X/2018/06 CCS: Information systems Rank aggregation Renzhe Zhou, Songyang Li, Feiran Zhu, Chenglei Dai, Yi Zhang, Yi Wang, Jingwei Zhuo Abstract . Multi-Task Fusion plays a pivotal role in industrial short-video search systems by aggregating heterogeneous prediction signals into a unified ranking score. However, existing approaches predominantly optimize for immediate engagement metrics, which often fail to align with long-term user satisfaction. While Reinforcement Learning (RL) offers a promising avenue for user satisfaction optimization, its direct application to search scenarios is non-trivial due to the inherent data sparsity and intent constraints compared to recommendation feeds. To this end, we propose SaFRO, a novel framework designed to optimize user satisfaction in short-video search. We first construct a satisfaction-aware reward model that utilizes query-level behavioral proxies to capture holistic user satisfaction beyond item-level interactions. Then we introduce Dual-Relative Policy Optimization (DRPO), an efficient policy learning method that updates the fusion policy through relative preference comparisons within groups and across batches. Furthermore, we design a Task-Relation-Aware Fusion module to explicitly model the interdependencies among different objectives, enabling context-sensitive weight adaptation. Extensive offline evaluations and large-scale online A/B tests on Kuaishou short-video search platform demonstrate that SaFRO significantly outperforms state-of-the-art baselines, delivering substantial gains in both short-term ranking quality and long-term user retention. 1. Introduction Industrial short-video search systems typically employ a cascaded architecture (as shown in Figure 1 ): a retrieval stage first fetches candidate items from a large-scale index, followed by one or more ranking stages that progressively refine the order using richer features and more sophisticated models. The ranking pipeline commonly adopts a two-stage paradigm comprising Multi-Task Learning (MTL) and Multi-Task Fusion (MTF) ( 19 ; 3 ; 6 ; 16 ; 27 ; 35 ) . Specifically, the MTL model jointly predicts multiple objectives (e.g., click-through rate, long-play rate), while MTF synthesizes these heterogeneous signals into a final ranking score. Since the correlation between immediate engagement and long-term user satisfaction is fundamentally complex and obscure, MTF is essential for scalarizing multi-objective predictions into a unified metric that aligns with user satisfaction. Figure 1. Cascaded search system architecture. Reinforcement learning (RL) ( 28 ) has emerged as a promising framework for multi-objective optimization in recommendation systems, primarily due to its ability to handle delayed reward signals and support long-horizon objectives ( 38 ; 2 ; 20 ; 5 ; 37 ; 20 ) . In short-video platforms like TikTok and Kuaishou, interactions in recommendation scenario (e.g., feed) generate dense and sequential user feedback, which makes RL particularly well-suited for optimizing cumulative outcomes such as revisit behavior or user retention ( 2 ; 33 ) . However, directly applying RL solutions from recommendation to search is non-trivial due to two fundamental differences: (i) Intent and relevance constraints. Search is driven by explicit queries that express well-defined user intent and impose a strong relevance constraint. Recommendation, by contrast, typically lacks an explicit intent signal and serves latent, exploratory preferences, where diversity and exploration are often encouraged rather than constrained. (ii) Output form and interaction dynamics. Search returns a page-level ranked list, where users aim to find a specific answer or target item and often terminate quickly once satisfied, making a session closely aligned with a single request. Recommendation delivers an infinite scroll content stream, promoting continuous consumption and requests within a session. Consequently, search generates significantly sparser interaction data compared to recommendation: users typically inspect only a few results per page and provide limited feedback before ending their session. For instance, on the Kuaishou short-video platform, the density of positive interactions (e.g., clicks, long-plays, and likes) in search is approximately one-tenth of those observed in recommendation feeds. Furthermore, the length of user search behavior sequences is roughly two orders of magnitude shorter than that of recommendation sequences. This data sparsity makes it challenging to attribute long-term outcomes to specific ranking decisions, posing a fundamental obstacle to directly optimizing user satisfaction in search. Therefore, we argue that effective MTF in short-video search must go beyond immediate interactions and explicitly incorporate delayed signals of user satisfaction. To this end, we propose SaFRO ( Sa tisfaction-aware F usion via dual- R elative policy O ptimization), a multi-task fusion framework tailored for short-video search. SaFRO leverages query-level signals together with user retention to construct a unified satisfaction-aware reward model. To enable efficient policy learning under sparse and delayed feedback, we propose Dual-Relative Policy Optimization (DRPO) that updates the policy through relative comparisons both within groups and across batches. Furthermore, to explicitly model interdependencies among different objectives, SaFRO incorporates a Task-Relation-Aware Fusion module to generate context-sensitive logit adjustments for adaptive trade-offs. Our contributions are summarized as follows: • To the best of our knowledge, we are the first to model user satisfaction for industrial short-video search, unifying query-level signals and retention into a satisfaction-aware model for optimizing long-term user satisfaction. • We propose Dual-Relative Policy Optimization, which employs dual-relative advantage for policy training, together with a composite reward that jointly incorporates engagement, satisfaction, and format constraints. • We introduce a Task-Relation-Aware Fusion module that explicitly models inter-task relationships via a learnable task-relation matrix, producing context-adaptive logit adjustments for dynamic multi-objective balancing. 2. Related Work Multi-Task Learning . MTL has become a prevalent paradigm in industrial ranking systems, enabling joint prediction of diverse user feedback signals such as clicks, likes, watch time, and conversions ( 19 ; 3 ) . By sharing representations across related tasks, MTL improves data efficiency and generalization compared to single-task models. Early approaches like Shared-Bottom architecture ( 4 ) and Cross-Stitch networks ( 17 ) explored various feature sharing mechanisms. More recent industrial applications have adopted advanced architectures such as MMoE (Multi-gate Mixture-of-Experts) ( 16 ) and PLE (Progressive Layered Extraction) ( 29 ) to mitigate negative transfer and capture task-specific nuances. Multi-Task Fusion and Reinforcement Learning. MTF serves as the critical bridge between multi-objective predictions and the final ranking decision. Traditional MTF methods typically rely on heuristic or black-box optimization techniques, such as Bayesian Optimization ( 18 ) and the Cross-Entropy Method ( 22 ) , to search for optimal fusion weights. Although straightforward, these approaches often fail to adapt to dynamic user contexts and predominantly focus on immediate feedback, lacking mechanisms to optimize for long-term satisfaction. To address these limitations, recent research has pivoted toward Reinforcement Learning (RL) based frameworks, wherein a policy dynamically generates personalized fusion weights to maximize cumulative user engagement over time. BatchRL-MTF ( 38 ) pioneered this direction by integrating offline batch RL with conservative online exploration. RLUR ( 2 ) and AURO ( 33 ) advanced the paradigm by explicitly modeling user retention and handling environment non-stationarity through state abstraction. More recently, generative paradigms ( 39 ; 15 ; 14 ) have reformulated retention optimization as supervised learning conditioned on return-to-go or probabilistic flows. Concurrently, other works have focused on refining credit assignment via future impact decomposition ( 31 ) , addressing reward sparsity through satisfaction-based shaping ( 7 ) , leveraging human preferences to reinforce long-term engagement ( 32 ) , enabling fine-grained user-oriented exploration through UOEP ( 36 ) , and uncovering behavioral drivers via interpretable retention modeling with rationale learning ( 8