AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking

Zhi Kou, Xiang-Rong Sheng, Shuguang Han, Zhishan Zhao, Yueyao Cheng, Han Zhu, Jian Xu, Bo Zheng · 2025

Read AIF: Asynchronous Inference Framework for Cost-Effective Pre-Ranking by Zhi Kou, Xiang-Rong Sheng, Shuguang Han, Zhishan Zhao, Yueyao Cheng, Han Zhu, Jian Xu, Bo Zheng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-12934.txt.

Description

Zhi Kou and seven co-authors restructure how recommendation systems score candidates before ranking. In industrial systems, a pre-ranking model fetches features and scores items only after an upstream stage delivers candidates. The same users and items are recomputed again and again, and everything waits in line. Their Asynchronous Inference Framework splits the model: parts that depend only on the user run in parallel with retrieval, parts that depend only on the item run nearline, ahead of time. Only the truly interactive part happens in real time.

That reorganization cuts latency and frees budget to enrich the precomputed parts with better features and bigger models. Approximate methods handle the interaction terms online. The framework is deployed in Taobao's display advertising system, where the authors report notable gains without meaningfully higher cost. It is a systems lesson: when computation happens can matter as much as what is computed.

Catalog record

CREATORZhi Kou, Xiang-Rong Sheng, Shuguang Han, Zhishan Zhao, Yueyao Cheng, Han Zhu, Jian Xu, Bo Zheng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2511-12934

Browse the full library · About the Tower of Babel