Write-Read Decoupling in Modern Large-Scale Search Engines: Architectures, Techniques, and Emerging Approaches

Xin LiangQing YangWenru QiuWenjie MaoTianyu MaMinghui ZhuNan Wang, [4pt] [email protected] · 2026

Read Write-Read Decoupling in Modern Large-Scale Search Engines: Architectures, Techniques, and Emerging Approaches by Xin LiangQing YangWenru QiuWenjie MaoTianyu MaMinghui ZhuNan Wang, [4pt] [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01260.txt.

Description

Xin Liang and six co-authors survey write-read decoupling in modern large-scale search engines. Frequent index updates keep results fresh, yet in the dominant Lucene-based architecture, segment merges triggered by writes compete with queries for CPU, disk, and cache, inflating latency. The survey systematically examines architectural solutions developed across industry and academia to separate write pressure from read latency.

Five principal patterns are identified, from node-level separation and compute-storage separation to full in-memory indexing, log-structured write paths, and in-place partial updates. Representative systems from major companies are surveyed. An emerging synthesis architecture combines several patterns with per-field update routing, sending simple fields through fast in-place updates while full-text fields follow a segment path. Open challenges include hybrid retrieval and serverless deployment.

Catalog record

CREATORXin LiangQing YangWenru QiuWenjie MaoTianyu MaMinghui ZhuNan Wang, [4pt] [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.01260

Browse the full library · About the Tower of Babel