CCD-Level and Load-Aware Thread Orchestration for In-Memory Vector ANNS on Multi-Core CPUs
Yuchen Huang 1 2 3 , Baiteng Ma 1 2 3 , Yiping Sun 3 , Yang Shi 3 , Xiao Chen 3 Xiaocheng Zhong 3 , Zhiyong Wang 3 , Yao Hu 3 , Chuliang Weng 2 · 2026
Read CCD-Level and Load-Aware Thread Orchestration for In-Memory Vector ANNS on Multi-Core CPUs by Yuchen Huang 1 2 3 , Baiteng Ma 1 2 3 , Yiping Sun 3 , Yang Shi 3 , Xiao Chen 3 Xiaocheng Zhong 3 , Zhiyong Wang 3 , Yao Hu 3 , Chuliang Weng 2 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10090.txt.
Description
Huang and eight colleagues tune in-memory vector search on multi-core CPUs, where approximate nearest neighbour search powers search engines, recommenders and ads. Recent indexes make CPUs cost-effective for million-scale search, yet per-core throughput stalls on memory-access latency and heavy distance computation. Simply adding cores on modern chiplet-based CCD processors does not scale well, because the multi-chiplet layout is ignored.
Their workload- and hardware-aware thread orchestration works at CCD level, unifying inter-query parallel HNSW search and intra-query parallel IVF search behind one interface. Cache-friendly, workload-adaptive task mapping plus CCD-aware task stealing fixes load imbalance, exploiting the high access locality and low cache use found in production request patterns. On real Xiaohongshu workloads from search, recommendation and advertising, throughput rises up to 3.7 times, with P50 and P999 latency falling thirty to ninety percent.
Catalog record
| CREATOR | Yuchen Huang 1 2 3 , Baiteng Ma 1 2 3 , Yiping Sun 3 , Yang Shi 3 , Xiao Chen 3 Xiaocheng Zhong 3 , Zhiyong Wang 3 , Yao Hu 3 , Chuliang Weng 2 |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.10090