ScaleGANN: Accelerate Large-Scale ANN Indexing by Cost-effective Cloud GPUs

Lan Lu, Peiqi Yin, Isaac Yang, Tao Luo, Hua Fan, Wenchao Zhou, Feifei Li, Boon Thau Loo · 2026

Read ScaleGANN: Accelerate Large-Scale ANN Indexing by Cost-effective Cloud GPUs by Lan Lu, Peiqi Yin, Isaac Yang, Tao Luo, Hua Fan, Wenchao Zhou, Feifei Li, Boon Thau Loo (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-10135.txt.

Description

Lu and seven colleagues accelerate large-scale graph-index building for approximate nearest neighbour search, the engine behind vector databases. CPU-based construction is slow for large, high-dimensional data, while GPUs are costly and memory-limited. Their end-to-end system, ScaleGANN, builds graph indexes on low-cost spot GPUs in a distributed cloud setup.

A divide-and-merge design with an optimised vector partitioning algorithm improves indexing time and space efficiency while preserving index quality. A novel resource allocation strategy enables multi-GPU indexing parallelism and cost-effectiveness for both build and query, with a task scheduler and cost model managing spot instances. On large real-world data sets, ScaleGANN builds indexes up to nine times faster at six times lower price than the DiskANN benchmark, with similar scalability and search quality.

Catalog record

CREATORLan Lu, Peiqi Yin, Isaac Yang, Tao Luo, Hua Fan, Wenchao Zhou, Feifei Li, Boon Thau Loo
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.10135

Browse the full library · About the Tower of Babel