PDX: A Data Layout for Vector Similarity Search Conference: The 2025 International Conference on Management of Data; June 22–27, 2025; Berlin, Germany
Leonardo Kuffo, Elena Krippner, Peter Boncz · 2025
Read PDX: A Data Layout for Vector Similarity Search Conference: The 2025 International Conference on Management of Data; June 22–27, 2025; Berlin, Germany by Leonardo Kuffo, Elena Krippner, Peter Boncz (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04422.txt.
Description
Leonardo Kuffo, Elena Krippner, and Peter Boncz turn the storage of vectors sideways to make similarity search run faster. Vector databases store each embedding as a row, but searching row by row wastes the processor. Their 2025 paper proposes PDX, a block layout that stores dimensions vertically across many vectors, so a search can sweep one dimension of thousands of vectors at once in tight loops.
On ordinary storage, PDX beat highly optimized distance kernels by about forty percent using only plain code that the compiler vectorizes itself. Dimension-pruning algorithms, which skip most coordinates, had paradoxically been losing to brute scans on the old layout; on PDX their advantage returned, two to seven times over. A new pruning strategy, PDX-BOND, works on data as stored, with no preprocessing, suiting databases that update constantly. The paper is a reminder that layout is an algorithm.
Catalog record
| CREATOR | Leonardo Kuffo, Elena Krippner, Peter Boncz |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-04422