FlexKV: Flexible Index Offloading for Memory-Disaggregated Key-Value Store

Zhisheng Hu, Jiacheng Shen, Ming-Chang Yang · 2025

Read FlexKV: Flexible Index Offloading for Memory-Disaggregated Key-Value Store by Zhisheng Hu, Jiacheng Shen, Ming-Chang Yang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-16148.txt.

Introduction

Zhisheng Hu, Jiacheng Shen, and Ming-Chang Yang present a data store design in a paper uploaded to arXiv in 2025. Disaggregated memory splits processors and memory into separate pools in a data center. Key-value stores on that layout manage remote data, but slow down because index lookups lean on one-sided atomic network operations and compute-side caches work poorly.

Their store, FlexKV, moves the index onto the compute nodes, whose strong processors speed up lookups and caching. Three fixes keep it efficient: a hotness detector balances index load across nodes, a two-level memory scheme stretches limited compute memory, and batched remote calls cut cache coherence costs. FlexKV improved throughput by up to 2.94 times and cut latency by up to 85.2 percent versus leading disaggregated stores.

Catalog record

CREATORZhisheng Hu, Jiacheng Shen, Ming-Chang Yang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-16148

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FlexKV: Flexible Index Offloading for Memory-Disaggregated Key-Value Store Zhisheng Hu, Jiacheng Shen, Ming-Chang Yang Abstract Disaggregated memory (DM) is a promising data center architecture that decouples CPU and memory into independent resource pools to improve resource utilization. Building on DM, memory-disaggregated key-value (KV) stores are adopted to efficiently manage remote data. Unfortunately, existing approaches suffer from poor performance due to two critical issues: 1) the overdependence on one-sided atomic operations in index processing, and 2) the constrained efficiency in compute-side caches. To address these issues, we propose FlexKV , a memory-disaggregated KV store with index proxying. Our key idea is to dynamically offload the index to compute nodes, leveraging their powerful CPUs to accelerate index processing and maintain high-performance compute-side caches. Three challenges have to be addressed to enable efficient index proxying on DM, i.e. , the load imbalance across compute nodes, the limited memory of compute nodes, and the expensive cache coherence overhead. FlexKV proposes: 1) a rank-aware hotness detection algorithm to continuously balance index load across compute nodes, 2) a two-level CN memory optimization scheme to efficiently utilize compute node memory, and 3) an RPC-aggregated cache management mechanism to reduce cache coherence overhead. The experimental results show that FlexKV improves throughput by up to 2.94 × 2.94\times and reduces latency by up to 85.2 % 85.2\% , compared with the state-of-the-art memory-disaggregated KV stores. 1 Introduction Disaggregated memory (DM), a promising data center architecture proposed to improve resource efficiency, has gained significant attention from both industry [ 35 , 79 , 6 , 67 , 80 ] and academia [ 18 , 60 , 39 , 40 ] . DM breaks the boundaries of monolithic servers by decoupling CPUs and memory into the compute pool (compute nodes, CNs) and the memory pool (memory nodes, MNs), connected via high-performance networks like RDMA [ 54 , 28 ] and CXL [ 10 , 85 , 36 , 86 ] . This architecture enables CPUs and memory to be flexibly allocated, enhancing overall resource efficiency. Key-value (KV) stores are emerging applications on DM [ 62 , 33 , 25 , 65 , 51 ] . Scalability, i.e. , the ability to serve more clients without significant performance degradation, is a key requirement for all memory-disaggregated KV stores. However, existing approaches still struggle to achieve this goal due to the following two primary issues. The overdependence on one-sided atomic operations in index processing. Existing memory-disaggregated KV stores extensively rely on one-sided atomic operations ( i.e. , CAS and FAA ) to resolve concurrency conflicts during index processing [ 88 ] . Unfortunately, the extensive number of one-sided atomic operations generates a severe performance bottleneck on DMs implemented with both RDMA and CXL. Specifically, for RDMA-based DM, atomic operations with more complex semantics consume more processing power than read or write operations [ 31 ] , making the processing units of RDMA network interface cards (RNICs) a bottleneck. Meanwhile, for CXL-based DM, atomic operations hinder system performance due to the expensive cross-node cache coherence protocol ( i.e. , CXL.cache in CXL 3.0) [ 10 , 48 , 63 , 41 ] . The constrained efficiency in compute-side caches. Due to the significant performance gap between remote and local memory, using the local memory of CNs as caches is a common practice in many memory-disaggregated KV stores [ 33 , 42 , 46 ] . To reduce cross-node cache coherence overhead, existing designs either cache only the index ( i.e. , addresses of KV pairs) [ 62 , 65 , 45 ] or cache both the index and KV pairs but partition the ownership of cached data to individual CNs [ 42 , 33 ] . However, both approaches are suboptimal in terms of system performance. The former requires at least one remote memory access per request even on cache hits, while the latter introduces additional overhead from forwarding KV requests to designated nodes that own the requested data. In this paper, we propose to offload part of the index to the compute pool to address the above issues simultaneously. Specifically, by offloading the index, CNs can help with index processing, converting expensive one-sided atomic operations into local CPU operations. Furthermore, during the offloaded index processing, CNs can efficiently cache KV pairs, enhancing compute-side cache efficiency. However, three challenges have to be addressed to achieve this idea in practice. (1) The load imbalance across compute nodes. Index partitions may experience vastly different access frequencies due to workload skewness. When offloading them to CNs with static assignment, hot partitions can overload certain CNs while leaving others underutilized, degrading overall performance. Moreover, workload patterns are dynamic in production environments [ 37 , 27 , 7 , 78 ] , requiring continuous hotness tracking and dynamic reassignment. However, frequent reassignments cause significant overhead, making it challenging to balance load effectively. (2) The limited memory of compute nodes. CNs have abundant CPUs but very limited memory [ 62 , 59 , 45 ] , making it impractical to store the entire index locally in CNs, especially for large-scale production datasets [ 11 , 78 , 7 ] . Furthermore, offloading the index, whether fully or partially, to CNs introduces additional memory overhead, particularly from metadata needed for index operations. This further strains the already limited CN memory, interfering with user request processing and degrading system performance. (3) The expensive coherence overhead of caching KV pairs. When KV pairs are cached across multiple CNs, maintaining cache coherence becomes crucial for ensuring data consistency and correctness in the memory-disaggregated KV store. However, implementing a coherence protocol across network-connected CNs introduces significant overhead. First is the routine metadata maintenance overhead, i.e. , the system must continuously manage and update cache metadata, such as the cache directory. Second is the bursty write invalidation overhead, i.e. , updating a KV pair that is cached on multiple CNs triggers invalidations across those nodes. These operations substantially amplify the number of remote accesses, making caching KV pairs expensive. To address the above challenges, we propose FlexKV , a memory-disaggregated KV store with index proxying that can dynamically offload the index to the compute pool and maintain efficient compute-side caches for KV pairs. First, to address load imbalance, FlexKV adopts a rank-aware hotness detection algorithm that continuously tracks partition access frequencies and triggers reassignments only when workload imbalance becomes significant. Second, to overcome limited CN memory, FlexKV employs a two-level CN memory optimization scheme: it uses slot-resolved index RPCs to eliminate heavy key-to-slot mappings on CNs, and introduces an adaptive index-cache splitting technique that efficiently partitions the limited CN memory between index and cache to maximize performance. Finally, to reduce cache coherence overhead, FlexKV leverages an RPC-aggregated cache management mechanism that piggybacks cache metadata maintenance ( e.g. , directory updates and hotness tracking) onto index processing, amortizing metadata maintenance costs while enabling selective caching of read-intensive KV pairs. We implement FlexKV and evaluate its performance with YCSB [ 9 ] and Twitter [ 78 ] workloads. The results show that FlexKV improves throughput by up to 2.94\times and reduces latency by up to 85.2\% , compared with Clover [ 65 ] , FUSEE [ 62 ] , and Aceso [ 25 ] , three state-of-the-art memory-disaggregated

Read the complete text