DiFache : Efficient and Scalable Caching on Disaggregated Memory using Decentralized Coherence
Hanze Zhang Kaiming Wang Rong Chen Xingda Wei Haibo Chen · 2025
Read DiFache : Efficient and Scalable Caching on Disaggregated Memory using Decentralized Coherence by Hanze Zhang Kaiming Wang Rong Chen Xingda Wei Haibo Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-18013.txt.
Introduction
Hanze Zhang, Kaiming Wang, Rong Chen, Xingda Wei, and Haibo Chen present a caching system for computers that share a pool of remote memory, in a research paper from 2025. Pooling memory across machines gives flexibility, but fetching data from afar is slow. Caching hot data locally helps, yet keeping copies in sync across machines is costly.
Their system, DiFache, drops the central sync manager. Each machine can void stale copies on others by itself, since the apps already order conflicting access on their own. It also adapts how much it caches based on how often each item is read or written. On real Twitter traces, it beat rivals several-fold.
Catalog record
| CREATOR | Hanze Zhang Kaiming Wang Rong Chen Xingda Wei Haibo Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-18013
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
DiFache : Efficient and Scalable Caching on Disaggregated Memory using Decentralized Coherence Hanze Zhang Kaiming Wang Rong Chen Xingda Wei Haibo Chen Abstract The disaggregated memory (DM) architecture offers high resource elasticity at the cost of data access performance. While caching frequently accessed data in compute nodes (CNs) reduces access overhead, it requires costly centralized maintenance of cache coherence across CNs. This paper presents DiFache , an efficient, scalable and coherent CN-side caching framework for DM applications. Observing that DM applications already serialize conflicting remote data access internally rather than relying on the cache layer, DiFache introduces decentralized coherence that aligns its consistency model with memory nodes instead of CPU caches, thereby eliminating the need for centralized management. DiFache features a decentralized invalidation mechanism to independently invalidate caches on remote CNs and a fine-grained adaptive scheme to cache objects with varying read-write ratios. Evaluations using 54 real-world traces from Twitter show that DiFache outperforms existing approaches by up to 10.83 × \times (5.53 × \times on average). By integrating DiFache , the peak throughput of two real-world DM applications increases by 7.94 × \times and 2.19 × \times , respectively. Introduction The disaggregated memory (DM) architecture, which decouples CPU and memory into network-attached compute nodes (CNs) and memory nodes (MNs), is becoming increasingly popular [ 33 , 11 , 5 , 34 , 28 ] . Datacenter applications, including commercial products like PolarDB [ 11 , 45 ] , are now specifically designed for DM, enabling the independent scaling of compute and memory resources. Typically, these applications run clients on CNs that directly access data stored in MN memory through network read/write operations such as one-sided RDMA [ 39 , 47 , 29 , 28 , 34 , 46 ] . Although the DM architecture enhances resource elasticity and utilization, it significantly degrades the data access performance of applications. Specifically, DM applications must access data over the network, leading to performance bottlenecks due to network latency and bandwidth constraints [ 39 , 47 , 28 ] . For instance, when executing real-world traces from Twitter [ 44 ] , the saturation of MN bandwidth often limits throughput scaling and elevates the latency of moving data from or to MNs (see “w/o Cache” in Fig. 1 ). Caching remote data in CN memory (referred to as CN-side cache ) can effectively reduce data access overhead on DM [ 32 , 6 , 38 , 12 ] . By converting remote reads and writes to local cache accesses, the throughput scales linearly, as it eliminates data movements across the network (see “noCC Cache” in Fig. 1 ). However, this approach cannot guarantee that updates to cached data on one CN are visible to others—it lacks cross-CN cache coherence . This limitation can produce incorrect results in DM applications that require all clients to see consistent updates, such as transactional databases [ 47 , 46 ] and indexes [ 39 , 29 , 28 ] , hindering its adoption in practice. Figure 1: Peak throughput scaling of different caching schemes on DM (left) and median latency breakdown of cache operations (right). Workload: a real-world Twitter trace (No. 4) with 93% reads. Testbed: 9 CNs and 1 MN, connected with 100 Gbps RDMA NICs. An intuitive approach to maintaining cache coherence across distributed nodes is to implement CPU cache designs in software [ 25 , 9 , 11 ] . When a write occurs on one node, it invalidates cached data on other nodes, as illustrated in Fig. 2 (top). Clients attempting to read from invalidated caches experience read misses and must retrieve updated data from remote nodes. A centralized manager handles all writes and read misses to ensure linearizability of data accesses. This manager serializes data movements between CN-side caches and MNs, as well as invalidates outdated cached data. To limit invalidation costs as the number of CNs increases, the manager keeps track of nodes with caches (i.e., owners ) and invalidates only those nodes during writes. Unfortunately, when applied to DM environments, this approach provides restricted benefits from caching and suffers from performance degradation with a large number of CNs (see “CMCache” in Fig. 1 ). This happens because all cache operations are handled by a centralized manager, which becomes a significant bottleneck as the operation rate increases. Even when the manager saturates all 16 cores on a dedicated CN, the serialization cost—primarily from the queuing delay of cache operations—still accounts for nearly 90% of the median operation latency when 8 CNs are involved. Opportunity. DM applications have weak consistency requirements for remote data access because they recognize that their reads and writes occur over the network rather than the local memory bus. As a result, these applications always serialize conflicting remote data access internally rather than relying on the cache layer [ 37 , 45 , 47 , 43 , 46 , 39 , 29 , 28 , 26 , 35 , 34 , 48 ] . For example, in Fig. 3 , when accessing a DM-based tree index, concurrent remote writes are prohibited by locking, and remote reads are retried if the retrieved data is inconsistent. Key insight. The consistency model of CN-side caches for DM applications can align with MNs rather than CPU caches . This allows the clients to independently maintain cross-CN cache coherence in a decentralized design, eliminating the need for a centralized manager to serialize cache accesses. Our approach. Based on this insight, we present DiFache , an efficient, scalable and coherent CN-side caching framework for DM applications. It translates remote operations in the application to its corresponding cache operations, accelerating remote data accesses without application redesigns. The key idea of DiFache is decentralized coherence , where cache coherence is maintained independently by clients instead of a centralized manager, as shown in Fig. 2 (bottom). For reads, clients fetch data from the cache on hits or directly from MNs on misses. For writes, clients flush modified data to MNs first, and then invalidate caches on other nodes. Compared to centralized approaches, decentralized coherence eliminates serialization costs and distributes invalidation tasks across individual clients. Consequently, the performance of decentralized coherence is now largely dominated by invalidation costs—a crucial factor for achieving efficient and scalable caching on DM (see “DiFache” in Fig. 1 ). DiFache is novel in two key ways to address this challenge. Decentralized invalidation. To maintain cache coherence independently, clients must update cache metadata in the remote memory of other CNs (owners) while bypassing their CPUs. However, since cached objects are allocated and evicted dynamically, their addresses cannot be predetermined. To tackle this, each CN uses a hopscotch hash table to index cached objects, enabling address lookup with a single remote operation. Furthermore, without a centralized manager to track owners, invalidations must be broadcast to all CNs, which can cause substantial network traffic when many CNs are involved. To enhance scalability, when the number of CNs exceeds a threshold, clients record owners in remote bitmaps during read misses to avoid unnecessary invalidations. Fine-grained adaptive caching. Enabling CN-side cache significantly improves read performance but introduces inevitable cache coherence costs—primarily invalidation costs in DiFache —for writes, as shown in Fig. 1 (right). Therefore, when writes dominate the