Jenga: Responsive Tiered Memory Management without Thrashing

Rohan KadekodiUniversity of Washington, Haoran PengUniversity of WashingtonGilbert BernsteinUniversity of WashingtonMichael D. ErnstUniversity of WashingtonBaris KasikciUniversity of Washington · 2025

Read Jenga: Responsive Tiered Memory Management without Thrashing by Rohan KadekodiUniversity of Washington, Haoran PengUniversity of WashingtonGilbert BernsteinUniversity of WashingtonMichael D. ErnstUniversity of WashingtonBaris KasikciUniversity of Washington (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-22869.txt.

Introduction

Tiered memory mixes fast memory with slower, larger memory. Rohan Kadekodi leads a team with four co-authors. In 2025 they present Jenga, a tiered memory manager. Prior systems suffered two flaws. Hot and cold objects shared the same pages. Hotness shifts also caused thrashing, or wasteful page swaps.

Jenga allocates pages by allocation context. That separates hot from cold objects early. Its hotness measures react fast to change. Yet they avoid thrashing in tests. Ten memory heavy apps ran faster than with the best prior system. Gains were large when fast tier size matched working set. Overhead stayed under a few percent.

Catalog record

CREATORRohan KadekodiUniversity of Washington, Haoran PengUniversity of WashingtonGilbert BernsteinUniversity of WashingtonMichael D. ErnstUniversity of WashingtonBaris KasikciUniversity of Washington
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE59 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-22869

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Jenga: Responsive Tiered Memory Management without Thrashing Rohan KadekodiUniversity of Washington, Haoran PengUniversity of WashingtonGilbert BernsteinUniversity of WashingtonMichael D. ErnstUniversity of WashingtonBaris KasikciUniversity of Washington Abstract A heterogeneous memory has a single address space with fast access to some addresses (a fast tier of DRAM) and slow access to other addresses (a capacity tier of CXL-attached memory or NVM). A tiered memory system aims to maximize the number of accesses to the fast tier via page migrations between the fast and capacity tiers. Unfortunately, previous tiered memory systems can perform poorly due to (1) allocating hot and cold objects in the same page and (2) abrupt changes in hotness measurements that lead to thrashing. This paper presents Jenga, a tiered memory system that addresses both problems. Jenga’s memory allocator uses a novel context-based page allocation strategy. Jenga’s accurate measurements of page hotness enable it to react to memory access behavior changes in a timely manner while avoiding thrashing. Compared to the best previous tiered memory system, Jenga runs memory-intensive applications ∼ \sim 28% faster across 10 applications, when the fast tier capacity matches the working set size, at a CPU overhead of <3% of a single core and a memory overhead of <0.3%. * * footnotetext: Both authors contributed equally. 1 Introduction Main memory cost and capacity poses a challenge in the face of data explosion. Memory is one of the most expensive resources in datacenter servers, amounting to 37% of Meta’s server costs [ 35 ] and about 50% of Microsoft Azure’s server costs [ 5 ] . Furthermore, DRAM technologies do not scale in capacity to accommodate the growing needs of modern memory-intensive applications such as large ML models and graph processing frameworks [ 18 ] . In order to reduce the cost of server memory, industry is innovating on memory expanders that use cache-coherent interconnects such as Compute eXpress Link (CXL) [ 14 ] . This enables the use of cheaper and denser non-DRAM memory technologies such as NVM (Non-Volatile Memory [ 25 ] ) or older generation DDR4 DRAM chips on the PCIe5.0 bus (PCI latency is more than memory bus latency). The memory is directly addressable, sharing a single space with DDR5 DRAM on the memory bus. A heterogeneous memory contains a fast tier of DRAM and a slower but larger capacity tier . Memory tiering enables a server to access terabytes of memory in an affordable manner [ 6 , 31 , 35 , 17 ] . The higher access latency of the capacity tier can significantly degrade performance [ 45 , 25 ] . In order to hide this latency, a tiered memory system in the OS migrates memory pages between the fast and capacity tiers. The goal of a tiered memory system is to maximize accesses to the fast tier (called the hit rate) while minimizing the number of page migrations between the fast and capacity tiers. This paper begins with a comprehensive analysis of 10 widely used applications previously used in evaluations of tiered memory systems. We categorize these applications based on their data allocation patterns (do they predominantly create small or large objects?) and their data access behaviors (is the hot data stable or does the hot data exhibit phase changes?). We find that previous tiered memory systems [ 29 , 35 , 40 , 50 , 42 , 17 , 31 , 52 , 54 , 3 ] perform well only on applications that create large objects and where the hot data remains stable. To understand this behavior, we analyzed previous tiered memory systems and discovered two shortcomings that lead to their poor performance. First, these systems allocate hot and cold data in the same memory page. This can prevent all the pages with hot data from fitting in the fast tier, resulting in a poor hit rate. Second, current tiered memory systems are unable to accurately determine page hotness. Some systems focus solely on recent access patterns while ignoring long-term historical trends [ 35 , 50 , 3 , 54 , 31 ] , resulting in poor hit rates. Other systems attempt to handle changing workload phases by abruptly changing page hotness at fixed intervals [ 29 , 40 , 42 , 17 , 43 ] , but this approach creates a performance trade-off: longer intervals cause slow adaptation to new hot data patterns, while shorter intervals trigger excessive and counterproductive data migrations as pages repeatedly move between tiers, both scenarios ultimately harming overall system performance. This paper introduces Jenga, which outperforms other tiered memory systems on applications with diverse memory access behaviors. Jenga’s two main components are a context-based heap allocator and smooth hotness tracking . Context-based allocator Jenga is the first tiered-memory system with a context-based heap allocator for allocating an application’s objects into pages in a way that is profitable for their migration to the appropriate memory tier. The three allocation strategies are the traditional size-based and time-based strategies, plus a new context-based grouping strategy that we introduce. Context-based allocation places objects allocated at the same stack trace and that are of the same size, into the same page. Objects with the same call stack may be semantically related and thus may be accessed together and/or a similar number of times. For instance, a tree implementation might have different call stacks when allocating internal nodes versus leaf nodes. When performing tree traversals, internal nodes are visited much more often than the leaf nodes. Smooth hotness tracking In order to adapt to phase changes in the hot data, Jenga introduces smooth hotness tracking . Rather than periodically abruptly changing the relative hotness of pages, Jenga applies a gradual and continuous hotness decay with every page access. Smooth hotness reduces the influence of hard-coded intervals, allowing a page’s hotness to change smoothly and responsively over time. Pages from no-longer-hot regions are migrated to the capacity tier without waiting for a lengthy interval, while frequently accessed pages in the current phase retain their hotness and continue to remain in the fast tier. This smooth hotness tracking approach reduces unnecessary migrations due to thrashing, increases the hit rate by considering historical access trends and promptly responding to phase changes, and ultimately improves overall performance. Smooth hotness tracking is inexpensive: it requires only one additional counter per page and little computation. Our contributions Analyses: We analyzed the behavior of previous tiered memory systems with 10 memory-intensive applications and reveal new findings: (1) allocation of objects within pages influences the performance of applications on tiered memory, and (2) today’s tiered memory systems are not timely enough to react to some changes in access patterns, and they over-react to other changes. Jenga design: Our Jenga tiered memory system solves the above problems using novel context-based allocation and smooth hotness tracking approaches. Evaluation: Jenga has small CPU overhead (<3% of a single CPU core) and memory overhead (<0.3%). We compared Jenga to 5 previous tiered-memory systems while varying the size of the fast tier and capacity tier. Jenga runs memory-intensive applications \sim 28% faster than the best other system when the fast tier capacity matches the working set of the application. Furthermore, Jenga is the only system that consistently achieves high performance in

Read the complete text