Revisiting Page Migration In Main-Memory Databases For Modern Hardware
Yeasir Rayhan and Walid G. Aref · 2025
Read Revisiting Page Migration In Main-Memory Databases For Modern Hardware by Yeasir Rayhan and Walid G. Aref (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17685.txt.
Introduction
Yeasir Rayhan and Walid Aref revisit how in-memory databases move data pages between memory nodes on modern servers. New hardware mixes fast and slower memory tiers, and a database only gains if it can shift hot pages to fast memory as workloads change. Current systems leave that shifting to the operating system, which knows nothing about the workload. The authors propose a custom system call that allows partial moves and exposes settings the database can tune. Their paper appeared in twenty twenty-five.
In tests with a common tree index and a standard-style workload, the custom call raised query throughput by more than double at best. It also moved far more pages than the built-in Linux call. The authors argue that giving the database direct control over page movement is the better design for current hardware.
Catalog record
| CREATOR | Yeasir Rayhan and Walid G. Aref |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 60 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-17685
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Revisiting Page Migration In Main-Memory Databases For Modern Hardware Yeasir Rayhan and Walid G. Aref Abstract . Modern hardware architectures, e.g., NUMA servers, chiplet processors, tiered and disaggregated memory systems have significantly improved the performance of Main-Memory Databases, and are poised to deliver further improvements in the future. However, realizing this potential depends on the database system’s ability to efficiently migrate pages among different NUMA nodes, and/or memory chips as the workload evolves. Modern Main-Memory Databases offload the migration procedure to the operating system without accounting for the workload and its migration characteristics. In this paper, we propose a custom system call move_pages2 as an alternate to Linux’s own move_pages system call. In contrast to the original move_pages, move_pages2 allows partial migration and exposes two configuration knobs, enabling a Main-Memory Database tailor the migration process to its specific requirements. Experiments on a main-memory B + -Tree for a YCSB-like workload show that the proposed move_pages2 custom system call improves the B + -Tree query throughput by up to 2.3 × \times and migrates up to 2.6 × \times more memory pages, outperforming the native Linux system call on modern hardware architectures. † † authors: . PVLDB Reference Format: PVLDB, 14(1): XXX-XXX, 2020. doi:XX.XX/XXX.XX † † This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing [email protected] . Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. 14, No. 1 ISSN 2150-8097. doi:XX.XX/XXX.XX 1. Introduction Main-Memory Databases (MMDB, for short) ( Faerber et al., 2017 ) have emerged as a promising alternative to conventional disk resident databases in order to mitigate the performance bottlenecks associated with disk I/O. By storing the entire database in main memory, an MMDB eliminates the need to access disks altogether. Despite early efforts, e.g., ( Gawlick and Kinkade, 1985 ; Lehman and Carey, 1986b ; Lehman and Carey, 1986a ; Lehman and Carey, 1987 ) , MMDBs have gained widespread traction only in recent years, mainly due to the decreasing cost and increasing capacity of main memory. This has led to full-fledged systems in both academia (e.g., H-Store ( Stonebraker et al., 2007 ; Kallman et al., 2008 ) , HyPeR ( Neumann et al., 2015 ) , Silo ( Tu et al., 2013 ) , Hyrise ( Grund et al., 2010 ; Dreseler et al., 2019 ) ), and industry (e.g., Hekaton ( Diaconu et al., 2013 ) , VoltDB ( Stonebraker and Weisberg, 2013 ) , Oracle TimesTen ( Lahiri et al., 2013 ) , SAP HANA ( Sikka et al., 2013 ) ). These MMDBs are built upon legacy hardware, based on the primary assumptions that the memory access latency is uniform and that the memory is tightly coupled to the compute cores. Page migration holds little to no value in these legacy hardware architectures, and as a result has largely been neglected in MMDBs until now. However, in the past decade, the hardware landscape has evolved rapidly with the introduction of NUMA architectures, chiplet-based processor designs ( amd, 2025a ; amd, 2025b ; amd, 2025c ; int, 2025 ) , and emerging interconnect technologies, e.g., CXL ( CXL, 2025 ) , OpenCAPI ( oca, 2025 ) , and CCIX ( cci, 2025 ) . These hardware developments invalidate the earlier assumptions of uniform memory access latency and tight coupling of memory and compute chips. In NUMA- and chiplet-based architectures, memory access latency is non-uniform. On top of that, the introduction of CXL-like interconnect technology has introduced memory configurations where memory is loosely coupled, i.e., a tiered-memory architecture, and is even decoupled, i.e., a disaggregated memory architecture, from the compute chip. The dominant theme underlying these new hardware advancements is the introduction of heterogeneity in hardware, and thus leading to variable access latencies across different hardware chips. For example, access to remote-socket memory in a NUMA server can be 4\times more expensive than accessing local-socket memory ( Bang et al., 2022 ; Bang et al., 2020a ) . In a CXL-enabled tiered memory architecture, accessing CXL memory can be 5\times more expensive than accessing local DRAM memory ( Liu et al., 2025 ) . For optimal performance on a NUMA server, data must be strategically placed close to the compute core that initiates the data request. This minimizes cross-socket data movement. Additionally, as the workloads evolve and the working sets shift, data must be continuously moved around among the NUMA nodes in an online manner to sustain optimal DBMS performance. For example, SAP-HANA ( sap, 2025 ) , a commercial main-memory column-store, demonstrates that an adaptive data-placement strategy ( Psaroudakis et al., 2016 ) can yield up to 4\times performance improvement in terms of query throughput. The same principle also applies to tiered memory architectures. Data needs to be continuously moved around so that hot data resides in the closest memory, i.e., local DRAM, and cold data resides in the farthest memory, i.e., CXL memory ( Hao et al., 2024 ) . To support this dynamic relocation of data, modern MMDBs ( Hao et al., 2024 ; Riekenbrauck et al., 2024 ; Fogli et al., 2024a ) often rely on the support of the operating system in the form of on-demand page migration that is the de facto standard for moving memory pages across NUMA nodes and volatile memory tiers. Thus, the efficiency of page migration is critical to achieving optimal performance in modern hardware architectures. For instance, the three-tier buffer pool manager ( Riekenbrauck et al., 2024 ) comprising DRAM, CXL Memory, and Disk in Hyrise ( Dreseler et al., 2019 ) , identifies page migration as a key bottleneck in a tiered buffer pool, that can cause up to 2 \times performance degradation. This further validates the importance of page migration in a Main-Memory Database in the context of modern hardware, and highlights the need for greater attention from the research community as we enter the next phase of main-memory database design for modern hardware. Linux ( Torvalds, 2025 ) handles page migration through the system call move_pages ( Kerrisk, 2025 ) . move_pages accepts as input a list of page addresses along with their corresponding destination NUMA nodes (memory tiers). It alters the physical address of a page by moving it to its destination NUMA node (memory tier) without changing the page’s virtual address. A typical page migration involves several key steps. 1 The user space application invokes a move_pages system call, and traps to the kernel to handle the migration; 2 The page is locked, and the page table entry (PTE, for short, is an entry that maps a virtual address to a physical address in a process’s page table) of the migrating page is marked as a migration entry to ensure exclusive access during migration; 3 A TLB (short for Translation Lookaside Buffer that caches recent virtual-to-physical address translations) shootdown is issued to invalidate all PTEs of the migrating page across the TLBs of all the computing cores; 4 A new page is created, and is locked; 5 The contents of the old page