On Usage of Non-Volatile Memory as Primary Storage for Database Management Systems

Naveed Ul Mustafa, Adrià Armejach, Ozcan Ozturk, Adrián Cristal, Osman S. Unsal · 2025

Read On Usage of Non-Volatile Memory as Primary Storage for Database Management Systems by Naveed Ul Mustafa, Adrià Armejach, Ozcan Ozturk, Adrián Cristal, Osman S. Unsal (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-09431.txt.

Introduction

Naveed Ul Mustafa and four co-authors test what happens when a database lives in memory that never forgets, in this 2025 paper. The catalog title carries a funding footnote; the head's clean title governs. Non-volatile memory keeps data without power while offering byte-level access, blurring the line between storage and RAM.

As a case study the team rebuilt PostgreSQL's storage engine for such memory and measured it on an emulation platform. Query times fell by up to forty-five percent against disk and thirteen percent against unmodified non-volatile setups. The gains were capped by a subtlety: data sat far from the processors when needed, causing long stalls. Their answer is a helper-thread library that prefetches data through a simple interface, pushing improvements to fifty-four and seventeen percent. The authors map the changes other database systems would need.

Catalog record

CREATORNaveed Ul Mustafa, Adrià Armejach, Ozcan Ozturk, Adrián Cristal, Osman S. Unsal
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-09431

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On Usage of Non-Volatile Memory as Primary Storage for Database Management Systems Thanks: The research leading to these results has received funding from the European Union's 7th Framework Programme under grant agreement number 318633, the Ministry of Science and Technology of Spain under contract TIN2015-65316-P, and a HiPEAC collaboration grant awarded to Naveed Ul Mustafa. Naveed Ul Mustafa, Adrià Armejach, Ozcan Ozturk, Adrián Cristal, Osman S. Unsal Abstract This paper explores the implications of employing non-volatile memory (NVM) as primary storage for a data base management system (DBMS). We investigate the modifications necessary to be applied on top of a traditional relational DBMS to take advantage of NVM features. As a case study, we modify the storage engine (SE) of PostgreSQL enabling efficient use of NVM hardware. We detail the necessary changes and challenges such modifications entail and evaluate them using a comprehensive emulation platform. Results indicate that our modified SE reduces query execution time by up to 45% and 13% when compared to disk and NVM storage, with average reductions of 19% and 4%, respectively. Detailed analysis of these results shows that while our modified SE is able to access data more efficiently, data is not close to the processing units when needed for processing, incurring long latency misses that hinder the performance. To solve this, we develop a general purpose library that employs helper threads to prefetch data from NVM hardware via a simple API. Our library further improves query execution time for our modified SE when compared to disk and NVM storage by up to 54% and 17%, with average reductions of 23% and 8%, respectively. ∎ 1 Introduction The traditional design of a DBMS assumes a memory hierarchy where datasets are stored in disks. Disks are a cheap and non-volatile storage medium suitable for storing large datasets. However, they are extremely slow for data retrieval. To hide their high data-access latency, DRAM is used as an intermediate storage between disks and the processing units. DRAM is orders of magnitude faster than a disk, and with increasing DRAM chip densities and decreasing memory prices, relational in-memory DBMSs have become increasingly popular [ 2 , 4 , 5 , 3 , 6 , 7 ] . Significant components of in-memory DBMSs, like index structures [ 8 , 9 ] , recovery mechanisms for system failure [ 10 , 11 ] , and commit processing [ 12 ] are tailored towards the usage of main memory as primary storage. An in-memory DBMS assumes that all data fits in main memory. However, for some applications, the database size can grow larger than DRAM capacity [ 13 ] . At the same time, inherent physical limitations related to leakage current and voltage scaling limit the further scaling of DRAM [ 15 , 14 ] . The capacity problem can be resolved by distributing the database across multiple machines but at the cost of performance degradation [ 13 ] . Furthermore, due to the volatile nature of DRAM, in-memory DBMSs still use a large pool of disks to provide a form of persistent storage for critical or non redundant data [ 2 , 16 , 17 , 18 ] . Non-volatile memory (NVM) is an emerging storage class technology that features persistency as well as significantly faster access latencies than hard disks, with read latencies on the same order of magnitude as DRAM [ 19 ] . It also offers byte-addressability like DRAM and higher density [ 20 , 21 ] . Prominent NVM technologies are PC-RAM 1 1 1 PC-RAM: Phase Change Random Access Memory [ 22 ] , STT-RAM 2 2 2 STT-RAM: Spin Transfer Torque Random Access Memory [ 14 ] , and R-RAM 3 3 3 R-RAM: Resistive Random Access Memory [ 23 ] . With read latency close to that of DRAM, especially in case of PC-RAM and R-RAM [ 24 , 25 ] , NVM technologies are a good candidate to improve the performance of decision support systems (DSS), which are dominated by read-only queries on vast datasets [ 26 ] . DSS are computer technology solutions that can be used to facilitate complex decision making [ 27 ] . They convert business information into tangible results [ 28 ] to help executives take knowledge-based decisions. A DBMS design should take into account the characteristics of NVM to benefit from its features. Simple ports of a traditional DBMS - designed to use disks as the primary storage medium - to NVM will show improvement due to the lower access latencies of NVM. However, adapting a DBMS to fit NVM characteristics can offer a number of benefits beyond lower access latencies. This adaptation requires modifications in the storage engine as well as other components of a DBMS [ 19 ] in order to take advantage of NVM features. In this paper, we study the implications of employing NVM in the design of a DBMS. Our main contributions can be summarized as follows: • We discuss and provide insights on the different available options when including NVM into the memory hierarchy of current systems. • We focus on investigating the necessary changes and challenges when modifying an existing, well-tested, widely used, and robust traditional DBMS to benefit from NVM features. As a case study, we selected PostgreSQL which is ranked as the 4 th most popular DBMS [ 29 ] . • Our modifications aim at providing fast access to data by bypassing the slow disk interfaces while maintaining all the functionalities of a robust DBMS such as PostgreSQL. Our modified SEs target read-dominant DSS queries, providing performance improvements by minimizing data movement operations in a traditional disk-based DBMS. • We evaluate our proposed modified SEs of PostgreSQL using a comprehensive emulation platform and the TPC-H [ 30 ] benchmark. In addition, we also evaluate an unmodified version of PostgreSQL using both a high-end solid state disk and the emulated NVM hardware. • We identify and quantify performance bottlenecks that appear when employing NVM hardware. To further improve the performance of our NVM-enabled SEs, we design and implement a general purpose data prefetching library based on helper threads that tackles the identified performance bottlenecks. Experimental results show that our modified SEs are able to reduce the kernel execution time, where file I/O operations take place, from around 10% to 3% on average. In terms of wall-clock query execution time, our modifications improve performance by around 19% and 4% on average when compared to unmodified PostgreSQL on disk and on NVM storage, respectively. We find that the performance of our modified SE is limited by the fact that data is not close to the processing units when needed for query processing since it is directly accessed from NVM hardware. This leads to long latency user-level cache misses that decrease the improvements achieved by avoiding expensive data movement operations. We employ a known technique like helper-threads for data prefetching in the context of a NVM-based database design to resolve the data readiness problem we identify. We implement a general purpose data prefetching library that exposes a simple API allowing the creation of a user-specified number of helper threads that seamlessly prefetch data for a given address and memory size with

Read the complete text