Exploring Next Token Prediction For Optimizing Databases
Yeasir Rayhan and Walid G. Aref · 2025
Read Exploring Next Token Prediction For Optimizing Databases by Yeasir Rayhan and Walid G. Aref (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-19619.txt.
Introduction
Yeasir Rayhan and Walid Aref ask whether the trick behind chatbots can also tune databases, in a 2025 paper. Large language models learn by predicting the next token in a stream, and that skill transfers to many tasks. Database tuning is a different world, though, built on costs and hardware rather than words.
The pair built a framework called Probe and Learn to bridge the two. It feeds the model tokens drawn from hardware signals and trains it with a decision-focused method. As a test case, they used it to schedule data indexes on many-core servers for in-memory databases. Early results showed better speed and better transfer to new cases than standard tuning. The authors present the work as a first proof that prediction-based learning can steer database internals.
Catalog record
| CREATOR | Yeasir Rayhan and Walid G. Aref |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-19619
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring Next Token Prediction For Optimizing Databases Yeasir Rayhan and Walid G. Aref Abstract . The Next Token Prediction paradigm (NTP, for short) lies at the forefront of modern large foundational models that are pre-trained on diverse and large datasets. These models generalize effectively, and have proven to be very successful in Natural Language Processing (NLP). Inspired by the generalization capabilities of Large Language Models (LLMs), we investigate whether the same NTP paradigm can be applied to DBMS design and optimization tasks. Adopting NTP directly for database optimization is non-trivial due to the fundamental differences between the domains. In this paper, we present a framework, termed Probe and Learn (PoLe), for applying NTP to optimize database systems. PoLe leverages Decision Transformers and hardware-generated tokens to effectively incorporate NTP into database systems. As a proof of concept, we demonstrate PoLe in the context of the index scheduling task over NUMA servers in main-memory database systems. Preliminary results for this scheduling task demonstrate that adopting NTP and PoLe can improve both performance and generalizability. 1. Introduction Hardware and workload are the two dominant factors that have shaped DBMS research over the past 50 years, and are termed the “Game Changers” in databases ( Ailamaki 2021 ) . Both factors are rapidly evolving, and are consistently reshaping the design principles and optimization techniques of modern DBMSs. On the hardware front, recent examples include designs and optimizations on GPUs, e.g., ( Boeschen and Binnig 2022 ; Mageirakos et al. 2022 ; Sioulas et al. 2019 ; Chrysogelos et al. 2019 ; He and Yu 2011 ; Breß 2013 ) , Persistent Memory (PMEM), e.g., ( Zuo et al. 2018 ; van Renen et al. 2018 ; Zhou et al. 2021 ; Arulraj et al. 2015 ) , SSDs, e.g., ( Tsirogiannis et al. 2009 ; Shah et al. 2008 ; Leis et al. 2018 ; Haas and Leis 2023 ) , Compute Express Link (CXL), e.g., ( Ahn et al. 2022 ; Lee et al. 2024 ; Lerner and Alonso 2024 ; Ahn et al. 2024 ) and NUMA servers, e.g., ( Teubner and Müller 2011 ; Albutiu et al. 2012 ; Li et al. 2013 ; Balkesen et al. 2013 ; Lang et al. 2013 ; Schuh et al. 2016 ; Porobic et al. 2012 ; Porobic et al. 2014 ; Leis et al. 2014 ; Wagner et al. 2021 ; Psaroudakis et al. 2015 ; Psaroudakis et al. 2016 ) . Recently, ARM processors have made significant progress in high-performance computing, and are becoming competitive with the x86 processors with high-end servers, e.g., Apple Silicon ( Wikipedia 2024 ) , Amazon Graviton ( Amazon 2024b ) , and NVIDIA Grace ( NVIDIA 2024 ) . On the workload front, the landscape is equally dynamic, and has taken a drastic turn over the recent years, with the rise of the Large Language Model (LLM, for short) workloads. LLM workloads impose distinct computational and memory demands, further pushing the boundaries of DBMS design and optimization ( Liu et al. 2024 ) . Another game-changing factor is the heterogeneity of both the hardware and the workloads. On the hardware front, this heterogeneity manifests itself in a variety of compute and memory chips. For example, modern servers are equipped with large cores, small cores, or a mixture of both. Larger cores are preferable for serialized execution, whereas parallelization favors smaller cores. Likewise, nowadays the same Integrated Memory Controllers (IMCs, for short) support both DRAM and PMEM. In addition, the heterogeneity leads to variable read and write access latencies across hardware, e.g., in Non Uniform Memory Access (NUMA) servers accessing data that resides in a remote DRAM chip can incur up to 4\times more latency than accessing data that resides in the local DRAM chip ( Bang et al. 2020 ) . Accessing the PMEM can be 3\times slower than accessing a local DRAM chip ( Izraelevitz et al. 2019 ) . On the workload front, this heterogeneity stems from the diverse range of applications that DBMSs must support, including OLTP, OLAP, HTAP, and modern ML workloads. Each of these hardware and workload applications has distinct performance characteristics that require customized optimization strategies. Moreover, the resurgence of cloud databases has taken over traditional on-site DBMSs. Amazon Elastic Compute Cloud (EC2) ( Amazon 2024a ) provides as many as 750 instance choices equipped with a diverse range of processors, memory, storage, and networks hardware to fit different workload applications. To address these challenges, DBMS engines need to generalize across heterogeneous hardware and workload applications without sacrificing performance. We need a complete paradigm shift in how DBMSs are designed; one that embraces hardware and workload heterogeneity to gain the best performance, while being able to generalize and adapt across new hardware architectures and workload applications without manual intervention. Rather than constantly reinventing the wheel to keep pace with the ever-changing hardware and workload landscape, the goal is to distill decades of DBMS knowledge for diverse workloads and hardware into a compact, transferable form that is both generalizable and adaptable. Large Language Models (LLMs) ( Brown et al. 2020 ; Radford et al. 2019 ; Radford and Narasimhan 2018 ; Touvron et al. 2023 ; OpenAI 2023 ; Bi et al. 2024 ) have proven to be very successful in generalization tasks across multiple fields, including but not limited to, Natural Language Processing (NLP), Computer Vision, and Robotics. Next Token Prediction (NTP) lies at the core of LLMs. It is a foundational concept in modern AI that involves training models to predict the next element in a sequence based on the preceding context. By mastering NTP, LLMs are responsible for a major share of all AI breakthroughs in many tasks, e.g., machine translation, text summarization, and code generation. It is only natural to question how this simple yet powerful NTP paradigm can be applied to DBMS design and optimizations, and whether the NTP Paradigm can help in building data systems that are capable of generalizing across diverse hardware and workload applications while maintaining high performance. In this paper, we examine the feasibility of applying the Next Token Prediction paradigm (NTP) in the context of database systems. We address the challenges associated with adopting this paradigm for database systems, and identify two building blocks, namely, Decision Transformers (DT, for short) and a new type of tokens that we term, DB-tokens. DB-tokens are hardware profiles that are essential in translating NTP to DBMS optimizations . We propose a general framework, Probe and Learn (PoLe, for short) for database systems to effectively incorporate the NTP paradigm , and evaluate its effectiveness through a case-study on the main-memory index scheduling task. Preliminary results on the main-memory index scheduling demonstrate that the PoLe framework can learn novel scheduling techniques improving the index performance in query throughput by up to 3 \times . The rest of this paper proceeds as follows. Section 2 discusses the Next Token Prediction paradigm, and how it is non-trivial to translate the Next Token Paradigm to DBMSs. Section 3 presents the core building blocks to enable the Next Token Prediction paradigm for database systems. Section 4 introduces the PoLe framework. To demonstrate the effectiveness of PoLe, in Sections 5 and 6