Vulcan : Instance-Optimal Systems Heuristics Through LLM-Driven Search
Rohit Dwivedula, Divyanshu Saxena, Sujay Yadalam, Daehyeok Kim, Aditya Akella · 2025
Read Vulcan : Instance-Optimal Systems Heuristics Through LLM-Driven Search by Rohit Dwivedula, Divyanshu Saxena, Sujay Yadalam, Daehyeok Kim, Aditya Akella (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-25065.txt.
Introduction
Rohit Dwivedula, Divyanshu Saxena, Sujay Yadalam, and others present Vulcan, a framework that uses language models to synthesize systems heuristics focused to each deployment copy. Resource management in operating systems and networks relies on hand-designed heuristics, but varied hardware and workloads make manual focus costly. Generated heuristics must run safely, integrate correctly, and still perform well.
Vulcan defines language-model-friendly surfaces that isolate core choice logic from surrounding system code, then searches for verifiable heuristics through that surface. Systems readers get automated heuristic design with safety checks. Instance focus could replace one-size-fits-all tuning across varied fleets.
Catalog record
| CREATOR | Rohit Dwivedula, Divyanshu Saxena, Sujay Yadalam, Daehyeok Kim, Aditya Akella |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 85 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2512-25065
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Vulcan : Instance-Optimal Systems Heuristics Through LLM-Driven Search Rohit Dwivedula, Divyanshu Saxena, Sujay Yadalam, Daehyeok Kim, Aditya Akella Abstract Resource-management tasks in modern operating and distributed systems continue to rely primarily on hand-designed heuristics for tasks such as scheduling, caching, or active queue management. Designing performant heuristics is an expensive, time-consuming process that we are forced to continuously go through due to the constant flux of hardware, workloads and environments. We propose a new alternative: synthesizing instance-optimal heuristics – specialized for the exact workloads and hardware where they will be deployed – using code-generating large language models (LLMs). To make this synthesis tractable, Vulcan separates policy and mechanism through LLM-friendly , task-agnostic interfaces. With these interfaces, users specify the inputs and objectives of their desired policy, while Vulcan searches for performant policies via evolutionary search over LLM-generated code. This interface is expressive enough to capture a wide range of system policies, yet sufficiently constrained to allow even small, inexpensive LLMs to generate correct and executable code. We use Vulcan to synthesize performant heuristics for cache eviction and memory tiering, and find that these heuristics outperform all human-designed state-of-the-art algorithms by upto 69% and 7.9% in performance for each of these tasks respectively. 1 Introduction Systems research has long treated heuristic design as a manual craft. Performance-critical systems rely on hand-written heuristics – tuned for typical conditions and deployed as static, one-size-fits-all policies. However, decades of work across caching [ 126 , 19 ] , congestion control [ 77 ] , kernel queueing disciplines [ 94 ] , memory tiering [ 109 ] , and more – demonstrate the same lesson: there is no universal heuristic. Different settings demand different heuristics, and specialization is essential for performance. So, we continually tune and redesign (§ 2.1 ): we tweak congestion control algorithms for new network environments [ 104 ] , tailor prefetching or caching policies for emerging workloads [ 16 ] or hardware [ 116 ] , and evolve queueing disciplines for new performance targets [ 24 ] . Yet despite this retuning, we continue to find instances of sub-optimal performance from these manually-designed heuristics [ 57 , 8 , 89 ] . The heuristic design space continually shifts with context-dependent behaviors and changing performance trade-offs, making it increasingly difficult to discover and maintain the ‘right’ ( i.e., performant) heuristics for each scenario. This raises a fundamental question: how can we discover “the right heuristics” fast enough , to keep pace with ever-evolving deployment conditions? We aim to tackle this ever-lasting challenge using Vulcan , a framework in which we recast the heuristic design problem as an an automated search problem using large language models (LLMs): instead of hand-crafting heuristics, we repeatedly invoke generative models to produce instance-optimal heuristics tailored to each deployment context. Our framework is inspired by recent advances in LLM-powered algorithm synthesis [ 86 , 71 , 18 ] , which combine generative models with evolutionary search to discover expressive code that maximizes a target quantitative reward function. Directly applying these search-based techniques to synthesize systems heuristics, however, is non-trivial (§ 2.3 ). Modern systems heuristics tightly couple policy (high-level decision logic) with mechanism (low-level state, data structures, and control paths). As a result, modifying or improving a heuristic requires coordinated reasoning about both intent and implementation. In this setting, simply prompting an LLM to synthesize a complete end-to-end heuristic proves futile (see § 2.3 ). On the other hand, constraining the search to tiny tweaks on top of known algorithms [ 35 ] limits the opportunity for genuine innovation. The central challenge, therefore, is identifying how to leverage generative models where they are strongest today – reasoning over structured signals and expressing high-level decision logic – without requiring them to correctly implement complex mechanisms or manage intricate system state. Vulcan resolves this tension by placing LLM-driven evolutionary search at the core of heuristic design, while exposing a simple, structured three-step pipeline ( Figure 3 in § 3 ) that cleanly separates policy from mechanism and makes the search tractable. In the first step of the pipeline, users define the search space by casting the problem into a narrow interface (§ 3.1 ). Our key observation is that many systems heuristics can be expressed in one of two forms: (i) a function that computes a value from system state (e.g., congestion controllers computing a congestion window), which we term a Value -type heuristic; or (ii) a function that ranks a set of objects and selects among them (e.g., schedulers ranking runnable tasks), which we term a Rank -type heuristic. This interface captures the policy logic cleanly. The corresponding mechanism – the concrete implementation that executes these decisions – is developed separately. This separation exposes a rich mechanism design space for both policy interfaces, which we systematically explore in this paper (§ 3.1.1 and § 3.1.2 ). Second, users define the instance for which they seek a specialized heuristic – either by specifying a concrete workload–hardware pair, or by configuring an automated instance generator (§ 3.2 ) that algorithmically delineates instances based on observed workload characteristics and system signals. Today, heuristics are redesigned only when workloads or objectives change substantially, not because smaller changes are unimportant, but because manual heuristic design is expensive and time-consuming; consequently, even redesigned heuristics are typically made as broadly applicable as possible. Vulcan fundamentally changes this cost model: by dramatically reducing the human cost of heuristic creation, it becomes practical to synthesize heuristics for much more narrower instances, enabling specialization even for modest shifts such as changes in input parameters, evolving access patterns, or transient workload phases. In effect, Vulcan enables instance-optimal policies : rather than relying on coarse, one-size-fits-all heuristics that underperform across heterogeneous conditions, specialization becomes the default. Finally, users specify how to invoke the evolutionary search process (§ 3.3 ). This includes describing the system context, objectives, and available state in natural language, and, critically, defining the evaluation harness used to assess candidate heuristics. As with earlier stages of the pipeline, this step exposes important design trade-offs, such as balancing evaluation speed against fidelity, choosing appropriate metrics of interest, and combining multiple objectives into a single optimization target. We discuss these considerations and illustrate, through concrete examples, how to effectively invoke the search process. In this paper, we instantiate Vulcan for two policies: cache eviction (§ 4 ) and page promotion in tiered-memory systems (§ 5 ). We discuss specific choices we made for both of these policies in the design space exposed by the aforementioned three-step process. Across ten different instances , cache eviction policies synthesized by Vulcan either nearly match state-of-the-art baselines or outperform them by 1.94% to 69%. Memory tiering policies designed with Vulcan yield 2.5-7.9% improvement over state-of-the-art baselines, for a variety of workloads. While Vulcan automates the search for effective heuristics, it does not obviate the role of human designers. Our position is that human expertise is most valuable not in hand-crafting policy minutiae, but in structuring the problem so algorithmic search can succeed. We argue that this structure arises from defining clean interfaces that control the space in which heuristics are discovered, providing the scaffolding that makes LLM-guided design both tractable and meaningful. Conversely, Vulcan advocates