Phaedrus: Predicting Dynamic Application Behavior with Lightweight Generative Models and LLMs

Bodhisatwa Chatterjee, Neeraj Jadhav, Santosh Pande · 2024

Read Phaedrus: Predicting Dynamic Application Behavior with Lightweight Generative Models and LLMs by Bodhisatwa Chatterjee, Neeraj Jadhav, Santosh Pande (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06994.txt.

Introduction

Bodhisatwa Chatterjee, Neeraj Jadhav, and Santosh Pande call which code paths a program will run, sight unseen, so the build can tune for each input, in a paper from 2024. One past run is the stock guide, yet live apps swing their paths with each input, which makes a lone guide weak. Their kit pairs two tricks to call paths with no run at all.

One trick has large chat models read the code and call the path from text clues. The twin grows fake past runs with a small net and bends them to new inputs. Hot paths that eat most run time are caught at high rates. Tuned builds ran six points fast on mean and shrank in size.

Catalog record

CREATORBodhisatwa Chatterjee, Neeraj Jadhav, Santosh Pande
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE83 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06994

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Phaedrus: Predicting Dynamic Application Behavior with Lightweight Generative Models and LLMs Bodhisatwa Chatterjee, Neeraj Jadhav, Santosh Pande Abstract . Application profiling is an indispensable technique for many software development tasks, such as code and memory layout optimizations, where optimization decisions are tailored to specific program profiles. Unfortunately, modern application codebases exhibit highly variant behavior across different inputs, creating challenges for conventional profiling approaches that rely on a single representative execution instance. In this paper, we propose Phaedrus , a new compiler-assisted deep learning framework designed to predict dynamic program behaviors across varied execution instances, specifically focusing on dynamic function call prediction. These predicted call sequences are subsequently used to guide input-specific compiler optimizations, producing code specialized for each execution instance. Traditional profile-guided optimization methods struggle with the input-dependent variability of modern applications, where profiling on different inputs yields divergent application behaviors. To address this, Phaedrus proposes two new approaches: Application Behavior Synthesis (Dynamis) , a profile-less approach where Large Language Models (LLMs) directly infer dynamic functions based on source code & static compiler analysis, bypassing the need for traditional profiling, and Application Profile Generalization (Morpheus) , which uses generative models trained on compressed and augmented Whole Program Path (WPP) based function profiles to predict application behavior under unseen inputs. Our experiments show that Phaedrus accurately identifies the most frequently executed and runtime-dominated hotspot functions, accounting for up to 85–99% of total execution time. Leveraging these predictions, Phaedrus enables superior profile-guided optimizations, delivering an average speedup of 6% (upto 25%) and a binary size reduction of 5.19% (upto 19%), without any program execution. In addition, Phaedrus reduces WPP function profile sizes by up to 10 7 × 10^{7}\times . ISBN: 978-1-4503-XXXX-X/2018/06 1. Introduction Application profiling and execution tracing are crucial in the software development process. Popular performance enhancement techniques such as profile-guided optimization (PGO) ( Radigan et al., 1995 ; Grove et al., 1995 ; Gupta et al., 1997 ) involve generating execution traces for applications and then performing optimization decisions specific to the collected profile, such as code transformation ( Chang et al., 1991 ; Visochan et al., 2022 ; Sydow et al., 2020 ; Joshi et al., 2004 ) , data layout restructuring and reorganization ( Pettis and Hansen, 1990 ) . Application profiles and traces are also used to accomplish a wide variety of tasks such as estimating resource usage for scheduling and resource allocation ( Agelastos et al., 2015 ; Song et al., 2009 ; Viswanathan et al., 2011 ; Rashti et al., 2015 ; Agelastos et al., 2016 ; Kestor et al., 2013 ) , memory management for scratchpads, caches, and other memory hierarchies ( Cho et al., 2009 ; Ansari et al., 2009 ; Bruno and Ferreira, 2017 ; Agarwal and Wenisch, 2017 ; Xu et al., 2024 ) , memory simulation ( Uhlig and Mudge, 1997 ; Uhlig and Mudge, 2001 ; Rico et al., 2011 ; Hassan et al., 2007 ) and debugging ( Arnold et al., 2007 ; Nagarajan et al., 2012 ; Ning and Zhang, 2017 ; Ning and Zhang, 2018 ; Verma et al., 2020 ) . Such profile-driven approaches have been successfully utilized and deployed for over two decades. Most profile-driven tasks rely on the users to manually select profiles representative of the application’s intended execution scenario. This ensures the usefulness of the profile analysis and optimization decisions during the application’s runtime across different inputs. However, recent work ( Chatterjee et al., 2022 ; Mururu et al., 2023 ; Ahmed et al., 2017 ) has shown that most modern applications are complex and exhibit a high amount of input dependence, resulting in variable application behavior across different execution instances on various input data sets. This variant behavior is exhibited in all program components, including program branches, loops, and function calls. As a result, the application profile collected during a particular execution instance on a given input data set can be entirely different from another execution instance collected using a different input data set. A naive approach to tackle this problem would be to profile the application on multiple inputs - in practice, however, this ‘excessive’ profiling approach does not scale for workloads whose input domain is not numerable, such as graph applications. In addition, profile mixing can also be problematic for specific path-based optimizations that optimize along hot trace paths, pushing overheads to off-trace paths. In this work, we study the problem of generating dynamic application behavior across different execution scenarios pertinent to different input data sets by capturing traces of application’s function calls and control flow behavior . Specifically, we explore the problem of hot function call prediction , where the objective is to obtain the set of computationally intensive (hot) functions called by the application during its runtime, specific to a given program input. In this paper, we propose Phaedrus , a new compiler-assisted deep learning framework which employs two different approaches to predict this dynamic program behavior: (1) application behavior synthesis , where a compiler-assisted LLM framework reasons about the applications’ algorithmic properties and extrapolates the application’s dynamic set of hot functions without any profiles; (2) application profile generalization , where lightweight generative models are synthesized to predict the application’s dynamic function call chains by learning a specialized representation of the smallest application profile. Phaedrus ’s end goal is to use this predicted dynamic behavior to perform profile-guided optimization (PGO), such as changing the application function layout based on most executed functions to seek code size and performance enhancements. Phaedrus Framework: In a nutshell, the idea of this work is to utilize generative models to predict the behavior of dynamic applications . The predictions from these models are then further used to perform profile-guided optimization. Towards this, the first new concept introduced in this work is application behavior synthesis , which attempts to reason about the dynamic program behavior without execution or collected profiles with the aid of Large-Language Models (LLMs). Owing to their remarkable successes in every domain, from natural language processing to software engineering, LLMs have been extensively used for generating and optimizing programs ( Ugare et al., 2024 ; Gu, 2023 ; Lin et al., 2024 ; Liu et al., 2024 ) . However, the primary challenge that prevents the adoption of LLM in place of the mainstream compilation pipeline, is the absence of correctness guarantees ( Zhong and Wang, 2024 ; Spiess et al., 2024 ) . Secondly, LLMs, being generative models at their core, cannot inherently perform static program analysis, which can enhance its decision-making capacities. Thus, naively querying the LLM for most code reasoning tasks fails spectacularly (as shown in fig. 1 ). Figure 1. An example of unsuccessful dynamic program analysis with the LLM, given the source code and input files ( 520.omnetpp_r ). The LLM is unable to pinpoint hot functions in the benchmark just by observing the source code & input files, and the output is quite unsatisfactory. Compiler-Assisted LLM Reasoning : The first contribution of this work is the utilization of Large-Language Models, supplemented by the knowledge of static program properties obtained by compiler analysis . The Dynamis framework operates in three broad phases: first, filtering the wide corpus of LLM knowledge through domain knowledge deduction

Read the complete text