Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation

Joon Ha Kim, Geonwoo Kim, Anoop Rachakonda, Daehyeok Kim · 2026

Read Dooly: Configuration-Agnostic, Redundancy-Aware Profiling for LLM Inference Simulation by Joon Ha Kim, Geonwoo Kim, Anoop Rachakonda, Daehyeok Kim (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07985.txt.

Description

Joon Ha Kim and three co-authors cut the cost of profiling language model serving, the measurements that feed inference simulators used to pick optimal configurations. Simulators normally hardcode their operation set to one configuration and re-profile everything from scratch, which makes exploring hardware, engines, and model variants prohibitively expensive.

Their system, Dooly, exploits structure: every operation's input dimensions are fixed either by the model configuration or by the incoming request, and configuration values recur across models. A single pass with taint propagation labels each dimension's origin, and only operations missing from the latency database get profiled. Across two GPU platforms and many architectures, Dooly matched simulator accuracy within a few percent while cutting profiling GPU-hours by more than half across twelve models.

Catalog record

CREATORJoon Ha Kim, Geonwoo Kim, Anoop Rachakonda, Daehyeok Kim
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07985

Browse the full library · About the Tower of Babel