ADOR: A Design Exploration Framework for LLM Serving with Enhanced Latency and Throughput
Junsoo Kim, Hunjong Lee, Geonwoo Ko, Gyubin Choi, Seri Ham, Seongmin Hong, Joo-Young Kim · 2025
Read ADOR: A Design Exploration Framework for LLM Serving with Enhanced Latency and Throughput by Junsoo Kim, Hunjong Lee, Geonwoo Ko, Gyubin Choi, Seri Ham, Seongmin Hong, Joo-Young Kim (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-04253.txt.
Description
Seven researchers wrote a computer engineering paper in two thousand twenty-five on designing hardware to serve large language models. Serving is split into two stages with opposite needs. Reading the prompt is compute-heavy, while writing the answer leans on memory speed. Standard graphics chips balance the two poorly, wasting capacity.
Batching requests helps the hardware but slows each reply, which users feel. Their framework, ADOR, searches over ready-made architecture templates to recommend hardware tuned for model serving. It aims to reconcile the vendor's wish for full hardware use with the user's wish for fast answers, across varied serving setups.
Catalog record
| CREATOR | Junsoo Kim, Hunjong Lee, Geonwoo Ko, Gyubin Choi, Seri Ham, Seongmin Hong, Joo-Young Kim |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 42 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-04253