PARD: Enhancing Goodput for Inference Pipeline via P ro A ctive R equest D ropping Conference: The 21st European Conference on Computer Systems; April 27–30, 2026; Edinburgh, UK. The 21st European Conference on Computer Systems (EuroSys ’26), April 27–30, 2026, Edinburgh, UK 96

Zhixin Zhao, Yitao Hu, Simin Chen, Mingfang Ji, Wei Yang, Yuhao Zhang, Laiping Zhao, Wenxin Li, Xiulong Liu, Wenyu Qu, Hao Wang · 2026

Read PARD: Enhancing Goodput for Inference Pipeline via P ro A ctive R equest D ropping Conference: The 21st European Conference on Computer Systems; April 27–30, 2026; Edinburgh, UK. The 21st European Conference on Computer Systems (EuroSys ’26), April 27–30, 2026, Edinburgh, UK 96 by Zhixin Zhao, Yitao Hu, Simin Chen, Mingfang Ji, Wei Yang, Yuhao Zhang, Laiping Zhao, Wenxin Li, Xiulong Liu, Wenyu Qu, Hao Wang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-08747.txt.

Description

Inference pipelines chain several neural models under tight latency limits, and timeouts cascade when requests pile up. Zhixin Zhao and ten co-authors argue that dropping requests only when forced, the reactive policy, drops too late and drops the wrong ones. Their arXiv paper proposes PARD, a system that drops proactively to raise goodput across the whole workload.

Dropping decisions use live pipeline state to choose timing, while an adaptive priority mechanism picks which requests to drop based on how much latency budget remains and how heavy the load is. Tests on a 64-GPU cluster over real workloads show goodput gains of 16 to 176 percent over leading systems, with drop rates and wasted compute cut severalfold. Readers get a counterintuitive lesson: timely, precise dropping serves more users.

Catalog record

CREATORZhixin Zhao, Yitao Hu, Simin Chen, Mingfang Ji, Wei Yang, Yuhao Zhang, Laiping Zhao, Wenxin Li, Xiulong Liu, Wenyu Qu, Hao Wang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE62 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-08747

Browse the full library · About the Tower of Babel