Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs

Kangqi Ni, Wenyue Hua, Xiaoxiang Shi, Jiang Guo, Shiyu Chang, Tianlong Chen · 2026

Read Chimera: Latency- and Performance-Aware Multi-agent Serving for Heterogeneous LLMs by Kangqi Ni, Wenyue Hua, Xiaoxiang Shi, Jiang Guo, Shiyu Chang, Tianlong Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-22206.txt.

Description

Ni and co-authors present Chimera, a serving system for multi-agent workflows on mixed language model clusters. Each workflow stage is a model call whose output feeds the next stage. Mixed clusters allow finer trade-offs between speed and quality, but scheduling gets harder. Chimera routes each request by estimated model confidence. It also tracks congestion from in-flight token volumes to balance load. On code and math workflows, it cuts latency up to 2.4 times and lifts task scores by 8 to 9.5 points.

Catalog record

CREATORKangqi Ni, Wenyue Hua, Xiaoxiang Shi, Jiang Guo, Shiyu Chang, Tianlong Chen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE40629
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.22206

Browse the full library · About the Tower of Babel