Tool-Aware Planning in Contact Center AI: Evaluating LLMs through Lineage-Guided Query Decomposition

Varun Nathan, Shreyas Guha, Ayush Kumar · 2026

Read Tool-Aware Planning in Contact Center AI: Evaluating LLMs through Lineage-Guided Query Decomposition by Varun Nathan, Shreyas Guha, Ayush Kumar (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-14955.txt.

Description

Nathan and colleagues build a testbed for AI planning in contact centers. Answering a business question means splitting it into runnable steps across database tools and text tools. Links mark which steps can run in parallel. They offer two ways to grade plans: a metric-wise evaluator covering seven dimensions and a one-shot evaluator. A further method refines plans through an evaluator-optimizer loop to build ordered revision histories.

In a study of 14 language models, compound queries and plans longer than four steps prove hard. The best overall score reaches about 85 percent, while the top one-shot match rate at the highest tier is only about 50 percent. Lineage helps some leading models and improves step executability for many, but gaps in tool choice and completeness remain.

Catalog record

CREATORVarun Nathan, Shreyas Guha, Ayush Kumar
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE107 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-14955

Browse the full library · About the Tower of Babel