AURA : A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents
Takyoung Kim, Janvijay Singh *, Shuhaib Mehri *, Sagnik Mukherjee, Nimet Beyza Bozdag, Sumuk Shashidhar, Dilek Hakkani-Tür · 2025
Read AURA : A Diagnostic Framework for Tracking User Satisfaction of Interactive Planning Agents by Takyoung Kim, Janvijay Singh *, Shuhaib Mehri *, Sagnik Mukherjee, Nimet Beyza Bozdag, Sumuk Shashidhar, Dilek Hakkani-Tür (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-01592.txt.
Description
Takyoung Kim and co-authors present AURA, a framework for judging how happy users are with planning agents, in a 2025 paper. AI planning agents chain many steps, from reading context to calling tools and replying. Most benchmarks score only whether the final task got done. The authors argue users judge the whole ride, not just the destination.
AURA breaks agent behavior into stages and scores each with small, clear criteria judged by language models. That detail lets builders spot which stage delights or annoys users. Tests show agents shine at different stages, and satisfaction tracks process as well as outcome. The paper also weighs multi-agent setups and the limits of fake users for testing. It aims to align agent design with what people actually value.
Catalog record
| CREATOR | Takyoung Kim, Janvijay Singh *, Shuhaib Mehri *, Sagnik Mukherjee, Nimet Beyza Bozdag, Sumuk Shashidhar, Dilek Hakkani-Tür |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-01592