Beyond expert users: agents should help users construct preferences, not just elicit them

Irena Saracay, Ludwig Schmidt, Carlos Guestrin · 2026

Read Beyond expert users: agents should help users construct preferences, not just elicit them by Irena Saracay, Ludwig Schmidt, Carlos Guestrin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30863.txt.

Description

Irena Saracay, Ludwig Schmidt and Carlos Guestrin argue that AI agents wrongly assume their users already know what they want. Agents typically treat users as experts with fixed preferences and answer underspecified requests with clarifying questions. But a user may be unable to state a preference for a feature without first learning something about the domain, through examples or explanations. Drawing on a Search-Experience-Credence framework from information economics, the authors propose CoPref, a model of users who build their preferences during the dialogue itself, shaped by what the agent does.

They test the idea in shopping with CoShop, a benchmark where an agent recommends to a simulated CoPref user. Five frontier models were evaluated over five turns of conversation, and none exceeded 56% accuracy. Failures came not from finding items but from how little the chat expanded what users knew.

Catalog record

CREATORIrena Saracay, Ludwig Schmidt, Carlos Guestrin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE166 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30863

Browse the full library · About the Tower of Babel