Beyond expert users: agents should help users construct preferences, not just elicit them
Irena Saracay, Ludwig Schmidt, Carlos Guestrin · 2026
Read Beyond expert users: agents should help users construct preferences, not just elicit them by Irena Saracay, Ludwig Schmidt, Carlos Guestrin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30863.txt.
Description
Irena Saracay, Ludwig Schmidt and Carlos Guestrin argue that AI agents wrongly assume their users already know what they want. Agents typically treat users as experts with fixed preferences and answer underspecified requests with clarifying questions. But a user may be unable to state a preference for a feature without first learning something about the domain, through examples or explanations. Drawing on a Search-Experience-Credence framework from information economics, the authors propose CoPref, a model of users who build their preferences during the dialogue itself, shaped by what the agent does.
They test the idea in shopping with CoShop, a benchmark where an agent recommends to a simulated CoPref user. Five frontier models were evaluated over five turns of conversation, and none exceeded 56% accuracy. Failures came not from finding items but from how little the chat expanded what users knew.
Catalog record
| CREATOR | Irena Saracay, Ludwig Schmidt, Carlos Guestrin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 166 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.30863