Evaluating Language Model Reasoning about Confidential Information
Dylan Sam Alexander Robey Andy Zou Matt Fredrikson J. Zico Kolter · 2025
Read Evaluating Language Model Reasoning about Confidential Information by Dylan Sam Alexander Robey Andy Zou Matt Fredrikson J. Zico Kolter (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-19980.txt.
Description
Dylan Sam and co-authors tested in 2025 whether language models can reason about confidential information. As models act as agents in high-stakes settings, they must follow user-set access rules. The team built a benchmark where a model must judge if a request is authorised by a correct password.
Current open and closed models struggled with this seemingly simple task. Stronger reasoning did not generally help, and reasoning traces often leaked the secret itself, raising doubts about exposing them. Tests grew harder with jailbreak pressure and longer multi-turn chats. The authors conclude that frontier models are not yet suited to guarding confidential information.
Catalog record
| CREATOR | Dylan Sam Alexander Robey Andy Zou Matt Fredrikson J. Zico Kolter |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-19980