NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles
Xiao Jia · 2026
Read NeuroState-Bench: A Human-Calibrated Benchmark for Commitment Integrity in LLM Agent Profiles by Xiao Jia (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-01847.txt.
Description
Xiao Jia introduces NeuroState-Bench, a benchmark that tests whether a language model agent keeps its commitments across a long, multi-turn task. Success at the end alone does not show whether the agent held to what the task required. The benchmark uses fixed side questions, built into each task, rather than guesses about hidden model states.
The set holds 144 tasks and 306 side probes across eight failure types, with clean and distracting versions and three levels of difficulty. Human checks on sampled tasks showed strong agreement. Across 32 model profiles, the leader on task success was not the leader on commitment, and most profiles changed rank when the score switched to commitment.
Catalog record
| CREATOR | Xiao Jia |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 82 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.01847