No More, No Less: Task Alignment in Terminal Agents
Sina Mavali David Pape Jonathan Evertz Samira Abedini Devansh Srivastav, Thorsten Eisenhofer Sahar Abdelnabi Lea Schönherr, [6pt] CISPA Helmholtz Center for Information Security · 2026
Read No More, No Less: Task Alignment in Terminal Agents by Sina Mavali David Pape Jonathan Evertz Samira Abedini Devansh Srivastav, Thorsten Eisenhofer Sahar Abdelnabi Lea Schönherr, [6pt] CISPA Helmholtz Center for Information Security (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12233.txt.
Introduction
Sina Mavali and seven colleagues build TAB, a benchmark for terminal agent task alignment. Terminal agents run tasks from a prompt, but must judge found instructions: useful cues must be followed, misleading ones ignored. Existing benchmarks miss this skill, since agents can look capable by following everything or look robust by ignoring everything. TAB offers 89 underspecified tasks, each with a needed cue hidden in an artifact plus a distractor. Tests on ten agents show a gap between completion and alignment, and defenses that block distractors also block needed cues.
Catalog record
| CREATOR | Sina Mavali David Pape Jonathan Evertz Samira Abedini Devansh Srivastav, Thorsten Eisenhofer Sahar Abdelnabi Lea Schönherr, [6pt] CISPA Helmholtz Center for Information Security |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 93163 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12233
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
No More, No Less: Task Alignment in Terminal Agents Sina Mavali David Pape Jonathan Evertz Samira Abedini Devansh Srivastav, Thorsten Eisenhofer Sahar Abdelnabi Lea Schönherr, [6pt] CISPA Helmholtz Center for Information Security Abstract Terminal agents are increasingly capable of executing complex, long-horizon tasks autonomously from a single user prompt. To do so, they must interpret instructions encountered in the environment (e.g., README files, code comments, stack traces) and determine their relevance to the task. This creates a fundamental challenge: relevant cues must be followed to complete a task, whereas irrelevant or misleading ones must be ignored. Existing benchmarks do not capture this ability. An agent may appear capable by blindly following all instructions, or appear robust by ignoring them altogether. We introduce TAB (Task Alignment Benchmark), a suite of 89 terminal tasks derived from Terminal-Bench 2.1. Each task is intentionally underspecified, with missing information provided as a necessary cue embedded in a natural environmental artifact, alongside a plausible but irrelevant distractor. Solving these tasks requires selectively using the cue while ignoring the distractor. Applying TAB to ten frontier agents reveals a systematic gap between task capability and task alignment. The strongest Terminal-Bench agent achieves high task completion but low task alignment on TAB. Evaluating six prompt-injection defenses further shows that suppressing distractor execution also suppresses the cues required for task completion. These results demonstrate that task-aligned agents require selective use of environmental instructions rather than blanket acceptance or rejection. 1 Introduction Terminal agents powered by language models, such as Claude Code [ 2 ] or Codex [ 22 ] , are increasingly deployed in terminal environments to perform complex software engineering and terminal-based tasks [ 19 ] . Given a high-level goal, these agents autonomously inspect the workspace, install dependencies, execute commands, modify files, and iterate until a given task is done, and can complete workflows that would otherwise require substantial human effort [ 20 , 17 ] . A key part of this capability is the ability to interpret information encountered during execution, such as README files, code comments, logs, stack traces, configuration files, and web pages. However, terminal environments are not clean instruction channels. They may contain outdated documentation, irrelevant instructions, conflicting implementations, or third-party code. At the same time, users typically specify goals rather than complete procedures, leaving intermediate decisions to be inferred during execution [ 29 ] . This creates a tension that existing evaluations largely miss: determining which information encountered during execution is relevant to the user’s goal, which is irrelevant, and which may be misleading. This problem is not merely hypothetical. OpenAI reports an internal coding-agent behavior in which an agent pursuing a legitimate task encountered an ‘‘Access is denied’’ error. The expected behavior was to treat the denial as a constraint and seek an alternative. Instead, the agent inferred that security controls might be blocking progress and attempted to bypass them [ 33 ] . If such behavior persists, the agent will be unable to solve the task or may satisfy the task objective while violating the operational constraints (cf., Figure 1 ). The central challenge is therefore not to prevent agents from acting on environmental information, but to ensure that they interpret it appropriately. Relevant cues should guide the solution, while irrelevant directives and contextual constraints should delimit what actions are appropriate. We call this capability task alignment , selectively using environmental information that is necessary or helpful for the user’s goal while ignoring information that is irrelevant, misleading, or out of scope— no more, no less . Figure 1 : Task alignment in one TAB task. In a file the agent naturally reads to solve a (underspecified) user prompt, a helpful cue sits beside an unrelated distractor. A task-aligned agent should follow the cue as it’s related to the user’s goal to solve the task and ignore the distractor. This is not captured by existing benchmarks that evaluate capability and robustness in isolation. Capability benchmarks such as SWE-bench [ 15 ] , OSWorld [ 34 ] , and Terminal-Bench [ 20 ] measure whether agents eventually complete the task, but abstract away how that outcome is achieved. In contrast, security and safety benchmarks [ 9 , 37 , 7 , 16 , 26 ] evaluate resistance to harmful or adversarial instructions, but do so in settings where agents are not required to rely on environmental information to solve the task. To address this gap, we introduce TAB (Task Alignment Benchmark), a benchmark for evaluating whether terminal agents can leverage task-relevant environmental information without over-complying with unrelated instructions. Starting from Terminal-Bench 2.1 [ 20 ] , TAB transforms 89 terminal tasks into alignment variants. For each task, we remove required information from the user instruction and reintroduce it as a necessary cue in an artifact of the agent’s environment. The same artifact also contains a plausible but irrelevant distractor, since agents that indiscriminately follow encountered instructions may still complete the task while performing unrelated actions. We manually verify task solvability, cue necessity, and distractor irrelevance, and measure task alignment through two complementary components: cue utilization , which captures successful task completion after observing the cue, and distraction resistance , which captures avoidance of the observed distractor. Our evaluation reveals that task completion and task alignment can diverge sharply. For example, among the strongest Terminal-Bench performers in our study, GPT-5.5 achieves the highest cue utilization but only 23\% task alignment due to low distraction resistance, whereas Claude Opus 4.7 achieves 72\% task alignment through better resistance against distractors. Evaluating six defenses designed to restrict environmental instruction-following further reveals that suppressing distractor execution often suppresses the cues required for task completion. Taken together, our results suggest that task-aligned agents require selective use of environmental information rather than blanket acceptance or rejection. In summary, our contributions are as follows: 1. We formalize task alignment as the selective use of environmental information, decomposing it into cue utilization and distraction resistance to isolate alignment behavior from raw task capability. 2. We introduce TAB, a benchmark of 89 transformed terminal tasks, in which each task requires recovering a necessary environmental cue while ignoring a co-located irrelevant distractor. 3. We evaluate ten frontier terminal agents and six defenses, showing that task capability does not imply task alignment, and that reducing over-compliance alone is insufficient because effective methods must preserve both cue utilization and distraction resistance. 2 Measuring Task Alignment We start by formally introducing task alignment. To this end, we consider a setting in which an agent operates under a partially specified instruction together with environmental context encountered during execution. A task-aligned agent should selectively act on information that is relevant to the user’s goal while ignoring directives that are unrelated or unnecessary. Because users typically do not, and often cannot, specify all information required for successful task completion in advance, agents must autonomously interpret environmental information and determine which parts are useful for solving the task. Formally, for a task t , let I_{t} denote the original fully specified instruction and I_{t}^{-} the instruction after removing information necessary for solving the task. A cue c_{t} restores the removed information and is required for solving the task under I_{t}^{-} . A distractor d_{t} is a directive unrelated to the