Behavior Cue Reasoning: Monitorable Reasoning Improves Efficiency and Safety through Oversight
Christopher Z. Cui Taylor W. Killian Prithviraj Ammanabrolu · 2026
Read Behavior Cue Reasoning: Monitorable Reasoning Improves Efficiency and Safety through Oversight by Christopher Z. Cui Taylor W. Killian Prithviraj Ammanabrolu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07021.txt.
Introduction
Christopher Z. Cui, Taylor W. Killian, and Prithviraj Ammanabrolu tackle oversight of language model reasoning. Harmful plans may only become visible after a model finishes thinking, which makes monitoring hard. Their answer is Behavior Cue Reasoning, where the model learns to emit special tokens just before certain behaviors.
Those cues act as both signals and control points for an outside monitor. In tests, a monitor using only cue information cut wasted reasoning tokens by up to half on hard math problems. In a safety setting, cues let monitors recover safe actions from most traces that would otherwise end unsafely, with no loss in task performance.
Catalog record
| CREATOR | Christopher Z. Cui Taylor W. Killian Prithviraj Ammanabrolu |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07021
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Behavior Cue Reasoning: Monitorable Reasoning Improves Efficiency and Safety through Oversight Christopher Z. Cui Taylor W. Killian Prithviraj Ammanabrolu Abstract Reasoning in Large Language Models (LLMs) poses a challenge for oversight as many misaligned behaviors do not surface until reasoning concludes. To address this, we introduce Behavior Cue Reasoning for making LLM reasoning more controllable and monitorable. Behavior Cues are special token sequences that a model is trained to emit immediately before specific implicit and explicit behaviors, acting as dual purpose signal and control levers. Our experiments reveal that a Behavior Cue Reasoning model has equal to greater performance as the base model, allows for steerable reasoning through external enforcement of Behavior Cues, and improves the monitorability of reasoning for external oversight monitors. When fine-tuning a weaker external monitor with Reinforcement Learning for reasoning oversight, a compressed view of only information surfaced by Behavior Cues is sufficient signal for the monitor to prune up to 50% of otherwise wasted reasoning tokens in complex math problem solving. When leveraged by an almost optimal rule-based monitor in an environment where excessive constraint violations results in failure, Behavior Cues allows for the recovery of safe actions from 80% of reasoning traces that would otherwise end with the proposal of an unsafe action, more than doubling the success rate from 46% to 96%. Through evaluation across two model families and three domains, we show that Behavior Cue Reasoning improves reasoning monitorability and controllability with no cost to performance. More broadly, our work progresses scalable oversight by demonstrating how the monitored model itself can be trained to reason more tractably to oversight. Code 1 Introduction Figure 1: We take reasoning traces from a base actor model (1) and elicit the immediate working answer at each step by truncating the reasoning, appending a stop phrase, and allowing the model to produce an answer (2). Working answers are filtered to emulate the behavior of the working answer only surfacing when changing and embedded back into the reasoning trace with Behavior Cues (3). Post SFT (4), the Behavior Cue Reasoning Model adopts this answer reporting behavior, creating ideal decision points for an external oversight monitor. In Efficiency Monitoring , the monitor optimizes stopping once the correct final answer surfaces. In Safety Monitoring , the monitor attempts to prevent the commitment of unsafe actions by continuing reasoning. In Safety Enforcement , the monitor records the latest safe action to commit instead, allowing for secured unsafe reasoning. For difficult tasks, Reasoning LLMs are trained to search within their token space by generating reasoning traces to refine their final answer. These traces allow the model to trade inference-time compute for better performance [ 27 , 41 , 2 ] , but are typically treated as artifacts excluded from the message history with the exception of tool-use [ 9 ] . Most critically, these behaviors can remain hidden in the reasoning trace, surfacing only when the model commits to an answer. Without deeper visibility into these reasoning traces, an external monitor tasked with oversight has limited ability to detect and interrupt these behaviors before their negative effects manifest in the model’s final response. Prior approaches either inject steering phrases into the reasoning trace at decoding time [ 19 , 42 , 49 , 40 , 25 ] , or train the model to condition on special tokens inserted externally into its context [ 11 , 21 , 39 ] . Some combine both, using special tokens the model is trained to respond to as steering phrases [ 30 ] . In all cases, the phrase or tokens are treated as external signals. The model itself does not surface information about its own underlying behavior, limiting applicability to oversight. To address this, we introduce Behavior Cues to progress scalable oversight by training the model to naturally generate special token sequences that signal specific behaviors during reasoning, including behaviors that would otherwise be internal to the model. We define this as Behavior Cue Reasoning. The explicit appearance of Behavior Cues in a model’s reasoning allows an external monitor to track behavioral progression through simple parsing, treating the model itself as a black box. The same tokens that surface behavior can also be externally enforced to trigger it. This allows an external oversight monitor to intervene in a model’s reasoning, extending monitorability into controllability. In this work, we investigate three Behavior Cues: [answer] , [continue] , and [stop] . We evaluate on Qwen3-8B (hybrid reasoning) and GLM-Z1-9B (pure reasoning) to test generalization across different model families in three separate problem domains. AIME serves as a single-turn evaluation for reasoning models in complex math problem solving. Textworld, a situated text environment with natural language observations, evaluates Behavior Cues in an out-of-domain, multi-turn, task completion setting. Finally, we extend Hazardworld with a text-based interface to allow for evaluation of task completion under constraint adherence in multi-turn settings. We evaluate Behavior Cues for scalable oversight through three research questions, one focused on the actor model itself and two on the monitorability of Behavior Cue Reasoning to an external oversight module. For the actor model, we evaluate To what degree does training a model to perform Behavior Cue Reasoning impact baseline task performance and controllability? For the monitorability of Behavior Cue Reasoning, we evaluate oversight module performance along two axes: To what degree do Behavior Cues enable an external monitor to reduce wasted reasoning tokens without sacrificing correctness and To what degree do Behavior Cues enable an external monitor to prevent unsafe actions? Our results demonstrate overall positive effects on baseline performance, adherence to external enforcement, and how Behavior Cues can enable oversight where standard reasoning would be too noisy or opaque. We observe performance improvements in all domains and both model families. In the majority of cases, the actor models follow injected Behavior Cues with the expected behavior. Most significantly, Behavior Cues serve as a critical interface for external oversight. We observe that Behavior Cues act as a necessary filter for decision points, limiting queries to the oversight module to when intervention results in the highest impact. This provides a learnable horizon for RL fine-tuning a non-reasoning monitor. For safety constraint enforcement, we find LLM-based monitors struggle with identifying unsafe actions. However, by parsing candidate actions surfaced at [answer] against rules constructed from training trajectories, a rule-based monitor achieves near perfect classification accuracy. An LLM augmented with this monitor achieves 50% more success in an environment where excessive constraint violations results in failure. Through these experiments, we show that improved monitorability through Behavior Cue Reasoning enables oversight for both safety and efficiency. In summary, our contributions are as follows: • We introduce Behavior Cues, a mechanism for making LLM reasoning more monitorable and controllable for oversight through trained token sequences ( [answer] , [continue],[stop] ) that act as surface-level hooks for reasoning behaviors. • We show a model trained to perform Behavior Cue Reasoning displays overall equal to greater baseline performance and reliably adheres to expected behavior when the cue is enforced externally. • For efficiency monitoring, we demonstrate that Behavior Cue Reasoning enables external oversight via tractable decision horizons, allowing for the training of a non-reasoning monitor and enabling a compressed-trace formulation that correctly prunes up to