How Useful Is Cross-Domain Generalization for Training LLM Monitors?
Sam Martin, Fabien Roger · 2026
Read How Useful Is Cross-Domain Generalization for Training LLM Monitors? by Sam Martin, Fabien Roger (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-12265.txt.
Introduction
Sam Martin and Fabien Roger ask whether multi-task training helps monitors on new domains. Prompted models classify with little data, but miss the robustness and gains of fine-tuning. Training on tasks, each with its own prompt, partly generalizes to adjacent domains, lifting unseen scores. Yet edge cases break prompt following, such as when the prompt changes while the domain stays the same. Mixing classification training with instruction training keeps the gains while fixing most failures, and no-thinking training even generalizes to with-thinking classification and summarization.
Catalog record
| CREATOR | Sam Martin, Fabien Roger |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 54717 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.12265
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
How Useful Is Cross-Domain Generalization for Training LLM Monitors? Sam Martin, Fabien Roger Abstract Using prompted language models as classifiers enables classification in domains with limited training data, but misses some of the robustness and performance benefits that fine-tuning can bring. We study whether training on multiple classification tasks, each with its own prompt, improves performance on new domains with new classification prompts. We show that such training partially generalizes to adjacent domains, improving classification performance on tasks that are unseen during training. However, we identify specific edge cases where the fine-tuned models fail to follow prompts, such as when the classification prompt changes completely while the data domain remains the same as during training. We show that classification training can be mixed with general instruction following training, and that (when done well) such training keeps the benefits of classification training and mitigates its generalization failures. Surprisingly, we see that this no-thinking supervised classification training can generalize to with-thinking classification and summarization, suggesting that no-thinking classification training might be instrumentally useful in building other kinds of classifiers and monitoring systems. Figure 1: We train a model with SFT on 1-token classification responses (left), then evaluate it on a new classification task, thinking classification, and summarization (right). The fine-tuned model improves across all three tasks, despite not being trained on them. Prompts and responses are illustrative; full prompts can be found in Appendix A.2 . 1 Introduction Language models are increasingly used as classifiers, for example to flag sessions where the user is trying to cause harm ( Markov et al. 2023 ; Sharma et al. 2025 ) or sessions where the AI itself is performing dangerous or misaligned actions ( Anthropic 2026a ; OpenAI 2026 ; Anthropic 2026b ) . Classifiers are sometimes specialized fine-tuned classifiers, which was shown to help improve robustness ( Sharma et al. 2025 ; Cunningham et al. 2026 ) , but it is also common to simply use prompted language models ( Anthropic 2026a ; OpenAI 2026 ; Anthropic 2026b ; Sumers et al. 2025 ) which rely on the general instruction following capabilities of the models and don’t require creating specialized training data. An intermediate approach is using models fine-tuned on multiple different classification tasks (with a separate classification prompt for each task) ( Inan et al. 2023 ; OpenAI 2025b ) . In this work, we attempt to better understand the generalization properties of such general classifier training, with the goal of improving AI control monitors: How well does such training transfer to new tasks? How much benefit comes from training a specialized classifier model, compared to including the same data in a regular instruction following training run? We show that classification training can generalize to new adjacent domains. Concretely, a model trained to answer "is this cyber harm?" can later be prompted to answer "does the AI do anything dangerous?" and recover 75% of the improvement obtained by training directly on the target task. If we have a limited amount of in-domain data available, then adding additional data from an adjacent domain improves performance even when the dataset is very imbalanced. However, we observe some generalization failures where the model infers the classification rule from the training data rather than looking at the prompt. This can lead to some cases where training makes the model worse. For example, single task training can cause the model to stick to the training task: when given a new prompt but a transcript similar to that used in training, the model sometimes applies the training classification task rather than following the new prompt. We are able to mitigate these failures by training on general instruction-following data alongside the classification data. We find that simply mixing the two datasets is not very effective since classification samples contain many fewer unmasked training tokens than the instruction following samples. Doing classification training before instruction training mitigates the downsides of dilution. We find that single-token classification training benefits extend beyond single-token classification . We evaluate models on thinking classification, where the model is asked to generate reasoning before giving an answer. Training Qwen3-8B only on a single-token cyber harm classification task increases with-thinking AUC from 0.84 to 0.94. We also find that single-token classification training can increase the rate at which models identify dangerous actions when used as summarizers. Finally, we show that this transfer from single-token classification to thinking classification can be used as an unsupervised elicitation technique by iteratively using thinking classification predictions in single-token classification training. Our contributions are as follows: 1. We show that classification training transfers across adjacent domains and that data from adjacent domains can be used to supplement limited amounts of in-domain data (Section 4 ); 2. We exhibit two kinds of generalization failures that result from single task fine-tuning and show that training on instruction-following data mitigates them (Section 4.6 ); 3. We evaluate classification-trained models on generative tasks (thinking classification and summarization) and find uplift even though we never train on these tasks (Section 5 ). 2 Related Work Misuse classifiers and AI control monitoring. Misuse classifiers detect harmful user intent such as requesting help with bioweapon synthesis so that responses can be blocked. They are typically either prompted classifiers ( Sumers et al. 2025 ) or models fine-tuned for a single task ( Sharma et al. 2025 ; Cunningham et al. 2026 ) . AI control monitors flag dangerous actions taken by a misaligned AI such as sabotaging research ( Greenblatt et al. 2024 ) . These are typically implemented as prompted language models ( Anthropic 2026a ; OpenAI 2026 ; Anthropic 2026b ) that rely on the broad instruction following capabilities of these models rather than task specific training. We study both misuse and AI control monitors. Training general purpose classifiers. LlamaGuard ( Inan et al. 2023 ) fine-tunes a model on multiple harm criteria and shows that it generalizes to unseen criteria at test time. Subsequent work has scaled this ( Zeng et al. 2024 ; Zhao et al. 2025 ) and more recently OpenAI 2025b extends this to reasoning classifiers. We extend existing work by studying how this type of training can be used alongside general instruction following training to produce general models that are more capable at classification tasks, rather than building specialized classifier models. We also study transfer beyond harm classification: to an adjacent domain (AI control), and to text generation abilities. 3 Experiment Setup 3.1 Classifier formats The classification tasks we use are binary. We prompt with the classification question as shown in Figure 1 . We consider the following classification modes: 1. 1-token. We prompt the model as shown in Figure 1 , asking the model to say 1 or 0, and then measure the probability of it saying 1 (normalized by P(1) + P(0)). 2. Thinking. We use the model’s extended thinking mode to think through its answer before providing a classification score between 0 and 100. After reasoning, the probability assigned to 1 or 0 no longer reflects the confidence as above, so we prompt the model to provide its own confidence score. 3.2 Datasets We use the following datasets: Chemical harm (50k) and Cyber harm (50k) are synthetic conversations labelled