But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectors
Leon EshuijsArchie Chaudhury * Alan McBeth *, Ethan Nguyen · 2025
Read But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectors by Leon EshuijsArchie Chaudhury * Alan McBeth *, Ethan Nguyen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17760.txt.
Introduction
Leon Eshuijs and co-authors try to help chat models that grade other models' answers spot false or sly replies. Their 2025 paper notes that graders with only text access miss soft forms of deceit, such as praise used to sway a reader. Their method works inside the model. It learns a steering shift that pushes the model toward honest takes, from just one training case.
That shift makes a clean, honest twin for a suspect reply, giving the grader a point of contrast. On a new test of sly replies, scores rose for two top graders. Gains shrank when tasks grew too hard for the grader, and mid layers of the model steered best.
Catalog record
| CREATOR | Leon EshuijsArchie Chaudhury * Alan McBeth *, Ethan Nguyen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 65 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17760
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
But what is your honest answer? Aiding LLM-judges with honest alternatives using steering vectors Leon EshuijsArchie Chaudhury * Alan McBeth *, Ethan Nguyen Abstract LLM-as-a-judge is widely used as a scalable substitute for human evaluation, yet current approaches rely on black-box access and struggle to detect subtle dishonesty, such as sycophancy and manipulation. We introduce Judge Using Safety-Steered Alternatives (JUSSA), a framework that leverages a model’s internal representations to optimize an honesty-promoting steering vector from a single training example, generating contrastive alternatives that give judges a reference point for detecting dishonesty. We test JUSSA on a novel manipulation benchmark with human-validated response pairs at varying dishonesty levels, finding AUROC improvements across both GPT-4.1 (0.893→0.946) and Claude Haiku (0.859→0.929) judges, though performance degrades when task complexity is mismatched to judge capability, suggesting contrastive evaluation helps most when the task is challenging but within the judge’s reach. Layer-wise analysis further shows that steering is most effective in middle layers, where model representations begin to diverge between honest and dishonest prompt processing. Our work demonstrates that steering vectors can serve as tools for evaluation rather than for improving model outputs at inference, opening a new direction for thorough white-box auditing. 1 1 1 Our code: https://github.com/watermeleon/judge_with_steered_response 1 Introduction As Large Language Models (LLMs) capabilities advance, ensuring they are trustworthy and aligned to the right objectives has emerged as a critical priority Bengio et al. (2024) . Recent work has found that certain problematic behaviors, such as sycophancy, get worse with model size Perez et al. (2023) , and that LLM agents can exhibit deceptive behavior when pursuing objectives Meinke et al. (2024) ; Greenblatt et al. (2024) . Yet most evaluations intended to measure honesty merely assess factual accuracy Ren et al. (2024) , leaving subtler forms of dishonesty largely undetected. We focus on manipulation , which we define as actions designed to influence beliefs, desires, or emotions for reasons beyond truth-telling Coons and Weber (2014) , encompassing deception, sycophancy, and emotional pressure. Black-box access alone is increasingly recognized as insufficient for rigorous safety evaluation Casper et al. (2024) , motivating a growing body of white-box interpretability work. Activation steering has been applied to enhance truthfulness and honesty at inference time Zou et al. (2023) ; Wang et al. (2025) ; Goel et al. (2025) ; Zhao et al. (2026) , while linear probes and unsupervised methods have been used to detect latent knowledge and strategic deception in model activations Burns et al. (2022) ; Goldowsky-Dill et al. (2025) . While existing steering works focus on improving model outputs during deployment, we argue that steering vectors - the directions in activation space Subramani et al. (2022) - are better suited for evaluation. In an auditing setting, a model that produces dishonest outputs under any condition warrants further scrutiny, so surfacing even a single failure is informative, whereas inference-time safety demands that steering work reliably across all inputs. Moreover, the predict/control discrepancy Wattenberg and Viégas (2024) suggests that steering-based evaluation may surface aspects of model behavior that probe-based detection methods miss. To this end, we introduce Judge Using Safety-Steered Alternatives (JUSSA), a framework that repurposes steering vectors as tools for evaluation rather than for improving model outputs at inference. JUSSA optimizes an honesty-promoting steering vector from a single training example Dunefsky and Cohan (2025) and uses it to generate a contrastive alternative for each response under evaluation. The core hypothesis is that while judges struggle to identify subtle dishonesty when evaluating responses in isolation, they benefit from comparison with an honest alternative generated by the assessed model itself. We evaluate JUSSA on an adapted sycophancy benchmark and a new manipulation dataset covering nine categories across three manipulation types, validated through human evaluation. JUSSA improves dishonesty detection for both GPT-4.1 (AUROC 0.893\rightarrow 0.946 ) and Claude Haiku ( 0.859\rightarrow 0.929 ) judges, with weaker judges benefiting on simpler tasks such as sycophancy detection and stronger judges on harder tasks such as subtle manipulation. Layer-wise analysis reveals that steering is most effective in middle layers (8–13), where model representations diverge between honest and dishonest prompt processing. In summary, our contributions are (1) the JUSSA framework for steering-based contrastive evaluation, (2) a human-validated manipulation benchmark across nine categories, and (3) empirical and mechanistic analysis of when and where steering aids dishonesty detection. Figure 1: Overview of the JUSSA evaluation framework. In the Single Judge baseline (top), the LLM judge fails to detect subtle dishonesty when evaluating responses in isolation. In JUSSA (bottom), an honesty-promoting steering vector produces a more honest alternative of the agent’s response, enabling correct manipulation detection. See Figure 2 for examples. 2 Related Work Dishonesty Evaluation Sharma et al. (2023) demonstrated that state-of-the-art models consistently exhibit sycophancy across text-generation tasks, a finding we build on directly by adapting their poem feedback dataset. More critically, work on strategic deception has shown that models can produce strategically misleading outputs that evade output-based detection Panfilov et al. (2025) ; Scheurer et al. (2024) . Yet Morabito et al. (2024) show that both LLM judges and humans struggle to detect subtle forms of bias in text, motivating evaluation approaches that go beyond surface-level response assessment. LLM-as-a-Judge Evaluation Zheng et al. (2023) show that strong LLM judges achieve over 80% agreement with human preferences, while also exhibiting position and verbosity biases. Pairwise comparison setups improve reliability over single-sample evaluation ( Zheng et al., 2023 ; Liusie et al., 2024 ) , and Zhang et al. (2025) further enhance judgments by providing multiple crowd-sourced responses as reference points. These approaches rely on responses from other LLMs or on extended inference-time compute, rather than leveraging the assessed model’s own internal representations. Steering Vectors for Evaluation Panickssery et al. (2023) introduced contrastive activation addition (CAA), computing steering vectors from activation differences between behavior examples, though Tan et al. (2024) identified reliability concerns, including anti-steerable behaviors. Dunefsky and Cohan (2025) addressed these limitations with single-sample gradient-optimized steering, achieving 96.9% success rates on safety benchmarks. While most steering work targets inference-time behavior improvement Li et al. (2023) ; Zou et al. (2023) , recent concurrent work has begun exploring steering for auditing: Cyberey et al. (2026) propose white-box sensitivity auditing that reveals biases invisible to black-box evaluation, and Hua et al. (2025) use steering to suppress evaluation-awareness. Our work takes a complementary approach, using steering to generate contrastive alternatives that serve as reference points for an external judge. 3 Judge Using Safety-Steered Alternatives We evaluate responses from a target model (the LLM-agent ) using a separate judge model (the LLM-judge ). For each user question, we construct a base prompt x_{\text{base}} designed to elicit honest responses and a provoking prompt x_{\text{prov}} designed to provoke dishonest responses, yielding corresponding responses y_{\text{base}} and y_{\text{prov}} from the LLM-agent. The judge evaluates each response on correctness ( s_{\text{corr}} ) and task-specific dishonesty ( s_{h} , where h\in\{\text{sycophancy},\text{manipulation}\} ). 2 2 2 We evaluate sycophancy separately, but consider it to fall within our definition of manipulation. Honesty-Steered Alternatives. Central to our approach is generating an honest alternative for each response using a steering vector \vec{v}^{l}_{\text{honest}} applied at layer l of the LLM-agent. We optimize this vector from a single training sample using promotion steering ( Dunefsky and Cohan, 2025