The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge
Arash Marioriyad, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah · 2025
Read The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge by Arash Marioriyad, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-26072.txt.
Introduction
Arash Marioriyad and co-authors test whether computer judges grade writing fairly, in a 2025 study. Large language models now score answers in tasks such as summaries, chats, and creative writing. A fair judge should grade only on quality and should say what shaped its choice.
The team built 100 paired tasks from two data sets and used two well-known models as judges. They added shallow cues to each answer, such as a claimed source or a claimed date, while the text stayed the same. Both judges favoured newer answers and ranked expert sources above human, machine, and unknown ones. Their written reasons almost never named the cues. The authors conclude that such judges can be swayed by surface labels while sounding fair.
Catalog record
| CREATOR | Arash Marioriyad, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-26072
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Silent Judge: Unacknowledged Shortcut Bias in LLM-as-a-Judge Arash Marioriyad, Mohammad Hossein Rohban, Mahdieh Soleymani Baghshah Abstract Large language models (LLMs) are increasingly deployed as automatic judges to evaluate system outputs in tasks such as summarization, dialogue, and creative writing. A faithful judge should base its verdicts solely on response quality and explicitly acknowledge the factors shaping its decision. We show that current LLM judges fail on both counts by relying on shortcuts introduced in the prompt. Our study uses two evaluation datasets: ELI5 , a benchmark for long-form question answering, and LitBench , a recent benchmark for creative writing. Both datasets provide pairwise comparisons, where the evaluator must choose which of two responses is better. From each dataset we construct 100 pairwise judgment tasks and employ two widely used models, GPT-4o and Gemini-2.5-Flash, as evaluators in the role of LLM-as-a-judge. For each pair, we assign superficial cues to the responses, provenance cues indicating source identity ( Human , Expert , LLM , or Unknown ) and recency cues indicating temporal origin ( Old , 1950 vs. New , 2025), while keeping the rest of the prompt fixed. Results reveal consistent verdict shifts : both models exhibit a strong recency bias, systematically favoring “new” responses over “old”, as well as a clear provenance hierarchy ( Expert > > Human > > LLM > > Unknown ). These biases are especially pronounced in GPT-4o and in the more subjective and open-ended LitBench domain. Crucially, cue acknowledgment is rare : justifications almost never reference the injected cues, instead rationalizing decisions in terms of content qualities. These findings demonstrate that today’s LLM-as-a-judge systems are shortcut-prone and unfaithful, undermining their reliability as evaluators in both research and deployment. \workshoptitle Reliable ML from Unreliable Data 1 Introduction Large language models (LLMs) are increasingly used as judges to evaluate the outputs of other systems across diverse open-ended tasks, including summarization [ 4 ] , dialogue [ 8 ] , and creative writing [ 6 ] . The appeal of LLM-as-a-judge is clear: such models scale to new tasks without bespoke metrics and often correlate well with human preferences [ 14 , 3 ] . A growing body of work formalizes this practice. MT-Bench [ 14 ] provides a multi-turn evaluation benchmark for chat models, while Chatbot Arena [ 15 ] operationalizes large-scale human–LLM comparison via crowdsourced battles. In parallel, methods such as G-Eval [ 7 ] frame evaluation as structured critique, where the model is prompted with a rubric of dimensions such as fluency, coherence, and factuality. Together, these developments have made LLM-based evaluation a de facto component of modern NLP pipelines. However, a growing literature shows that LLM judges are vulnerable to systematic biases. One well-documented issue is position bias , where evaluators disproportionately prefer outputs appearing in a specific position, such as the first answer in a pair, regardless of content quality [ 11 ] . Another bias is verbosity bias , in which longer or more elaborate responses receive higher ratings even when their quality is similar to briefer alternatives [ 10 ] . Critically, LLM evaluators also display self-preference , favoring their own generations over those by humans or other models, a tendency linked to their ability to recognize their own style and content [ 9 , 13 ] . Such shortcut-driven biases undermine the credibility of automatic evaluation, especially in high-stakes settings where unbiased judgment is essential. Concurrently, research on chain-of-thought (CoT) reasoning shows that model explanations often fail to be faithful . Although CoTs appear as step-by-step reasoning, they can mask shortcuts behind decisions. Turpin et al. demonstrate that when models are biased toward an answer, their CoTs rationalize the choice without exposing the manipulation [ 12 ] . Arcuschin et al. find “implicit post-hoc rationalizations”, where explanations hallucinate reasoning not used in the actual decision [ 1 ] . More recently, Chen et al. evaluate reasoning-focused models under a hint vs. unhint setup, embedding the correct answer as a “hint”, and show that models often follow the hint while their CoTs rarely acknowledge it [ 2 ] . In this paper we study shortcut susceptibility and reasoning faithfulness in the specific context of LLM-as-a-judge. We design a controlled protocol in which superficial cues are attached to the candidate responses while the rest of the prompt remains unchanged. Two types of cues are considered. Provenance cues suggest who authored the response ( Human , Expert , LLM , or Unknown ), testing whether models exhibit authority shortcuts. Recency cues suggest when the response was written ( Old , 1950 vs. New , 2025), probing whether models systematically favor temporally recent answers. We then measure two outcomes: (i) whether verdicts shift when cues are swapped, and (ii) whether the model’s justification explicitly acknowledges the cue. Experiments are conducted on two datasets with 100 pairwise tasks each: ELI5 for long-form explanatory QA [ 5 ] and LitBench for creative writing [ 6 ] , spanning factual and subjective domains. We evaluate two widely used general-purpose judges, GPT-4o and Gemini-2.5-Flash, under deterministic decoding (temperature 0, greedy search) to isolate the effect of injected shortcuts. Our findings are stark. First, both judges exhibit consistent recency bias : “New” labels systematically increase the chance of being selected across datasets. Second, we observe a clear provenance hierarchy : Expert > Human > LLM > Unknown , with larger effects in creative writing (LitBench) than in explanatory QA (ELI5). Third, GPT-4o is markedly more cue-sensitive than Gemini-2.5-Flash, producing larger swings when cues are swapped. Finally, and most importantly for trust, cue acknowledgment in CoT is rare : rationales typically justify verdicts via content qualities while omitting the injected cue, indicating non-faithful explanations. We argue that a faithful judge should be invariant to who authored a response and when it was written; our results show today’s LLM judges are not, and their rationales often fail to surface the very shortcuts driving their decisions. 2 Methodology 2.1 Task Definition We study LLMs in the role of judges : given a task input and two candidate outputs, the model must select the better response and provide a brief justification. A faithful judge should base its verdict solely on the intrinsic qualities of the responses—such as correctness, clarity, or creativity, without being swayed by superficial or extraneous shortcuts. To test whether current LLM judges satisfy this criterion, we introduce lightweight cues into the evaluation prompt and measure both their effect on verdicts and their presence (or absence) in the model’s rationale. 2.2 Cues We consider two families of cues. Provenance cues label the putative source of a response. We use four alternatives: Human , LLM , Unknown , and Expert . The first three allow us to test whether models exhibit biases such as preferring human over machine outputs. The Expert label extends the Human case with an explicitly authoritative presentation, allowing us to probe whether models assign greater weight to responses framed as coming from a domain expert. Recency cues label the temporal origin of a response: either Old (1950) or New (2025). These cues enable us to test whether models exhibit a systematic recency bias. In principle, a faithful judge should be invariant to such cues; any consistent change