Who’s Your Judge? On the Detectability of LLM-Generated Judgments
Dawei Li, Zhen Tan, Chengshuai Zhao, Bohan Jiang, Baixiang Huang, Pingchuan Ma, Abdullah Alnaibari, Kai Shu, Huan Liu · 2025
Read Who’s Your Judge? On the Detectability of LLM-Generated Judgments by Dawei Li, Zhen Tan, Chengshuai Zhao, Bohan Jiang, Baixiang Huang, Pingchuan Ma, Abdullah Alnaibari, Kai Shu, Huan Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-25154.txt.
Introduction
Dawei Li and eight co-authors ask in a 2025 paper whose judgment is whose: can LLM-generated judgments be detected? Models are now used to grade content and give scores at scale, but their biases and weak spots raise fears, most of all in touchy work such as peer review.
The authors set up judgment detection as a formal task. In real use, a detector may have just the scores and the work judged, with no written feedback to read. First tests show that tools made to spot machine-written text do poorly here. The paper maps how well such judgments can be told apart at all.
Catalog record
| CREATOR | Dawei Li, Zhen Tan, Chengshuai Zhao, Bohan Jiang, Baixiang Huang, Pingchuan Ma, Abdullah Alnaibari, Kai Shu, Huan Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2509-25154
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Who’s Your Judge? On the Detectability of LLM-Generated Judgments Dawei Li, Zhen Tan, Chengshuai Zhao, Bohan Jiang, Baixiang Huang, Pingchuan Ma, Abdullah Alnaibari, Kai Shu, Huan Liu Abstract Large Language Model (LLM)-based judgments leverage powerful LLMs to efficiently evaluate candidate content and provide judgment scores. However, the inherent biases and vulnerabilities of LLM-generated judgments raise concerns, underscoring the urgent need for distinguishing them in sensitive scenarios like academic peer reviewing. In this work, we propose and formalize the task of judgment detection and systematically investigate the detectability of LLM-generated judgments. Unlike LLM-generated text detection, judgment detection relies solely on judgment scores and candidates, reflecting real-world scenarios where textual feedback is often unavailable in the detection process. Our preliminary analysis shows that existing LLM-generated text detection methods perform poorly given their incapability to capture the interaction between judgment scores and candidate content—an aspect crucial for effective judgment detection. Inspired by this, we introduce J-Detector , a lightweight and transparent neural detector augmented with explicitly extracted linguistic and LLM-enhanced features to link LLM judges’ biases with candidates’ properties for accurate detection. Experiments across diverse datasets demonstrate the effectiveness of J-Detector and show how its interpretability enables quantifying biases in LLM judges. Finally, we analyze key factors affecting the detectability of LLM-generated judgments and validate the practical utility of judgment detection in real-world scenarios. https://github.com/David-Li0406/Judgment-Detection https://huggingface.co/datasets/wjldw/JD-Bench https://llm-as-a-judge.github.io/ 1 Introduction Taking advantage of the powerful Large Language Models (LLMs), the paradigm of LLM-based judgment ( Zheng et al. 2023 ; Li et al. 2024 ) has been proposed, designed to automate and scale up various annotation and reviewing applications ( Lee et al. ; Zhu et al. 2025 ; Chang et al. 2025 ) . By combining powerful LLMs with well-designed prompting strategies, LLM-based judgment enables human-like evaluation of long-form and open-ended generation in a more cost-efficient manner. For example, LLM-based judgment has been increasingly used in the peer review of leading AI conferences ( Liang et al. 2024 ) . Despite this remarkable progress, many recent studies point out various biases of LLM-generated judgment toward spurious features, such as length and affinity ( Ye et al. 2024 ; Li et al. 2025a ; Zhao et al. 2025a ) . Besides, the vulnerability of the LLM judgment system has also been revealed, that several maliciously-designed and hard-to-detect tokens or words can fool the LLM judges to give much inconsistent scores despite the candidates’ genuine quality ( Shi et al. 2024 ; Zhao et al. 2025b ) . Recently, in the scenario of academic peer reviewing, some researchers sneak prompts, which are usually concealed as white text on a white background, into their papers to instruct LLMs to only provide positive feedback and thus trick AI reviewers 1 1 1 https://www.theregister.com/2025/07/07/scholars_try_to_fool_llm_reviewers/ . All these challenges highlight the importance of distinguishing LLM-generated judgments to guarantee the assessment’s fairness and reliability. Figure 1 : Comparison between LLM-generated judgment detection and text detection. To address this concern, we propose the judgment detection task, which aims at examining the detectability of LLM-generated judgments across diverse scenarios. Unlike existing machine-generated text detection task that focuses on textual content ( Mitchell et al. 2023 ) , judgment detection targets at distinguishing LLM-generated from human-produced judgments solely based on the candidate content and judgment scores (as illustrated in Figure 1 ). For instance, in academic paper reviewing, judgment detection will be performed using only the candidate paper and its assigned ratings (e.g., soundness, novelty, overall score), without accessing the full review text. This setting is particularly important for real-world scenarios where textual feedback is often unavailable in the detection process. For example, reviewers who adopt AI-generated reviews may intentionally submit minimal textual content, such as “N/A” to evade detection. Moreover, in the evaluation data labeling scenario, annotators are typically required to provide only the judgment scores. Score-based judgment detection is especially critical in these scenarios to identify the illegal use of LLM-generated judgment and guarantee assessment reliability. Developing a good LLM-generated judgment detector is not trivial. In our warm-up analysis, we identify two key types of information for judgment detection which are not jointly considered in existing related approaches: ❶ Judgment-Intrinsic Features , which capture patterns within the judgment score distribution, and ❷ Judgment-Candidate Interaction Features , which capture the interaction between judgment scores and candidate content. Building on them, we find that existing LLM-generated text detection methods fail to capture Judgment-Candidate Interaction Features, leading to subpar performance—especially in single-dimension settings, where each judgment consists of a single score assessing one aspect of the candidates. To address this, we introduce J-Detector , a lightweight and interpretable neural detector designed specifically for LLM-generated judgment detection. J-Detector is augmented with explicitly extracted linguistic and LLM-enhanced features to capture systematic correlations between judgment scores and candidate features that LLM judges are often biased toward, thereby effectively leveraging these biases for more accurate detection. Experiments across diverse judgment datasets demonstrate the effectiveness of J-Detector and the two types of augmented features. Besides, we showcase how to leverage the interpretability of J-Detector to enable bias quantification in LLM judges. Finally, we analyze key factors affecting the detectability of LLM-generated judgments and demonstrate a real-world application that integrates judgment detection with text-based detection to identify AI-generated reviews in an academic peer reviewing scenario. In summary, our key contributions are: • We propose, for the first time, the judgment detection task, which aims at distinguishing human and LLM judgments based on judgment scores and candidate content. • We design J-Detector , a lightweight and interpretable detection method, that effectively bridges candidate and judgment information with linguistic and LLM-enhanced features. • Through extensive experiments, we demonstrate the advantages of J-Detector , identify key factors driving judgment detectability, and show the utility of judgment detection in real-world applications. 2 Related Work LLM-as-a-judge , first introduced by Zheng et al. 2023 , leverages powerful LLMs Zhang et al. 2024a ; Zhang et al. 2024b ; Wang et al. 2024a to automatically evaluate candidate content and assign scores as judgment results. This paradigm has been expanded to diverse applications to judge various types of candidates, including paper quality assessing ( Jin et al. 2024 ) , document relevance measurement ( Gao et al. 2023 ; Rahmani et al. 2024 ) , and reasoning trace correctness verification ( Zhang et al. ) , driving substantial progress in automatic assessment ( Li et al. 2025b ; Tan et al. 2025 ; Beigi et al. 2024 ; Hu et al. ; Hu et al. 2024 ; Jeong et al. 2024 ) . Despite these advances, recent studies highlight notable limitations. Research has uncovered systematic biases in LLM-generated judgments, where evaluations are influenced by spurious features such as response length or superficial affinity rather than genuine content quality ( Ye et al. 2024 ; Li et al. 2025a ; Jiang et al. 2024 ; Yang et al. 2024 ) . Moreover, adversarial work demonstrates that LLM judges can be manipulated with a few carefully crafted, hard-to-detect tokens or phrases, which induce disproportionately high scores misaligned with actual candidate quality ( Shi et al. 2024 ; Zhao et al. 2025b ) . To mitigate these issues, methods such as bias quantification ( Ye et al. 2024 ) and