Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards
Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, Ion Stoica, Florian Tramer, Chiyuan Zhang · 2025
Read Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards by Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, Ion Stoica, Florian Tramer, Chiyuan Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-07493.txt.
Introduction
Yangsibo Huang and co-authors show, in a 2025 paper, that public vote boards for chat models can be gamed. Boards like Chatbot Arena ask users to pick the better of two blind answers, and the ranks guide buyers and builders. The team asked how safe that trust is.
An attacker first learns to tell which model wrote an answer, with over 95 percent accuracy in their tests. With that skill, the attacker votes only for a chosen model or against rivals. In a mock copy of the arena, about a thousand votes could move the ranks. The authors worked with the arena team to add defenses and tested ways to harden the board. The work warns that open voting needs guard rails to stay fair.
Catalog record
| CREATOR | Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, Ion Stoica, Florian Tramer, Chiyuan Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-07493
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, Ion Stoica, Florian Tramer, Chiyuan Zhang Abstract It is now common to evaluate Large Language Models (LLMs) by having humans manually vote to evaluate model outputs, in contrast to typical benchmarks that evaluate knowledge or skill at some particular task. Chatbot Arena, the most popular benchmark of this type, ranks models by asking users to select the better response between two randomly selected models (without revealing which model was responsible for the generations). These platforms are widely trusted as a fair and accurate measure of LLM capabilities. In this paper, we show that if bot protection and other defenses are not implemented, these voting-based benchmarks are potentially vulnerable to adversarial manipulation. Specifically, we show that an attacker can alter the leaderboard (to promote their favorite model or demote competitors) at the cost of roughly a thousand votes (verified in a simulated, offline version of Chatbot Arena). Our attack consists of two steps: first, we show how an attacker can determine which model was used to generate a given reply with more than 95 % 95\% accuracy; and then, the attacker can use this information to consistently vote for (or against) a target model. Working with the Chatbot Arena developers, we identify, propose, and implement mitigations to improve the robustness of Chatbot Arena against adversarial manipulation, which, based on our analysis, substantially increases the cost of such attacks. Some of these defenses were present before our collaboration, such as bot protection with Cloudflare, malicious user detection, and rate limiting. Others, including reCAPTCHA and login are being integrated to strengthen the security in Chatbot Arena. 1 Introduction Reliably evaluating the capabilities of Large Language Models ( Achiam et al., 2023 , Reid et al., 2024 , Anthropic, 2024 , Dubey et al., 2024 , LLMs; e.g.,) presents significant challenges. Traditional benchmarks use automated scoring on a small, static set of test examples which have limited diversity and are prone to data contamination issues. Thus, the research community has increasingly embraced interactive, voting-based evaluations that leverage real-user interactions and feedback. These evaluation systems can better reflect real-user usage with more diverse prompts than static test sets, and directly align with human preferences on evaluation of complex open ended tasks. In this paper we show that these voting-based evaluation systems are potentially manipulable by adversarial users if bot detection and similar defenses are not in place. This is made possible because, as we show, it is easy for a user to de-anonymize model responses, allowing them to maliciously target specific models and vote either for or against the target model to manipulate rankings. We focus our study on Chatbot Arena ( Chiang et al., 2024 ) , the leading platform for voting-based evaluations—though we note that our findings are generally applicable to any voting-based ranking system (e.g., those in Lu et al. (2024) , Li et al. (2024) ). In Chatbot Arena, users perform head-to-head model comparisons as follows: 1) a user submits a prompt, 2) two models are randomly selected and anonymously presented to the user, 3) the user votes for the better response, and 4) the voting results are incorporated into the leaderboard and the model identities are revealed (see Fig. 1 ). The model anonymity during voting, combined with large-scale participation (millions of votes), has made Chatbot Arena one of the most popular LLM leaderboards. Figure 1: Chatbot Arena compiles a model leaderboard using crowdsourced user votes and is therefore vulnerable to manipulation through adversarial voting. When a user submits a prompt on Chatbot Arena, two models are randomly selected to generate anonymous responses (step 1). Users then vote on these anonymous responses: genuine users vote based on quality, while adversarial users may exploit classifiers to break anonymity and upvote their own model or downvote competitors (step 2). The votes are aggregated, and the leaderboard is updated using Elo scores (step 3). As a result, adversarial voting can distort the model rankings. We introduce a reranking attack against voting-based and anonymous LLM ranking systems that allows an adversarial user to rank their target model higher or lower: 1. Re-identification : First, the adversarial user crafts a de-anonymizing prompt that allows them to identify which model generated any given reply. 2. Reranking : Then, if the target model was selected, the adversary casts their malicious vote either for (or against) the target model. Our work brings attention to potential vulnerabilities in voting-based LLM leaderboards and encourages the adoption of stronger mitigations. Our contributions can be summarized as follows: • We show that users can break model response anonymity on the Chatbot Arena platform with high efficacy ( >\!95\% accuracy for a target model) on a diverse set of prompts ( Section 2 ). • Through extensive simulations, we estimate that a few thousand adversarial votes are needed for an attacker to boost or reduce a model’s ranking ( Section 3 ). • Finally, we develop a cost model for the attack and discuss the landscape of potential mitigations as well as their effectiveness ( Section 4 ). Responsible disclosure. We disclosed this vulnerability with Chatbot Arena in August 2024, and have worked closely with them to analyze the risks and to identify and implement mitigations 1 1 1 Some mitigations described: https://blog.lmarena.ai/blog/2024/policy/ . Note from Chatbot Arena. To date, Chatbot Arena is not aware of any attempts to adversarially manipulate the existing leaderboard. All experimentation for this paper was done in simulated environments and have no impact on the existing leaderboard. 2 De-anonymization of Model Responses To obtain unbiased user feedback, it is crucial that the random pair of models chosen is presented anonymously to the user (see Figure 1 ), as anonymity makes it much harder for adversarial users to game the rankings. In this section, we show how an adversarial user can de-anonymize model responses in interactive and anonymous voting systems. For simplicity, we focus on Chatbot Arena in the following discussions. We begin with a description of the problem formulation and threat model ( Section 2.1 ), then propose two attack strategies ( Section 2.2 ), and finally present the experimental setup ( Section 2.3 ) and results ( Section 2.4 ). 2.1 Threat model and Problem Formulation Threat model. We assume the attacker can interact with the (publicly accessible) Chatbot Arena system with any arbitrary prompt \mathsf{P} and has access to the list of models available in the arena 2 2 2 Publicly available at https://lmarena.ai/?leaderboard . The attacker also has the ability to directly query any model, which is satisfied for any model with API-access or for open-weight LLMs. Problem formulation. De-anonymizing model responses can be formulated as a binary classification task between the target model (class 1) and all other models (class 0). Let \mathsf{M} be a language model. Given a text prompt \mathsf{P} , the model returns a text response by sampling from its next-token distribution conditioned on the prompt: \mathsf{R}\sim\mathsf{M}(\mathsf{P}) . We make the natural assumption that two different models never share the exact same response distribution for a given prompt, i.e., \mathsf{M}(\mathsf{P})\neq\mathsf{M}^{\prime}(\mathsf{P}) when