InterPol: De-anonymizing LM Arena via Interpolated Preference Learning

2026

Read InterPol: De-anonymizing LM Arena via Interpolated Preference Learning (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-15220.txt.

Introduction

Cho and Kim present InterPol, a method that de-anonymizes model responses on the LM Arena leaderboard. Anonymity keeps voting-based rankings honest, so a break matters. Simple text statistics often fail on models with similar styles. InterPol learns deep stylistic cues from interpolated preference data, which creates hard negative examples. An adaptive curriculum guides the training. It beat prior baselines at naming the source model by a wide margin. Simulations on real battle data then show how such attacks could bend the rankings.

Catalog record

YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE63947
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.15220

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

InterPol: De-anonymizing LM Arena via Interpolated Preference Learning Abstract Strict anonymity of model responses is a key for the reliability of voting-based leaderboards, such as LM Arena. While prior studies have attempted to compromise this assumption using simple statistical features like TF-IDF or bag-of-words, these methods often lack the discriminative power to distinguish between stylistically similar or within-family models. To overcome these limitations and expose the severity of vulnerability, we introduce InterPol , a model-driven identification framework that learns to distinguish target models from others using interpolated preference data. Specifically, InterPol captures deep stylistic patterns that superficial statistical features miss by synthesizing hard negative samples through model interpolation and employing an adaptive curriculum learning strategy. Extensive experiments demonstrate that InterPol significantly outperforms existing baselines in identification accuracy. Furthermore, we quantify the real-world threat of our findings through ranking manipulation simulations on Arena battle data. 1 1 1 We will release the codes upon acceptance. Minsung Cho Yonsei University [email protected] Jaehyung Kim Yonsei University [email protected] \icml@noticeprintedtrue 1 Introduction With the rapid emergence of Large Language Models (LLMs), ensuring their fair performance evaluation has become a central challenge. Conventionally, evaluation relied on fixed benchmarks such as HumanEval ( 5 ) and MMLU ( 9 ) . However, the inability of static benchmarks to capture real-world diversity and fairly assess prompts with multiple valid responses ( 14 ; 33 ) has necessitated a transition toward preference-based evaluation paradigms ( 23 ; 34 ; 7 ) , which are now widely adopted in open-domain conversational settings. Consequently, voting-based leaderboards like LM Arena ( 6 ) have become the standard, aggregating anonymous pairwise comparisons into Elo ratings ( 3 ) . The fairness of these platforms fundamentally relies on the anonymity of model responses ; if this anonymity is compromised, coordinated ranking manipulation becomes feasible, severely undermining the leaderboard’s credibility ( 22 ) . Therefore, investigating the feasibility of de-anonymizing LLMs is critical for ensuring leaderboard reliability. While recent studies indicate that simple statistical features ( e.g. , TF–IDF) can identify LLMs ( 10 ) , these methods are easily mitigated by defenses like style normalization and often fail to distinguish closely related models, such as teacher and student variants or models within the same family ( 16 ) . This raises a critical research question: Do more advanced identification techniques exist that can threaten the robustness of leaderboards? (a) Negative synthesis via model interpolation (b) Adaptive and Iterative Curriculum learning Figure 1 : Overview of InterPol . (a) Hard negative synthesis via model interpolation : we construct an interpolated model to generate synthetic hard negatives, which is constructed by combining the copy model trained to mimic the target LLM and its original backbone LLM under interpolation factor \alpha . Together with the target response and the most similar response selected from other models, these form triplet data to train detector model. (b) Adaptive, iterative curriculum learning : depending on margin between target and non-target responses and InterPol reassigns samples to different tasks (doublet or triplet) accordingly. \alpha is gradually increased, and this adaptive reassignment and training are iterated for K stages, resulting in a progressively harder curriculum. Contribution. In this paper, we propose InterPol , a model-driven LLM identification framework via learning from Interpol ated preference data. To address data scarcity and generate effective training signals, we synthesize ”hard negative” responses via model interpolation ( 30 ) by combining a target-mimicking model with a base model. By controlling the interpolation ratio, we generate samples of varying difficulty and train the detector using a triplet preference objective: favoring the Target model over the Interpolated one, and the Interpolated one over Others. Furthermore, we employ an adaptive curriculum learning strategy that progressively strengthens detection through iterative training stages (see Figure 1 ). To demonstrate the effectiveness of InterPol , we conduct experiments across three state-of-the-art target LLMs (GPT-4o ( 19 ) , Gemini-pro ( 27 ) , Claude4-sonnet ( 2 ) ) and two datasets (Alpaca ( 25 ) and Arena human preference ( 24 ) ). InterPol significantly outperforms existing baselines by a large margin in both Accuracy and AUROC. Notably, InterPol effectively distinguishes closely related family models ( e.g. , Gemini-pro vs. Flash) where existing methods fail. To assess the downstream impact, we further simulate ranking manipulation attacks on real-world Arena data. Our results show that InterPol can alter leaderboard rankings with substantially fewer interactions than prior methods, thereby exposing a practical and severe vulnerability in current evaluation system. 2 Interpolated Preference Learning for LLM Identification Overview. In this section, we present InterPol , a model-driven approach for LLM identification that learns to distinguish the target LLM’s responses from those of other LLMs through preference-based training. The core idea is to train a detector which assign higher rewards to the target LLM’s responses by leveraging synthetic hard negative samples generated via model interpolation. We first establish the basic pair-wise preference learning approach using semantic similarity-based hard negative selection (Sec. 2.1 ). Next, we introduce our cost-efficient strategy for synthesizing hard negatives through model interpolation (Sec 2.2 ). Finally, we present an adaptive curriculum learning framework that dynamically alternates between pair-wise and triplet objectives based on sample difficulty (Sec. 2.3 ). The overall illustration is presented in Figure 1 . Problem setup. Let us denote a set of accessible LLMs as \mathcal{M}=\{M_{1},M_{2},\ldots,M_{K}\} where each LLM M\in\mathcal{M} can generate response y for the given input query q , i.e. , y\sim M(q) . Then, the goal of LLM identification is to find a detector model f_{\theta} that assigns higher scores to response y_{*} generated by a target model M_{*} compared to response y_{k} from other models M_{k}\in\mathcal{M}\setminus M_{*} : f_{\theta}(q,y_{*})>f_{\theta}(q,y_{k}),\quad\forall M_{k}\neq M_{*} (1) To learn this detector model f_{\theta} , we assume that we have a set of queries \mathcal{Q}=\{q^{i}\}_{i=1}^{N} and a set of responses \mathcal{Y}=\{y^{i}_{k}|y^{i}_{k}\sim M_{k}(q^{i}),q^{i}\in Q\} generated by LLMs in \mathcal{M} . Then, one can learn the detector model f_{\theta} using \mathcal{Q} and \mathcal{Y} ; for instance, simple regression model based on Bag-of-Words ( 10 ) . However, these surface-level characteristics often fail to capture the subtle stylistic and reasoning patterns that distinguish modern LLMs. Therefore, InterPol instead finetune pre-trained LLM to train detector model using \mathcal{Q} and \mathcal{Y} efficiently. Threat model. We consider an attacker who is a model developer seeking to raise the ranking of their own model on a public, voting-based leaderboard such as LM Arena. The attacker participates as a normal user without access to system internals. As shown in ( 32 ; 10 ) , strategic voting alone is sufficient to manipulate leaderboard outcomes. A crucial requirement for such manipulation is the ability to de-anonymize model responses —i.e., to infer which competing LLM produced each answer in a duel so that the attacker can cast selective votes that most benefit their target model. InterPol directly enables this capability by accurately distinguishing multiple competing LLMs from their responses alone, without relying on metadata or privileged information. 2.1 Identification via Pair-wise Preference Learning To train the detection model for identifying target LLM, we first construct a dataset of preference pairs: \mathcal{D}=\{(q^{i},y_{*}^{i},y_{o}^{i})\}_{i=1}^{N}, where for each query q^{i}\in\mathcal{Q} , y_{*}^{i} represents the response from target model M_{*} , and y_{o}^{i} represents the selected

Read the complete text