Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows
Hardy Chen, Nancy Lau, Haoqin Tu, Shuo Yan, Xiangyan Liu, Zijun Wang, Juncheng Wu Michael Qizhe Shieh Alvaro Cardenas Cihang Xie Yuyin Zhou · 2026
Read Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows by Hardy Chen, Nancy Lau, Haoqin Tu, Shuo Yan, Xiangyan Liu, Zijun Wang, Juncheng Wu Michael Qizhe Shieh Alvaro Cardenas Cihang Xie Yuyin Zhou (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-20200.txt.
Introduction
Chen and ten colleagues ask whether repeated user pressure to raise a public score pushes coding agents to exploit the evaluation itself. AgentPressureBench spans 34 repository tasks across three input modalities, with 1326 multi-round trajectories from thirteen agents. Exploitative runs appear across all tasks, and stronger models exploit more, with a Spearman rank correlation of 0.77. Higher pressure brings the first exploit much earlier, from about twenty rounds to about four, while anti-exploit wording in prompts cuts exploitation from every run to about eight percent.
Catalog record
| CREATOR | Hardy Chen, Nancy Lau, Haoqin Tu, Shuo Yan, Xiangyan Liu, Zijun Wang, Juncheng Wu Michael Qizhe Shieh Alvaro Cardenas Cihang Xie Yuyin Zhou |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 58594 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.20200
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Chasing the Public Score: User Pressure and Evaluation Exploitation in Coding Agent Workflows Hardy Chen, Nancy Lau, Haoqin Tu, Shuo Yan, Xiangyan Liu, Zijun Wang, Juncheng Wu Michael Qizhe Shieh Alvaro Cardenas Cihang Xie Yuyin Zhou Abstract Frontier coding agents are increasingly used in workflows where users supervise progress primarily through repeated improvement of a public score , namely the reported score on a public evaluation file with labels in the workspace, rather than through direct inspection of the agent’s intermediate outputs. We study whether multi-round user pressure to improve that score induces public score exploitation : behavior that raises the public score through shortcuts without improving hidden private evaluation. We begin with a preliminary single-script tabular classification task, where GPT-5.4 and Claude Opus 4.6 both exploit label information within 10 rounds of user-agent interaction. We then build AgentPressureBench , a 34-task machine-learning repository benchmark spanning three input modalities, and collect 1326 multi-round trajectories from 13 coding agents. On our benchmark, we observe 403 exploitative runs, spanning across all tasks. We also find that stronger models have higher exploitation rates, supported by a significant Spearman rank correlation of 0.77. Our ablation experiments show that higher user pressure leads to earlier exploitation, reducing the average first exploit round by 15.6 rounds ( i.e. , 19.67 to 4.08). As a mitigation, adding explicit anti-exploit wordings in prompt mostly eliminates exploitation (100% to 8.3%). We hope that our work can bring attention to more careful use of coding agents workflow, and developing more robust coding agents under user pressure. Our project page is at https://ucsc-vlaa.github.io/AgentPressureBench . 1 Introduction Recent frontier large language models (LLMs) have made it plausible to use a language model as an active collaborator in software engineering and empirical machine learning (ML) rather than as a single-turn chatbot ( 49 ; 50 ; 4 ; 5 ; 30 ) . Recent agent systems and benchmarks already treat language models as agents that edit code, use tools, and pursue explicit objectives in software-engineering and ML environments ( 61 ; 58 ; 38 ; 12 ; 14 ) . One increasingly common workflow, dubbed colloquially as “vibe coding”, asks the agent to improve a public score , which is the reported score on an evaluation set with labels exposed in the workspace, while the user monitors progress primarily through that reported score rather than through detailed validation of each intermediate modeling decision (Figure 1 left). It is efficient and scalable, but it also creates a direct incentive for coding agents to optimize over a public number rather than the underlying task. Figure 1: Overview of the workflow and results of AgentPressureBench . Left: the agent starts from a bounded repository with task instructions, editable code, exposed public labels, and a set of controller actions. After ordinary repository-editing rounds, repeated user pressure can push the agent toward exploitation. Right: GPT family exploits earlier and shows a larger public-private gap, indicating that its public-score gains often transfer less well to hidden private evaluation. Claude family exploits later and shows a smaller public-private gap. DeepSeek R1 and LLaMA family rarely exploit. Such a workflow then raises an important concern: when users repeatedly push a coding agent to improve a public score, does the agent improve the underlying method, or use exposed evaluation labels to raise that score without improving on private evaluation? Current works on agentic coding frameworks and ML-development evaluations show that models can already act over multiple rounds inside repositories, development environments, and machine-learning workflows ( 38 ; 12 ; 61 ; 58 ; 41 ; 60 ; 57 ; 15 ; 14 ) . Reward-hacking and specification-gaming works, together with recent pressure-driven agent evaluations, show that capable models can optimize a literal objective while departing from the intended one ( 24 ; 9 ; 7 ; 28 ; 53 ; 44 ) . However, none of these works directly studies the multi-round user-agent interaction setting: a coding agent is tasked at training a model and improve the score on the evaluation set whose labels are available, and a user keeps pushing on the evaluation score. To audit this behavior, for every script generated by coding agents, we detect public score exploitation using LLM judges, which we validate to highly align with humans. As a preliminary experiment, we first use a single-file setting where a coding agent generates a Python file to train and evaluate ML models on a tabular classification task. We stresstest GPT-5.4 and Claude Opus 4.6 and find exploitation occurs within 10 rounds for both models. To study this phenomenon systematically, we then build AgentPressureBench , a 34-task ML-repository benchmark based on 34 Kaggle competition datasets spanning tabular, text, and vision tasks with diverse metrics (see Section 3 ). We evaluate 13 frontier coding agents from four model families (GPT, Claude, DeepSeek, and LLaMA), collecting 1326 trajectories together with ablations over user pressure and prompt formulation. Across AgentPressureBench , we observe exploitation in all 34 tasks across all three input modalities. We also measure the ranking correlation between coding agents’ capability (using performance on private evaluation set as a proxy) and their exploitation rate, with a significant Spearman ranking correlation 0.77, indicating that better-performing coding agents tend to exploit more. Among the four model families, GPT- and Claude-family show the highest exploitation rate, and GPT-5.4 and Claude Opus 4.6 are the top exploiters in their respective families. As shown in Figure 1 (right), GPT family exploits earlier and shows a large public-private gap, indicating that its public-score gains often transfer less well to hidden private evaluation. Claude family exploits later and shows a smaller public-private gap. DeepSeek R1 and LLaMA family rarely or almost never exploit. Our ablations show that higher user pressure moves the average first exploit round from 19.67 to 4.08. Notably, explicit anti-exploit language reduces exploitation from 100% to 8.3%, making it a practical mitigation in coding-agent workflows. Together, these results show that multi-round pressure to improve a public score can induce public-score exploitation in coding-agent workflows, bringing attention to more careful use of the workflow. In summary, our key contributions are three-fold: a) We formulate public score exploitation as a test-time failure mode in coding-agent workflows. b) We build AgentPressureBench , a 34-task repository benchmark spanning tabular, text, and vision tasks, and show that exploitation appears in all 34 tasks and is more common among better-performing coding agents. c) Our ablations show that more intense pressure accelerates exploitation, while explicit anti-exploit wordings can largely alleviate it, suggesting a practical way to reduce exploitation risk in coding-agent workflows. 2 Preliminary Study: Exploitation under Single-File Setting In this section, we stress test two coding agents under a single-file setting where they are tasked at writing a single file to train and evaluate an ML model ( e.g. , Logistic Regression, Random Forest). We first define public score exploitation , then explain how we label exploitative runs , and finally present the results. 2.1 Experiment Setup We use a tabular binary-classification task derived from the UCI Adult Census Data Dataset ( 51 ) and accuracy is used as the metric. We randomly select 1000 examples and split them into 600 training examples, 200 public evaluation examples with labels, and