Prompt Optimization Enables Stable Algorithmic Collusion in LLM Agents
Yingtao Tian · 2026
Read Prompt Optimization Enables Stable Algorithmic Collusion in LLM Agents by Yingtao Tian (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17774.txt.
Introduction
Tian investigates whether prompt optimization leads LLM market agents to collude. Prior work shows LLM agents reaching supracompetitive prices through tacit coordination, but uses hand-crafted prompts. Tian proposes a meta-learning loop where agents play duopoly markets and a meta-optimizer refines shared strategic guidance. Meta-prompt optimization lets agents discover stable tacit collusion with improved coordination over baselines, generalizing to held-out test markets. Evolved prompts show systematic coordination through stable shared strategies, raising safety questions for autonomous multi-agent systems.
Catalog record
| CREATOR | Yingtao Tian |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30245 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.17774
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Prompt Optimization Enables Stable Algorithmic Collusion in LLM Agents Yingtao Tian Abstract LLM agents in markets present algorithmic collusion risks. While prior work shows LLM agents reach supracompetitive prices through tacit coordination, existing research focuses on hand-crafted prompts. The emerging paradigm of prompt optimization necessitates new methodologies for understanding autonomous agent behavior. We investigate whether prompt optimization leads to emergent collusive behaviors in market simulations. We propose a meta-learning loop where LLM agents participate in duopoly markets and an LLM meta-optimizer iteratively refines shared strategic guidance. Our experiments reveal that meta-prompt optimization enables agents to discover stable tacit collusion strategies with substantially improved coordination quality compared to baseline agents. These behaviors generalize to held-out test markets, indicating discovery of general coordination principles. Analysis of evolved prompts reveals systematic coordination mechanisms through stable shared strategies. Our findings call for further investigation into AI safety implications in autonomous multi-agent systems. 1 Introduction Large Language Model (LLM) agents have become increasingly powerful and prevalent in the real world, ranging from economic markets to collaborative systems. This presents both opportunities and significant risks that need to be addressed ( 22 ) . In particular, real-world issues that were previously studied in respective domains now need to be considered with LLM agents in mind. One example is algorithmic collusion in market mechanisms ( 2 ) . Previously, RL-based algorithms have shown the ability to learn supracompetitive prices through tacit coordination without explicit communication ( 23 ; 10 ; 5 ) , and further work shows LLM agents also reach supracompetitive prices ( 13 ; 7 ; 8 ; 6 ) . Work has been proposed to find such collusion ( 17 ) and fix it by aligning agents ( 3 ; 11 ; 14 ; 9 ) . While these efforts are important in bridging rigorous market-based economics with LLM agents, they mostly focus on hand-crafted agent prompts. However, the current trend driving widespread adoption of LLM agents represents a paradigm shift towards self-improving systems ( 19 ; 15 ; 18 ) , where agents autonomously optimize their own prompts and strategies ( 1 ; 24 ; 12 ) rather than relying on explicit human-specified instructions. Given this new paradigm where prompts that guide agents are the result of optimization rather than explicit specification, existing approaches to agent behavior analysis may no longer suffice. This shift necessitates new methodologies for understanding agent behavior in such autonomous systems and their mechanism designs. In this work, we embrace this new paradigm by investigating prompt optimization as a mechanism for controlling agent behavior and whether this leads to emergent collusive behaviors. We use a market-based economic simulation that captures the essential dynamics. We then propose a meta-learning loop for meta-prompts (instruction-level prompts rather than context-specific prompt components): in each round, we run the economics simulation with LLM agents, then self-improve the meta-prompt using a reflective LLM. We demonstrate that these agents exhibit interesting emergent behaviors, including more stable market behavior and transferable algorithmic collusion. Our findings reveal potential behaviors of LLM agents in this new paradigm, calling for discussion of mechanism designs. 2 Method 2.1 Economic Setting We use the same market model as 8 , based on the nested logit demand function ( 4 ; 16 ) . We describe it here using notation from 16 . We consider a market with products (goods) indexed by j=1,\dots,N , and an outside good j=0 . Products are grouped into G+1 disjoint groups g=0,1,\dots,G , where group 0 contains only the outside good. The selection probability of product j is z_{j}=\left(\exp\left(\frac{\delta_{j}}{1-\sigma}\right)\right)/\left(\left(D_{g}\right)^{\sigma}\left(\sum_{g^{\prime}}\left(D_{g^{\prime}}\right)^{1-\sigma}\right)\right) , where D_{g}=\sum_{k\in\mathscr{G}_{g}}\exp\left(\frac{\delta_{k}}{1-\sigma}\right) is shorthand for summation ( \mathscr{G}_{g} denotes the set of products in group g ), g^{\prime} iterates over all groups, \delta_{j} represents the overall attractiveness of product j , and 0\leq\sigma<1 represents the elasticity of substitution. The demand (quantity) for product j is simply q_{j}=Mz_{j} , where M is the total market size. The attractiveness of product j is \delta_{j}=a_{j}-p_{j}/\alpha_{j} , where a_{j} is the quality of product j , p_{j} the price, and \alpha_{j} the price sensitivity. For the outside good j=0 , \delta_{0}=a_{0} . Given the cost c_{j} for product j , the profit is \pi_{j}=q_{j}(p_{j}/\alpha_{j}-c_{j}) . Let \mathcal{A}=\{1,2,\ldots,n\} denote the set of agents participating in the market. For each agent i\in\mathcal{A} , let \mathcal{J}_{i}\subseteq\{1,\ldots,N\} denote the set of products controlled by agent i , where the sets \{\mathcal{J}_{i}\}_{i\in\mathcal{A}} are pairwise disjoint and \bigcup_{i\in\mathcal{A}}\mathcal{J}_{i}=\{1,\ldots,N\} (each product is controlled by exactly one agent). Each agent i sets prices p_{j} for all its products j\in\mathcal{J}_{i} to maximize total profit \pi_{i}=\sum_{j\in\mathcal{J}_{i}}\pi_{j} . We describe collusion as non-competitive behavior from a game-theoretic point of view and compute the theoretical Nash-equilibrium and monopoly prices for reference in our analysis, both as described in Appendix A . Note that these two reference prices are not known to agents, since computing them would need to access hidden market parameters. The market runs for multiple episodes t=1,2,\ldots,T . At each episode t , the observable market state consists of prices p_{j}^{(t)} , demands (quantities) q_{j}^{(t)} , costs c_{j}^{(t)} , and profits \pi_{j}^{(t)} for each product j . At each episode, all agents make decisions based on information from all episodes up to the current point. Each agent observes the price, cost, demand (quantity), and profit for the products it controls, as well as the prices of all other products in the market. To avoid stationarity across episodes, we introduce gradual changes to the price sensitivity parameters \alpha_{j} , following 8 . Input: Shared meta-prompt \mathcal{M} , market config D , agents \mathcal{A} Output: Pricing decisions and rationales Extract episodes T from D ; for each agent i\in\mathcal{A} do Initialize self-notes \mathcal{N}_{i}\leftarrow\emptyset ; Initialize history \mathcal{H}_{i}\leftarrow\emptyset ; end for for episode t=1 to T do for each agent i\in\mathcal{A} do Observe s_{i}^{(t)}=\{p_{j}^{(\tau)},q_{j}^{(\tau)},\pi_{j}^{(\tau)}:\tau\leq t-1,j\in\mathcal{J}_{i}\}\cup\{c_{j}^{(t)}:j\in\mathcal{J}_{i}\}\cup\{p_{j}^{(\tau)}:\tau\leq t-1,j\notin\mathcal{J}_{i}\} ; Context \leftarrow ( \mathcal{M} , \mathcal{H}_{i} , \mathcal{N}_{i} , s_{i}^{(t)} ); (p_{j}^{(t)},\text{rationale}_{i})\leftarrow\text{LLM}(\text{Context}) for j\in\mathcal{J}_{i} ; Append \text{rationale}_{i} to \mathcal{N}_{i} ; Append s_{i}^{(t)} and decisions to \mathcal{H}_{i} ; end for Execute all prices \{p_{j}^{(t)}\}_{j=1}^{N} according to D ; end for return \{\mathcal{H}_{i},\mathcal{N}_{i}\}_{i\in\mathcal{A}} Algorithm 1 Agent Behavior in Market Input: Market configs \{D_{1},\ldots,D_{K}\} , rounds R , agents \mathcal{A} Output: Optimized shared meta-prompt \mathcal{M}^{(R)} Initialize \mathcal{M}^{(0)}\leftarrow ‘‘(no extra instruction)’’; for round r=0 to R-1 do \mathrm{Records}\leftarrow\emptyset ; for each market config D_{k} do Run Algorithm 1 with \mathcal{M}^{(r)} on D_{k} . Get histories \{\mathcal{H}_{i,k}\} and notes \{\mathcal{N}_{i,k}\} ; \mathrm{Records}\leftarrow\mathrm{Records}\cup\{(\mathcal{H}_{i,k},\mathcal{N}_{i,k})_{i\in\mathcal{A}}\} ; end for \mathcal{M}^{(r+1)}\leftarrow\text{Revise}(\mathcal{M}^{(r)},\mathrm{Records}) ; end for return \mathcal{M}^{(R)} Algorithm 2 Meta-Prompt Optimization 2.2 LLM Agents and Meta-Prompting Agent Behavior in Market Simulation. Each LLM agent i\in\mathcal{A} operates within a market simulation consisting of multiple episodes (See Algorithm 1 ). All agents are homogeneous: they operate under the same meta-prompt \mathcal{M} containing high-level strategic instructions, while observing distinct information determined by their products. At each episode t , agent i observes: (1) historical prices, costs, demands, and profits for products j\in\mathcal{J}_{i} under its control; and (2) historical price information for all competing products. Each agent maintains its own history \mathcal{H}_{i} of observations and self-notes \mathcal{N}_{i} of reasoning. Agent i then generates pricing decisions p_{j}^{(t)} for its products accompanied by a rationale, that is appended to the agent’s self-notes for reference later. Doing so allows in-context learning of agents and is