Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce
Shimaa Ahmed *, Yiwei Cai *, Mohsen Minaei *, Rahul Rachuri * · 2026
Read Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce by Shimaa Ahmed *, Yiwei Cai *, Mohsen Minaei *, Rahul Rachuri * (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2608-00102.txt.
Introduction
Ahmed and co-authors test whether LLM agents can price well in live markets. Agentic commerce is moving from concept to deployed infrastructure. Payment networks, retailers, and AI platforms set the stage for agent transactions. Real markets hide customer preferences and shift demand without warning. Competitors also adapt in real time.
Their Bazaar benchmark models dynamic sealed-bid auctions with multiple attributes. Closed-form customer utilities allow exact scoring. Eleven frontier models from four providers were tested. Leaders on customer acquisition were often not leaders on profit. Demand shocks reshuffled the rankings again. Fast pre-shock learners were often slowest to revise beliefs after a shock. Even the best agent captured under a third of hindsight-optimal profit.
Catalog record
| CREATOR | Shimaa Ahmed *, Yiwei Cai *, Mohsen Minaei *, Rahul Rachuri * |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 81 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2608.00102
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Can LLM Agents Price Competitively? A Dynamic Multi-Attribute Auction Benchmark for Agentic Commerce Shimaa Ahmed *, Yiwei Cai *, Mohsen Minaei *, Rahul Rachuri * Abstract Agentic commerce is moving from concept to deployed infrastructure: payment networks, retailers, and AI platforms are setting the stage for agents to transact on behalf of merchants and consumers. Yet whether the LLMs behind these agents can price competently in real markets, where customer preferences are hidden, competitors adapt in real time, and demand can shift without warning, has not been systematically tested. We introduce Bazaar, a dynamic sealed-bid benchmark for multi-attribute auction under these conditions. Despite its dynamics, the benchmark is grounded in closed-form customer utilities, enabling exact evaluation. Across 11 frontier LLMs from four providers, the leading agents on customer acquisition (e.g. Gemini 3.1 Pro) are often not the leading agents on profit (e.g. Opus 4.6). The ranking shifts again under demand shocks: agents that learned fastest pre-shock are typically the slowest to revise their beliefs afterwards, while Gemini 3.1 Pro recovers fastest despite not leading on profit. However, even the strongest agent captures less than a third of hindsight-optimal profit, suggesting current LLMs are progressing in agentic commerce but leave substantial headroom. * * footnotetext: Authors are listed in alphabetical order. 1 Introduction Language models are beginning to act as economic agents. In commerce, an agent can negotiate with customers, place bids, choose product bundles, and set prices on behalf of a buyer or seller Bansal et al. (2025) ; Zhu et al. (2025) ; Yu et al. (2026) . Recent systems such as Anthropic’s Project Vend Anthropic (2025) , Microsoft’s Magentic Marketplace Bansal et al. (2025) , and Alibaba’s Shopping Companion Yu et al. (2026) point toward marketplaces populated by LLM merchants that respond to customers and competitors in real time Bansal et al. (2025) ; Yu et al. (2026) . This trend raises a concrete evaluation question: when an LLM is placed inside a competitive market loop, can it learn what customers value, choose profitable offers, and adapt when demand changes? Existing benchmarks capture important pieces of this problem. E-commerce benchmarks Peng et al. (2024) ; Chen et al. (2025) evaluate product knowledge, intent understanding, and recommendation quality on fixed instances. Economic and game-theoretic benchmarks Chen et al. (2023) ; Shah et al. (2025) ; Bianchi et al. (2024) ; Hua et al. (2024) ; Shapira et al. (2024) study auctions, bargaining, and negotiation under stylized rules. Neither setting combines learning, pricing, and belief revision in the repeated, nonstationary competition that deployed merchants face. We introduce Bazaar, a framework for evaluating LLM merchants in dynamic multi-attribute auction markets. In Bazaar, an LLM merchant repeatedly chooses product configurations and prices for customers with hidden preferences, while competing against rival merchants with different cost advantages. The agent observes only sparse market feedback, so it must infer what each customer values, decide how much margin to charge, and update its strategy over time. Markets also shift: new trends, sudden events, and seasonal changes can reorder what customers want. To capture this pressure, Bazaar introduces unannounced preference shifts midway through the game. This lets us ask whether agents that learn quickly before a shift can also revise their beliefs afterward. We evaluate total profit together with whether an agent chooses the right configuration, charges the right price, and adapts after demand changes (Figure 1 ). Figure 1: Abstract overview of Bazaar. This paper makes three contributions: A diagnostic framework for dynamic LLM pricing (§ 2 ). Bazaar specifies a repeated sealed-bid multi-attribute auction protocol with private costs, hidden heterogeneous customers, sparse feedback, adaptive competitors, and preference shifts. Despite the dynamics, closed-form customer utilities and merchant costs enable an exact evaluation. Its surplus and oracle-regret diagnostics separate performance into preference learning (configuration choice), margin extraction (price), and post-shift revision. This lets us classify agents into diagnostic failure archetypes ( Loser , Underpricer ) and measure adaptation directly, rather than inferring from a single score. Thompson Sampling Thompson (1933) and EXP4 Auer et al. (2002) provide non-LLM baselines for binary win/loss learners in the same action space. Margin discipline is the discriminating skill, and thinking budget moves models along a two-dimensional failure surface (§ 4 ). Across 11 frontier LLMs, profit is almost perfectly aligned with margin per win ( r{=}0.99 ), and less so with win rate ( r{=}0.88 ). Increasing thinking effort shifts models along the failure surface: it lowers loss regret (fewer missed wins) but raises underprice regret (leaving money on the table), moving the same base model from a Loser archetype toward an Underpricer . This shift is economically large: GPT-5.4 (high) earns 7.3\times more than GPT-5.4 (none), and Opus 4.6 (adaptive, xhigh) substantially outperforms Opus 4.6 (off, high). Two distinct paths to the top of the leaderboard, and a strong-learner / weak-adapter reversal under shocks (§ 4.2 ). Opus 4.6 (adaptive, xhigh) leads on profit at $2,976 through margin discipline, earning $2.22 per win. Gemini 3.1 Pro (dynamic, high) leads on win rate (77.9%), recovers fastest from preference shocks, writes strategies roughly a third the length of competitors, and is also the most cost-effective agent we evaluate. The shock also reveals a strong-learner, weak-adapter pattern: the models that climb fastest before the shock (GPT-5.3 and Opus 4.5, gaining +46 and +40 pp in win rate over rounds 1–30) are among the worst at adapting afterward, losing -19 pp on average and not recovering. A static benchmark would never surface this failure mode. Our work connects four threads: LLM agents in auctions and negotiation, multi-agent LLM benchmarks, LLMs in e-commerce, and algorithmic pricing with regret-based evaluation. Appendix A discusses prior work in each. 2 Bazaar: A Dynamic Multi-Attribute Pricing Framework In this section, we define a framework for evaluating whether a merchant can learn profitable pricing policies in repeated, competitive, partially observed markets. In each round, merchants configure a product and set a price for each customer. The customer selects the offer that maximizes its utility, and these utilities may change over time. This abstraction captures a core challenge of agentic commerce: a merchant must learn what each customer values, how much margin it can charge, and when earlier beliefs should be revised. We use focal merchant to denote the merchant whose behavior is being measured; in our experiments this is the LLM agent under evaluation, but the framework definitions are merchant-agnostic. Figure 4 shows the market schematic. Symbol Meaning \mathcal{M} , f\in\mathcal{M} Merchants; focal merchant \mathcal{I} Customers t\in\{1,\ldots,T\} Global round index K,L Attributes and levels per attribute \mathbf{x}\in\{1,\ldots,L\}^{K} Attribute-level configuration p Offered price V_{i,t}(\mathbf{x}) Customer i ’s hidden value at round t U_{i,t}(\mathbf{x},p) Customer utility for offer (\mathbf{x},p) c_{m}(\mathbf{x}) Merchant m ’s production cost \pi_{m}(\mathbf{x},p) Merchant m ’s profit if it wins Table 1: Core notation for the framework. Diagnostic symbols ( q_{i,t} , \pi^{\star}_{i,t} , S_{i,m,t} ) are introduced inline in § 2.5 . 2.1 Repeated Multi-Attribute Auctions Let \mathcal{M} be the set of merchants and \mathcal{I} the set of customers. Time is divided into global rounds t\in\{1,\ldots,T\} . In every round, each customer i\in\mathcal{I} runs a sealed-bid auction among all merchants. A bid from merchant m to customer i is b_{m,i,t}=(\mathbf{x}_{m,i,t},p_{m,i,t}), (1) where \mathbf{x}_{m,i,t}=(x_{1},\ldots,x_{K})\in\{1,\ldots,L\}^{K} is a product configuration