A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions
Junhan Li, Yuxin Zhang, Haoran Wang, Minghao Chen · 2026
Read A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions by Junhan Li, Yuxin Zhang, Haoran Wang, Minghao Chen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28943.txt.
Introduction
Li, Zhang, Wang and Chen introduce A3M, a learning framework for bidding in repeated auctions where feedback is limited to outcomes. Bidders must learn while rivals change tactics. Existing methods often follow rigid explore-then-exploit schedules, assume rivals stay fixed, and chase bidder profit alone, which limits adaptability and robustness.
A3M combines three ingredients. An actor-critic deep reinforcement learning core balances exploration and exploitation dynamically as results arrive. An opponent model enables fictitious play, planning against imagined rivals, to handle adversaries whose strategies shift over time. A composite reward jointly pursues bidder utility, auctioneer revenue and fairness, with tunable trade-offs. In tests on both discriminatory and uniform price auctions, A3M cut final regret by thirty to forty percent in standard settings. It stayed robust when rivals shifted strategy and scaled well as units grew. Ablations confirmed each component was necessary.
Catalog record
| CREATOR | Junhan Li, Yuxin Zhang, Haoran Wang, Minghao Chen |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28943
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions Junhan Li, Yuxin Zhang, Haoran Wang, Minghao Chen Abstract Learning to bid in repeated multi-unit auctions with bandit feedback poses a fundamental challenge. Existing methods often rely on rigid explore-then-exploit schedules, assume stationary adversaries, and optimize solely for bidder utility, thereby limiting adaptability and strategic robustness. To address these limitations, we introduce the A3M framework, which integrates adaptive deep reinforcement learning (DRL), explicit adversarial reasoning, and principled multi-objective reward design for online auction strategy optimization. A3M employs an actor-critic DRL backbone to dynamically balance exploration and exploitation, an opponent model for fictitious play against non-stationary adversaries, and a composite reward function to jointly maximize utility, auctioneer revenue, and fairness. We provide the first comprehensive empirical evaluation of this integrated approach against established baselines in both discriminatory and uniform price auctions. Results show that A3M reduces final regret by 30–40% in standard settings, maintains robust performance against adversarial strategy shifts, scales favorably with the number of units K K , and enables tunable multi-objective trade-offs. An extensive ablation study confirms the necessity of each core component. Our work establishes A3M as a powerful and flexible framework for learning in complex auction environments. Figure 1: Motivation of this work. Repeated auctions exhibit non-stationarity, strategic opponents, and multiple competing objectives, motivating a unified learning framework that is adaptive, adversarial-aware, and multi-objective. 1 Introduction Uniform-price and discriminatory-price auctions are fundamental market mechanisms for allocating multiple identical items, widely used in domains such as electricity markets and treasury bill sales. Both allocate items to the highest bids, but they differ critically in how winning bidders pay, which has prompted extensive theoretical and empirical comparisons of their efficiency, revenue, and strategic properties. In repeated auctions, bidders can refine their strategies over time via online learning—a problem known as learning to bid —where performance is typically measured by regret against the best fixed bid in hindsight. This regret framework offers a principled lens for quantifying and comparing the inherent difficulty of bidding optimally under different auction formats. Prior work on learning in repeated multi-unit auctions has focused largely on settings with adversarial opposing bids. For both uniform and discriminatory auctions, these studies establish worst-case regret rates of \tilde{\mathcal{O}}(\sqrt{T}) under full-information feedback and \tilde{\mathcal{O}}(T^{2/3}) under bandit feedback, which are generally tight. Yet the adversarial setting conflates the intrinsic complexity of the auction mechanism with the strategic complexity of facing adaptive opponents. As a result, a systematic comparison of the two auction formats under a stochastic opponent model—which isolates the mechanism’s inherent learning difficulty—remains an open question. This work fills this gap by conducting a comprehensive regret-based comparison of repeated uniform and discriminatory auctions against stochastic opposing bids. Our core methodological contribution is the A3M (Adaptive, Adversarial & Multi-objective) framework, which departs from earlier estimate-then-commit approaches. Inspired by recent advances in reinforcement learning 46 ; 44 ; 64 ; 48 , A3M integrates deep reinforcement learning for adaptive strategy optimization, explicit opponent modeling for adversarial reasoning, and a principled multi-objective reward design. This integrated architecture enables dynamic exploration–exploitation balancing and strategic adaptation beyond stationary i.i.d. assumptions. Our analysis yields several key insights. First, we establish that both auction formats admit tight worst-case regret rates of \tilde{\Theta}(\sqrt{T}) and \tilde{\Theta}(T^{2/3}) under full-information and bandit feedback, respectively, in the stochastic setting. Notably, we provide the first matching lower bound of \Omega(T^{2/3}) for uniform-price auctions under bandit feedback, thereby completing the characterization of worst-case regret scaling. Second, beyond worst-case analysis, we identify and formalize families of instances where the two mechanisms exhibit a separation in achievable regret. For example, when opponents are symmetric and unit-demand, or when their bid distributions are well-separated, the uniform-price auction can achieve \tilde{\mathcal{O}}(\sqrt{T}) regret while the discriminatory auction remains at \Omega(T^{2/3}) . 13 ; 75 ; 15 ; 85 ; 88 ; 27 ; 14 Empirically, we show that the proposed A3M framework sets a new state-of-the-art adaptive baseline that significantly outperforms prior approaches 45 ; 63 ; 65 ; 33 ; 32 ; 34 . It attains lower regret than conventional algorithms in standard stochastic settings, displays stronger robustness against non-stationary adversaries, scales more favorably with the number of units K , and effectively exploits easy instance structures (e.g., \Delta -separated distributions). Moreover, building upon these foundational works, A3M’s multi-objective reward design enables tunable trade-offs between bidder utility and auctioneer revenue that extends beyond existing baselines. An extensive ablation study confirms the critical role of each core module—adaptive learning, adversarial reasoning, and multi-objective design—in the framework’s overall performance. 81 ; 84 ; 82 ; 83 ; 80 ; 38 ; 78 ; 86 The remainder of the paper is organized as follows. Section 2 reviews related work. Section 3 presents the A3M framework in detail. Section 4 analyzes learning with bandit feedback, including theoretical results and the A3M baseline. Section 5 extends the analysis beyond worst-case settings. Section 6 provides comprehensive ablation studies. Section 7 presents additional empirical evaluations. We conclude in Section 8 . 2 Related Work We review the literature on the simultaneous auction of multiple identical items, a standard model encompassing formats such as uniform-price, discriminatory-price, and Vickrey-Clarke-Groves (VCG) auctions 49 ; 16 ; 21 ; 74 ; 24 ; 89 . Empirical and theoretical comparisons of these mechanisms have primarily focused on their revenue generation 3 ; 9 ; 42 ; 11 ; 12 ; 70 and social welfare performance 30 ; 36 ; 71 ; 69 ; 79 , with significant attention given to symmetric and unit-demand settings 35 ; 68 ; 51 ; 5 . The study of repeated auctions has increasingly utilized online learning tools to analyze dynamic strategic behavior 60 ; 6 ; 61 ; 39 ; 53 . Early work in this line focused on the auctioneer’s problem of learning optimal reserve prices 28 ; 43 ; 52 ; 55 . Subsequent research shifted to the bidder’s perspective, introducing and analyzing the problem of learning to bid in various repeated single-item auction formats 23 ; 19 ; 2 ; 73 ; 40 ; 56 . Building upon these foundations, recent work has extended online learning to federated and privacy-preserving settings 67 ; 66 ; 50 ; 87 ; 41 . The problem of learning to bid in repeated multi-unit auctions is a more recent development. Initial results were provided for uniform-price auctions by 20 ; 77 ; 8 . For discriminatory auctions, 4 ; 72 ; 22 established regret rates of \tilde{\mathcal{O}}(K\sqrt{T}) and \tilde{\mathcal{O}}(KT^{2/3}) under full-information and bandit feedback, respectively. Extending these baselines with modern deep learning techniques, recent advances in transformer architectures and learning technologies 47 ; 65 ; 62 ; 76 have opened new avenues for sequence modeling in strategic settings that achieve better sample complexity and adaptive performance. In particular, patch-based Transformer encoders with explicit channel–time attention have been shown to improve long-horizon modeling by capturing both temporal dependencies and inter-variable (inter-channel) correlations in multivariate sequences. In the full-information setting for uniform-price auctions, recent work 7 ; 59 ; 58 derived a \tilde{\mathcal{O}}(K^{3/2}\sqrt{T}) regret bound, alongside sub-optimal bandit results. For the bandit setting under a Last Accepted Bid (LAB) pricing rule, 1 ; 57 ; 17 proposed an algorithm achieving \tilde{\mathcal{O}}(K^{4/3}T^{2/3}) regret and