When AI Trading Agents Compete: Adverse Selection of Meta-Orders by Reinforcement Learning-Based Market Making
Ali Raza Jafree, Konark Jain, Nick Firoozye · 2025
Read When AI Trading Agents Compete: Adverse Selection of Meta-Orders by Reinforcement Learning-Based Market Making by Ali Raza Jafree, Konark Jain, Nick Firoozye (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-27334.txt.
Introduction
Ali Raza Jafree, Konark Jain, and Nick Firoozye study how high-frequency market makers learn to profit from slower traders large orders. Medium-frequency traders executing a meta-order, meaning one big order split over time, suffer adverse selection as opportunistic fast traders exploit the price drift the order creates. The team models the limit order book with a Hawkes process, which captures self-exciting order flow and endogenous price impact.
A reinforcement learning market maker, trained with policy optimization and self-imitation under an impulse control framing, learns to capitalize on meta-order drift when trained against such a trader. Empirical motivation notes slippage costs for medium-frequency traders likely rise as fast trading proliferates. Interestingly, higher market maker profits do not necessarily cause significantly larger slippage for the slower trader in their simulations.
Catalog record
| CREATOR | Ali Raza Jafree, Konark Jain, Nick Firoozye |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-27334
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
When AI Trading Agents Compete: Adverse Selection of Meta-Orders by Reinforcement Learning-Based Market Making Ali Raza Jafree, Konark Jain, Nick Firoozye Abstract We investigate the mechanisms by which medium-frequency trading agents are adversely selected by opportunistic high-frequency traders. We use reinforcement learning (RL) within a Hawkes Limit Order Book (LOB) model in order to replicate the behaviours of high-frequency market makers. In contrast to the classical models with exogenous price impact assumptions, the Hawkes model accounts for endogenous price impact and other key properties of the market ( Jain et al. 2024a ) . Given the real-world impracticalities of the market maker updating strategies for every event in the LOB, we formulate the high-frequency market making agent via an impulse control reinforcement learning framework ( Jain et al. 2025 ) . The RL used in the simulation utilises Proximal Policy Optimisation (PPO) and self-imitation learning. To replicate the adverse selection phenomenon, we test the RL agent trading against a medium frequency trader (MFT) executing a meta-order and demonstrate that, with training against the MFT meta-order execution agent, the RL market making agent learns to capitalise on the price drift induced by the meta-order. Recent empirical studies have shown that medium-frequency traders are increasingly subject to adverse selection by high-frequency trading agents. As high-frequency trading continues to proliferate across financial markets, the slippage costs incurred by medium-frequency traders are likely to increase over time. However, we do not observe that increased profits for the market making RL agent necessarily cause significantly increased slippages for the MFT agent. Code — https://github.com/konqr/lobSimulations/tree/master/HawkesRLTrading Introduction In this study, we use the LOB simulation by Jain and Firoozye ( Jain et al. 2024a ; Jain et al. 2024b ; Jain et al. 2024c ) to construct a simulated trading environment in which a medium-frequency trader faces strategic adversarial selection from HFTs, replicating the empirically observed phenomenon of medium-frequency traders (MFTs) being adversely selected by high-frequency traders (HFTs) ( Cartea, Chang, and Graumans 2025 ) . The LOB is simulated using a compound Hawkes process, which differs from traditional models which are generally based on Brownian motion; most significantly, it accounts for the market impact of endogenous factors ( Jain et al. 2025 ) . That is, the simulation responds to orders and trades made by the trading agents, making it suitable for our goal to effectively replicate the behaviour of HFTs adversely selecting MFTs. Our methodology encompasses three phases. Firstly, we develop trading agents within the LOB to simulate MFT meta-order flows and a HFT market maker. Additionally, testing is required to establish a baseline for the MFT’s slippages and the HFT’s performance. For the MFT agent, we chose to create an agent that executes a meta-order using the time-weighted average price (TWAP) execution method. While TWAP is not the most sophisticated meta-order execution method, it is still actively used in less developed markets and as a part of quantitative trading strategies ( Velu, Hardy, and Nehren 2020 ) . To create a trading agent that accurately simulates a HFT market maker, we utilise the RL trading agent framework previously developed ( Jain et al. 2025 ) . The RL framework approximates the solution to the high dimensional Hamilton-Jacobian-Bellman Quasi-Variational Inequality (HJB-QVI), brought about by the fact that a market maker will only respond to a subset of all LOB events due to real-world technological limiting factors and practical constraints. The RL model’s enabling of the agent to have a higher degree of decision making when it comes to when orders are placed further enhances its accuracy, modelling the precise and purposeful intervention decisions made by real-world market makers. Secondly, to test the agents when trading together and investigate whether the ’unaware’ RL (uRL) trading agent could, by default, capitalise on the TWAP meta-order execution without additional training. Finally, to train an RL model with the meta-order execution agent, giving it full information as to whether the TWAP is present or not and whether it is executing a buy meta-order or a sell meta-order. The eventual goal of this work is to develop defensive mechanisms for medium-frequency traders and their meta-order executions against adverse selection by the RL trained and enabled HFT. Background For readers unfamiliar with high-frequency trading and market microstructure, we briefly introduce the key concepts underlying this work. Limit Order Book (LOB) A limit order book is the core mechanism of modern electronic markets. It contains two lists of pending orders: buy orders (bids) and sell orders (asks). The highest bid and lowest ask define the best bid and ask, and their difference is the bid-ask spread. Traders interact with the LOB through two main order types. A limit order (LO) is passive—it rests in the book at a chosen price until matched. A market order (MO) is aggressive—it executes immediately against the best available prices. The flow of order arrivals, cancellations, and executions determines both price movements and liquidity. Market Making Market makers continuously post buy and sell limit orders, earning profits by capturing the bid-ask spread—buying at the bid and selling at the ask. They face several risks: inventory risk (holding large unbalanced positions), adverse selection risk (trading against informed agents), and execution risk (orders not being filled). High-frequency market makers operate on millisecond timescales, constantly adjusting quotes in response to market changes using automated algorithms and low-latency infrastructure. Meta-orders and Execution Strategies Large institutional investors must often trade amounts far exceeding available liquidity. Executing such “meta-orders” at once would cause strong price impact—buying pushes prices up, selling pushes them down. To limit impact, these orders are split into smaller “child” orders executed over time via algorithms such as: • TWAP: Time Weighted Average Price execution - executes equal-sized slices at regular intervals. • VWAP: Volume Weighted Average Price execution - follows historical volume patterns. • POV: Percentage of Volume based execution - maintains a constant share of total market volume. While these methods reduce costs, their predictable patterns can be detected and exploited by faster traders. Adverse Selection Adverse selection arises when an agent trades with better-informed agents, leading to losses. During meta-order execution, high-frequency traders may detect and anticipate the large order’s direction, profiting from the predictable price drift. For the slower trader, this results in slippage—the difference between the expected and actual execution price. As markets become faster and more competitive, such adverse selection effects intensify. Hawkes Processes Simple Poisson or Brownian models fail to capture the clustering and feedback seen in real order flows. Hawkes processes, by contrast, are self-exciting point processes where each event increases the likelihood of future ones. In the LOB context, this means market and limit order arrivals trigger further activity, producing realistic volatility clustering and feedback. Our compound Hawkes process model captures key empirical features: • Endogenous price impact: Price changes emerge from order flow dynamics. • Self-excitation: Trading activity breeds more trading activity. • Mean reversion: Temporary impact decays over time. This modelling framework allows us to study how strategic trading actions and endogenous feedback jointly shape market dynamics. Research has also shown that foundational models can be used with retrieval augmented generation, as shown in ( Zhu et al. 2025 ) . Related Work Our work primarily relates to the