An Impulse Control Approach to Market Making in a Hawkes LOB Market
Konark Jain, Nick Firoozye, Jonathan Kochems, Philip Treleaven · 2025
Read An Impulse Control Approach to Market Making in a Hawkes LOB Market by Konark Jain, Nick Firoozye, Jonathan Kochems, Philip Treleaven (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-26438.txt.
Introduction
Konark Jain, Nick Firoozye, Jonathan Kochems, and Philip Treleaven study market making in a 2025 paper. A market maker posts buy and sell prices and earns from the gap between them. Their market is a limit order book, a list of waiting buy and sell orders.
Prices move in a Hawkes process, a model where past trades make new trades more likely. The maker acts at set times, not at each event, by placing, ending, or filling orders. Two learning methods are tested, and one earns strong risk-adjusted gains in tests. A hard math form is also solved for comparison. The work links control theory with deep learning for fast, jumpy markets.
Catalog record
| CREATOR | Konark Jain, Nick Firoozye, Jonathan Kochems, Philip Treleaven |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 67 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-26438
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
An Impulse Control Approach to Market Making in a Hawkes LOB Market Konark Jain, Nick Firoozye, Jonathan Kochems, Philip Treleaven Abstract We study the optimal Market Making problem in a Limit Order Book (LOB) market simulated using a high-fidelity, mutually exciting Hawkes process. Departing from traditional Brownian-driven mid-price models, our setup captures key microstructural properties such as queue dynamics, inter-arrival clustering, and endogenous price impact. Recognizing the realistic constraint that market makers cannot update strategies at every LOB event, we formulate the control problem within an impulse control framework, where interventions occur discretely via limit, cancel, or market orders. This leads to a high-dimensional, non-local Hamilton-Jacobi-Bellman Quasi-Variational Inequality (HJB-QVI), whose solution is analytically intractable and computationally expensive due to the curse of dimensionality. To address this, we propose a novel Reinforcement Learning (RL) approximation inspired by auxiliary control formulations. Using a two-network PPO-based architecture with self-imitation learning, we demonstrate strong empirical performance with limited training, achieving Sharpe ratios above 30 in a realistic simulated LOB. In addition to that, we solve the HJB-QVI using a deep learning method inspired by Sirignano and Spiliopoulos 2018 [ 40 ] and compare the performance with the RL agent. Our findings highlight the promise of combining impulse control theory with modern deep RL to tackle optimal execution problems in jump-driven microstructural markets. keywords Market Making; Limit Order Book; Hawkes Process; Impulse Control; Reinforcement Learning; Hamilton–Jacobi–Bellman Equation; Quasi-Variational Inequality; Deep Learning; Financial Microstructure Funding. This work was funded by JPMorganChase † † runningheads: An Impulse Control Approach to Market Making in a Hawkes LOB Market / K. Jain, N. Firoozye, J.Kochems, and P. Treleaven AMS Primary: 93E20; Secondary: 91G80, 60G55, 35Q93, 68T07, 49L25 1 Introduction Market Making in Limit Order Books (LOBs) is a high frequency trading task, where liquidity is provided with the goal of capturing the bid-ask spread. While many classical formulations rely on continuous-time dynamics and control, the true microstructure of the LOB is inherently discrete and driven by a pure jump process. The distinction between liquidity provision and liquidity consumption represents a fundamental dichotomy in modern market microstructure theory. As noted in [ 24 ] , while optimal execution strategies primarily focus on liquidity-taking behavior—the efficient liquidation of large positions or implementation of hedging strategies—market making fundamentally concerns itself with the provision of liquidity to the marketplace. This shift in perspective necessitates a fundamentally different analytical framework, one that accounts for the unique risks and opportunities inherent in standing ready to transact at posted prices. Market makers, defined as traders who continuously provide liquidity by quoting bid and ask prices, serve as the backbone of price discovery mechanisms across diverse market structures ( [ 24 ] ). The heterogeneity of market making arrangements reflects the varied institutional frameworks within which these participants operate. On order-driven markets, official market makers such as Designated Market Makers (DMMs) on the NYSE operate under explicit contractual obligations to maintain fair and orderly markets, including mandatory participation in opening and closing auctions and adherence to National Best Bid and Offer (NBBO) quoting requirements. Conversely, proprietary market makers, including high-frequency trading firms, provide liquidity without formal obligations, seeking to profit from the bid-ask spread while maintaining flexibility in their market participation. The fundamental economic proposition underlying market making involves the temporal arbitrage of buying at the bid price and selling at the ask price, with the bid-ask spread representing the market maker’s expected compensation for providing immediacy services. However, this seemingly straightforward profit mechanism is complicated by the inevitable inventory risk that market makers must bear. The asynchronous nature of buy and sell transactions means that market makers typically hold non-zero inventory positions, exposing them to adverse price movements during the holding period ( [ 24 ] ). Optimal Market Making (MM) in Limit Order Book (LOB) settings has been approached primarily through two methodological lenses: stochastic control and reinforcement learning. Early stochastic control models often assume Markovian mid-price dynamics with exogenously specified order flow. The seminal work of [ 5 ] established the theoretical foundation for modern market making models by formulating the problem as a continuous-time stochastic control problem. Their framework, along with the closed-form approximations developed by [ 20 ] , provides elegant analytical solutions for optimal bid and ask quote placement. However, these continuous-time models face significant limitations when applied to actual order-driven markets, particularly in their treatment of price discreteness arising from tick size constraints and their inability to capture the granular dynamics of limit order book evolution. For instance, [ 23 ] model the mid-price as a continuous-time Markov chain and the bid-ask spread as a discrete Markov process, allowing the MM to place limit orders, market orders, and even aggressive internalizing strategies. These controls are optimized through a hybrid control framework involving a Hamilton-Jacobi-Bellman (HJB) equation. More recent efforts have focused on incorporating more realistic LOB dynamics. [ 38 ] consider a Hawkes-like LOB, where market orders impact a latent alpha signal and fill probabilities, making them informative. The MM, restricted to placing only limit orders, solves a stochastic control problem in this non-Markovian environment. Similarly, [ 30 ] study a quote-driven market in which market takers’ actions are modeled via Hawkes processes. They derive HJB solutions in the exponential kernel case and extend to more realistic power-law kernels via approximations. A more discrete approach is taken by [ 1 ] , who model LOB dynamics using a state-dependent Poisson process and then extend the model to an exponential Hawkes process, solving the corresponding Markov Decision Process (MDP). While most of these works rely on analytically tractable models, they tend to assume known dynamics, stationarity, and full observability, which limit their applicability to real-world markets. Reinforcement Learning (RL) has emerged as a compelling alternative, capable of handling partial observability, path-dependence, and model uncertainty. A recent survey by [ 21 ] challenges classical assumptions made in stochastic control—such as complete knowledge of the environment and fixed time horizons—and organizes RL approaches by control logic (inventory-based, signal-driven, robust) and state observability (tabular vs. deep RL). A second taxonomy classifies methods by algorithmic design, highlighting the shift from analytical and tabular methods to deep actor-critic variants. Most RL frameworks define the agent’s state using features like inventory, PnL, spread, imbalance, and short-term technical signals such as RSI. To tackle sample inefficiency, several works employ simulators or historical replay mechanisms. For instance, [ 22 ] use an 8-dimensional Hawkes-based simulator in which the MM interacts via limit and market orders in a uni-episodic loop; their Soft Actor-Critic agent outperforms a stochastic control baseline, while DQN and TD3 fail to converge. Other approaches, like those of [ 41 ] , rely on historical replay with simpler discrete actions and no explicit modeling of market impact. Recent innovations include adversarial training and auxiliary signal units to improve policy robustness and generalization. Despite RL’s flexibility, fundamental challenges remain— [ 24 ] highlights issues such as non-stationarity, the absence of feedback loops in historical data, and the inherent difficulty in designing risk-sensitive reward structures. These limitations continue to motivate hybrid models that blend the tractability of control theory with the adaptability of learning-based methods. The discrete nature of price movements in modern electronic markets, coupled with the complex