Test-Time Compute Games

Ander Artola Velasco §, Dimitrios Rontogiannis §, Stratis Tsirtsis †, Manuel Gomez-Rodriguez § · 2026

Read Test-Time Compute Games by Ander Artola Velasco §, Dimitrios Rontogiannis §, Stratis Tsirtsis †, Manuel Gomez-Rodriguez § (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-21839.txt.

Introduction

Ander Artola Velasco and co-authors study a new market dynamic created by reasoning models. Giving a language model more compute at test time improves its reasoning. But cloud providers charge users for that compute, so better answers now come with metered bills.

That pricing turns reasoning into a game among providers and users, each with their own incentives. The team's 2026 preprint frames these interactions as test-time compute games and analyzes them. The framing matters because the economics will shape how much reasoning users actually buy, how providers set prices. And whether careful thinking becomes a luxury feature. Game theory offers tools to predict where such a market settles before it fully forms.

Catalog record

CREATORAnder Artola Velasco §, Dimitrios Rontogiannis §, Stratis Tsirtsis †, Manuel Gomez-Rodriguez §
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE124 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-21839

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Test-Time Compute Games Ander Artola Velasco §, Dimitrios Rontogiannis §, Stratis Tsirtsis †, Manuel Gomez-Rodriguez § Abstract Test-time compute has emerged as a promising strategy to enhance the reasoning abilities of large language models (LLMs). However, this strategy has in turn increased how much users pay cloud-based providers offering LLM-as-a-service, since providers charge users for the amount of test-time compute they use to generate an output. In our work, we show that the market of LLM-as-a-service is socially inefficient—providers have a financial incentive to increase the amount of test-time compute, even if this increase contributes little to the quality of the outputs. To address this inefficiency, we introduce a reverse second-price auction mechanism where providers bid their offered price and (expected) quality for the opportunity to serve a user, and users pay proportionally to the marginal value generated by the winning provider relative to the second-highest bidder. To illustrate and complement our theoretical results, we conduct experiments with multiple instruct models from the Llama and Qwen families, as well as reasoning models distilled from DeepSeek-R1 , on math and science benchmark datasets. 1 Introduction The massive computational resources required to run large language models (LLMs) have led many users to rely on a growing market of cloud-based service providers that offer LLM-as-a-service. A key driver of this computational demand is the use of test-time compute (TTC)—additional computations performed by an LLM at inference time to improve its performance using techniques such as chain-of-thought ( 66 ) , tree-of-thought ( 70 ) , and best-of-n sampling ( 13 ) . Importantly, in the LLM-as-a-service market, the cost of these additional computations ultimately falls on the users, 1 1 1 https://ai.google.dev/gemini-api/docs/pricing , https://openai.com/api/pricing/ , https://www.claude.com/pricing . who pay for all tokens generated by a model during inference, including intermediate tokens that are not visible in the final outputs observed by the users. In this context, providers have the flexibility to (dynamically) adjust the amount of test-time compute an LLM uses to respond to a user’s query. 2 2 2 For example, GPT-5 uses real-time routing to adjust reasoning depth, and thus test-time compute. See https://openai.com/index/introducing-gpt-5/ . However, in a competitive market, this flexibility creates a new strategic dimension beyond how providers price their services. In particular, providers can strategically increase the amount of test-time compute allocated to a user’s query to maximize profit, even if the additional test-time compute contributes little to the quality of the response. Consider a simple illustrative example where, for a given set of queries with verifiable ground truth ( e.g. , diagnosing patients based on their medical records), two different providers can run their LLMs in either a low-TTC mode ( e.g. , standard generation) or a high-TTC mode ( e.g. , chain-of-thought), with average accuracies and generation costs for the providers given by: Provider 1 Avg. accuracy Avg. gen. cost Low TTC 70% $0.25 High TTC 90% $1 Provider 2 Avg. accuracy Avg. gen. cost Low TTC 50% $0.5 High TTC 95% $10 If both providers price their models with a 25\% profit margin over their generation costs, it is easy to see that a user who values each percentage point of accuracy as \$0.02 would always select the first provider, who offers them higher value ( i.e. , \$0.02\times\text{accuracy}-\text{price} ) independently of the TTC mode chosen by the second provider. However, to increase their profit, the first provider is financially incentivized to choose the high-TTC mode, even though the low-TTC mode would maximize the sum of the user value and provider profit, and would therefore be socially optimal. 3 3 3 We formally introduce user value and provider utilities in Section 2 . For the exact derivations related to this example, refer to Appendix A . In our work, we formally characterize the above gap and show that, when providers act to maximize profit in a competitive market, they cannot, in general, be expected to use the socially optimal amount of test-time compute. Further, we conceptualize and analyze a forward-looking market where users and providers participate in a reverse second-price auction, and show that, in contrast to the current pay-for-compute market, this auction mechanism is guaranteed to be socially optimal. Our contributions. We start by modeling the current LLM-as-a-service market as a normal-form game ( 43 ) between N providers who choose their test-time compute allocations to maximize profit, and users pay according to the price per compute set by their selected provider. Building on this characterization, we make the following contributions: 1. We show that the LLM-as-a-service market admits a pure Nash equilibrium and that, under natural assumptions, the market providers are guaranteed to reach this equilibrium in finite time. 2. We characterize the test-time compute used by providers in an LLM-as-a-service market at the Nash equilibrium and show that, in general, it is not socially optimal. 3. We introduce a reverse second-price auction mechanism where providers bid with their offered price and (expected) quality for the opportunity to serve a user, and the user’s price is determined by the marginal value generated by the winning provider relative to the second-highest bidder. 4. We show that the above auction mechanism is socially optimal, and it guarantees that users always obtain at least their second-best achievable value for the task and providers secure non-negative profits. To complement our theoretical results, we conduct experiments with multiple instruct models from the Llama and Qwen families, as well as reasoning models distilled from DeepSeek-R1 , on math and science benchmark datasets. Our results show that the existing pay-for-compute market is up to 19\% (socially) inefficient, as measured by the Price of Anarchy. 4 4 4 The code for our experiments is publicly available at https://github.com/Networks-Learning/strategic-ttc . Further related work. Our work connects to the rapidly growing literature on the economic and strategic aspects of machine learning systems ( 26 ; 59 ; 17 ; 53 ; 48 ; 19 ) , and more specifically, LLMs-as-a-service ( 16 ; 30 ; 34 ; 31 ; 8 ; 52 ; 5 ; 61 ; 60 ; 23 ; 10 ; 55 ) . Therein, most closely related to ours is a strand of recent works that analyzes the incentives of providers of LLM-as-a-service when they act strategically. Specifically, 61 ; 60 study how providers can overcharge users by misreporting the number of tokens used by the LLM to encode a given text, and introduce an auditing method to detect such behavior. In a similar vein, 55 and 10 consider a setting where a provider injects additional reasoning tokens to inflate the user’s payment, while 52 and 8 study a scenario in which providers have a financial incentive to be unfaithful to users by deploying cheaper-to-run models to maximize profit. Further, our work relates to a line of work that analyzes the capabilities enabled by test-time compute ( 67 ; 68 ; 7 ; 33 ; 54 ) . While most of the literature has focused primarily on performance aspects, there is increasing interest in analyzing the economic implications of test-time compute. In particular, recent works by 64 , 71 and 18 argue for incorporating the substantial costs of test-time compute into LLM evaluation and ranking, while 63 , 27 and

Read the complete text