The Unreasonable Effectiveness of Scaling Laws in AI Logical Compute, Hidden Efficiency, and the Burden of Diminishing Returns

Chien-Ping Lu · 2026

Read The Unreasonable Effectiveness of Scaling Laws in AI Logical Compute, Hidden Efficiency, and the Burden of Diminishing Returns by Chien-Ping Lu (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-28507.txt.

Introduction

Lu argues that AI progress can continue only if efficiency keeps doubling on a steady beat. Scaling laws track logical compute, not any fixed chip or code base. As returns shrink, each new unit of compute costs more in real resources. Hardware, algorithm, and systems gains must therefore repeat, much as Moore's law once organized chip progress. Recent trends look at least Moore-like, he notes. Without such doublings, scaling stalls under its own weight.

Catalog record

CREATORChien-Ping Lu
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE19401
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.28507

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Unreasonable Effectiveness of Scaling Laws in AI Logical Compute, Hidden Efficiency, and the Burden of Diminishing Returns Chien-Ping Lu Abstract Classical AI scaling laws are effective in the ordinary sense that they make progress predictable despite diminishing returns. Their more surprising effectiveness is that they remain portable across changing regimes while AI progress stays empirically active and economically meaningful despite those same diminishing returns. This paper argues that both facts have the same source. The compute variable in the classical law is best read as logical compute , an abstraction over model-side work that suppresses many realization details. Practical burden therefore depends not only on the loss exponent, but on the efficiency with which physical resources realize that compute. Under that interpretation, diminishing returns mean rising operational burden, not merely a flatter curve. Once efficiency is made explicit and allowed to compound over time, continued progress depends on repeated efficiency doublings in hardware, algorithms, and systems. Scaling laws travel because they abstract; progress continues because the efficiency stack keeps changing what level of logical compute is feasible. 1 Introduction Classical AI scaling laws, especially for pre-training, are unusually compact. Under compute-optimal conditions, training loss is often well approximated by a power law in compute [ 3 , 2 ] : L\propto C^{-\kappa}, where L denotes loss, C denotes compute, and \kappa is the scaling exponent. This is the ordinary effectiveness of scaling laws: they make progress forecastable. Returns decline, but they do not become arbitrary. But prediction is only the first fact. Scaling laws are also unreasonably effective in a stronger, Wigner-like sense [ 10 ] . They remain surprisingly portable across changing architectures, precisions, sparsity patterns, and training-adjacent regimes, and AI progress has continued despite the diminishing returns they encode. The deeper question is therefore not only why the law predicts, but why it keeps traveling and why movement along it keeps happening in practice. The thesis of this paper is that both facts have the same source. Scaling laws are unreasonably effective because they abstract over realization details. The compute variable is best read not as a record of a specific implementation, but as logical compute : an abstract measure of model-side work. Once that distinction is made, the practical burden of scaling depends on a separate efficiency term that governs how physical resources are converted into logical compute. The same abstraction that makes the law portable also creates a persistent efficiency game in hardware, algorithms, and systems, because it leaves many ways to realize additional logical compute more cheaply. This framing keeps the paper focused on one interpretive claim and one formal extension. The interpretive claim is that C in the classical law should be read as logical compute. The formal extension is that once efficiency compounds over time, progress depends not only on the static exponent \kappa , but on the rate of efficiency doubling. Under that view, diminishing returns are not merely a flatter loss curve. They are rising operational burden. Recent work extends classical scaling analyses to inference-aware training, precision, and sparse routing [ 9 , 5 , 6 , 7 ] . Those refinements matter, but they are not the paper’s main target. Here they serve as evidence that the empirical compute–loss relation remains useful across changing realization choices. The goal is not another fitted variant of the law. It is to clarify what the classical variable means and what the classical formulation leaves out when used as a guide to real progress. The paper therefore proceeds in three main moves. First, it restates the classical law and identifies what it omits. Second, it separates logical compute from physical burden by making hidden efficiency explicit. Third, it introduces a time-indexed efficiency-doubling extension that connects static diminishing returns to dynamic, continuing progress. The later sections then draw out the operational and architectural implications of that extension. 2 Classical Scaling Law and What It Omits The standard starting point in the literature is a separable loss model over model size and dataset size: L(N,D)\;=\;E+AN^{-\alpha}+BD^{-\beta}, where N is the number of parameters, D is the number of training tokens, E is an irreducible loss floor, and \alpha,\beta>0 are empirical exponents. This form captures a familiar idea: increasing either model size or data improves loss, but with diminishing returns. To connect this expression to compute, one imposes a training-cost constraint. In dense pre-training, total compute is commonly approximated as C\propto ND, or, in a standard back-of-the-envelope form, C\approx 6ND, where the constant reflects forward pass, backward pass, and parameter-update costs. Under a fixed compute budget, the compute-optimal choice of N and D balances the two loss terms, yielding N^{*}\propto C^{\tfrac{\beta}{\alpha+\beta}},\qquad D^{*}\propto C^{\tfrac{\alpha}{\alpha+\beta}}. Substituting these relations back into the loss gives the familiar compute-only law L(C)\approx E+KC^{-\kappa},\qquad\kappa=\frac{\alpha\beta}{\alpha+\beta}, for some constant K>0 . This derivation helps clarify both the power and the limits of the classical law. It captures a robust empirical relationship between loss and compute, but it does so at a highly abstract level. In particular, \kappa is a property of how loss responds to scaling in parameters and data under a compute budget. It is not itself a measure of hardware quality, precision format, or system efficiency. That is why the classical law is best understood as a ratio law. It does not claim that a particular absolute compute budget guarantees a universal level of capability. Rather, it specifies how loss changes relative to changes in compute. This is what gives the law its immediate predictive value, and it is also part of what makes scaling laws portable across changing training stacks: they preserve a higher-level relation while abstracting away from many of the mechanisms that realize compute in practice. Recent extensions that incorporate inference demand, numerical precision, or Mixture-of-Experts structure [ 9 , 5 , 6 , 7 ] do not overturn this basic picture. They show that one can refine the law by exposing variables that the baseline formulation leaves implicit. For the present paper, however, the important point is simpler: the empirical compute–loss relation continues to survive across changing realization choices. That is evidence for the robustness of the abstraction, not a reason to replace the present paper’s central question with a survey of every variant. Accordingly, the contribution here is not a new fitted law for a particular architecture or regime. It is interpretive and unifying. The paper proposes a stable reading of the compute variable in classical scaling laws, separates that variable from the efficiency with which it is realized, and then uses that separation to introduce a simple dynamic extension indexed by efficiency doublings over time. 3 Logical Compute and Hidden Efficiency The central omission in the compact law L(C)\approx E+KC^{-\kappa} is not compute itself, but the realization of compute. Classical scaling laws treat compute as an input, yet they do not ask how long it takes to supply that compute, how much power it requires, or how much system engineering is needed to sustain it. In that sense, efficiency is hidden rather than absent. This motivates a distinction between two quantities: • Logical compute : the abstract model-side work that enters the scaling law, defined against a dense, uniform-precision reference realization of the model rather than against any

Read the complete text