An Information-Theoretic Criterion for Efficient Data Synthesis

Hanyu Li, Zhengqi Sun, Xiaotie Deng · 2026

Read An Information-Theoretic Criterion for Efficient Data Synthesis by Hanyu Li, Zhengqi Sun, Xiaotie Deng (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-16379.txt.

Introduction

Li, Sun, and Deng argue learning latches onto the most information-efficient signal, speeding training when intended and driving reward hacking when a simpler spurious pattern wins. Synthetic data help only when the generation-training loop stays open to outside signals. Verifiers or rubrics add task-relevant information the model lacks. Shut the loop to its outputs, and the data processing inequality says task-relevant information can only shrink, so collapse follows. Among open loops, binary correctness treats acceptable outputs alike, so its lesson travels across tasks and domains.

Catalog record

CREATORHanyu Li, Zhengqi Sun, Xiaotie Deng
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45700
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.16379

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

An Information-Theoretic Criterion for Efficient Data Synthesis Hanyu Li, Zhengqi Sun, Xiaotie Deng Abstract Synthetic data becomes crucial for large language model training, but its effectiveness is highly inconsistent. We provide an information-theoretic account of this inconsistency: synthetic data improves a model only when the generation-training loop is information-open, i.e., shaped by external signals (verifiers, environments, or rubrics) that inject task-relevant information beyond the model’s current distribution. When the loop is information-closed (relying on the model’s own outputs without such signals), the data processing inequality ensures that task-relevant information can only decrease, making collapse a predicted outcome. Among information-open pipelines, both efficiency and generalization hinge on the meta-level of supervision: a coarser signal such as binary correctness treats all acceptable outputs as equivalent, so the behavior it teaches is not tied to any particular domain or surface form and generalizes naturally across tasks and domains. These observations lead to a guiding thesis: learning preferentially converges to the most information-efficient signal component available, which accelerates learning when that component is the intended one, but causes reward hacking when a spurious pattern happens to be simpler. 1 Introduction Scaling laws have driven steady progress in large language models (LLMs), with performance improving predictably as model size, compute, and training data grow ( 17 ) . This paradigm, however, rests on a premise that is becoming increasingly fragile: the continued availability of abundant, high-quality real-world text. Recent analyses suggest that such data may be approaching practical limits ( 34 ) . At the same time, as LLMs are pushed toward long-horizon reasoning, multi-step planning, and agentic interaction, the most effective forms of supervision increasingly rely on explicit constraints, feedback, and verification—going well beyond what generic text corpora can provide. These two trends—data scarcity and the demand for richer supervision—have made synthetic data a central ingredient in modern training pipelines. Yet synthetic data remains a double-edged sword. On one hand, generated supervision has produced substantial gains: verifier-guided synthesis reaches olympiad-level geometry ( 32 ) , and reinforcement learning from verifiable rewards has driven major reasoning breakthroughs ( 14 ; 37 ) . On the other hand, iterative self-training—where models are trained on their own outputs—often leads to distribution collapse and progressive capability degradation ( 28 ; 2 ; 12 ) . Despite numerous empirical mitigation techniques, a principled account of when and why synthetic data helps or hurts is still lacking. This motivates a concrete question: under what conditions does synthetic data yield sustained improvements, and when does it induce degradation? We approach this question through an information-theoretic lens, anchored in the data processing inequality (DPI). The DPI states a simple but powerful fact: no computation on observed data can create new information about the underlying source. Applied to self-training, the implication is immediate. When a model trains on samples drawn from its own distribution and no additional task-relevant signal enters the loop, the pipeline is information-closed : each iteration can only preserve or lose information about the true task. In practice, finite sampling and optimization errors make the loss strict, so the model’s knowledge of the task degrades over iterations. Model collapse is thus not an anomaly—it is the predicted outcome of a closed information loop. This DPI argument, however, is too pessimistic on its own: it cannot explain why synthetic data so often works spectacularly well. The resolution is that successful pipelines are not actually closed systems. They incorporate external signals —verifiers, rubrics, environments, or fixed teacher models—that carry task-relevant information independent of the current model state. Once we account for these signals, the DPI argument sharpens rather than breaks: the pessimistic monotonicity holds only when such signals are absent. When an external signal is present, the relevant information bound includes not just the training data but also what the signal contributes beyond it. The question then shifts from “can synthetic data add information?” to “how much task-relevant information does the external signal carry beyond what the model already knows?” Sustained improvement is possible precisely when this quantity remains positive—that is, when the loop stays information-open . Given that a pipeline is information-open, a natural follow-up is: how efficiently does it inject information? We argue that a key determinant is what we call the meta-level of the supervision signal—the granularity at which it distinguishes outputs. A low meta-level signal specifies a particular target (e.g., a single reference answer); a high meta-level signal specifies only a coarse property (e.g., “is the answer correct?”) and treats all outputs satisfying that property as interchangeable. Because a high meta-level signal collapses many surface-level differences into a single judgment, each unit of supervision constrains a much larger region of the output space, making learning markedly more sample-efficient. A natural implication, and a guiding thesis of this paper, is that learning preferentially converges to the most information-efficient signal component available in the training data. This thesis cuts both ways: when the intended signal is the most efficient one, it explains why high meta-level supervision produces rapid learning and strong cross-domain generalization; when a spurious signal happens to operate at a coarser granularity than the intended one, the same dynamic produces reward hacking. Conflict of Interest Disclosure. The authors declare no financial conflicts of interest related to the work presented in this paper. 2 Case Studies on Synthetic Data We begin by defining precisely what we mean by synthetic data and identifying three recurring empirical patterns that any theoretical account must explain. We use synthetic in a mechanism-based sense: supervision is synthetic if it is produced by an automated pipeline (often an LLM generator plus tools) and used for training. Under this definition, “synthetic” does not mean “fake”: it only indicates that the supervision originates from an automated procedure rather than direct human annotation. In much of the literature, “synthetic data” is used roughly synonymously with model-generated text . Our definition is broader in what counts as supervision (not only instruction–response pairs but also preferences, rewards, verifier feedback, and retained trajectories). Concretely, even when prompts (and sometimes final answers) come from real datasets, reinforcement learning with verifiable rewards (RLVR) can still generate intermediate trajectories that are retained only if they pass verifiable checks; those retained trajectories function as synthetic supervision in our sense. With this terminology, we summarize three recurring patterns below: closed-loop self-training (failure), hard verifiers (success I), and soft but stable references (success II). Failure (closed-loop self-training). When training primarily recycles the model’s own generations without a persistent external signal, the loop tends to contract the output distribution: tail patterns are under-sampled and errors become self-reinforcing ( 28 ; 2 ) . The effect is strongest when synthetic data progressively replaces real data, whereas mixing real data each iteration can mitigate collapse ( 12 ) . Synthetic-heavy regimes can also distort scaling behavior by weakening long-tail modeling ( 10 ) . Success I (hard, verifiable signals). A high-reliability path is generate-and-verify : candidates are filtered, ranked, or rewarded by an external procedure (proof checker, compiler/tests, benchmark harness), via rejection sampling, best-of- N , or RLVR ( 14 ; 11 ) . The headline outcome is qualitative performance jumps anchored by checkable signals : in math, verifier-guided search and distillation can reach medal-level performance (e.g., IMO 2024 silver-medal

Read the complete text