Agentic System as Compressor: Quantifying System Intelligence in Bits
Zihan Qin, Hongrui Zhang · 2026
Read Agentic System as Compressor: Quantifying System Intelligence in Bits by Zihan Qin, Hongrui Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-25960.txt.
Introduction
Zihan Qin and Hongrui Zhang propose measuring the intelligence of an AI agent system in bits. Their starting point is the old idea that compression reflects intelligence. Under a fixed task set, interface, and compute budget, a stronger system should rebuild a target object from fewer bits. Tools, retrieval, verifiers, and search all count as part of the system being measured.
Arithmetic coding, seed coding, and a fallback channel turn the idea into a working meter. Five test settings probe it: protein sequences, chess moves, reversed text, story compression, and retrieval-augmented question answering. In every setting, agent components shrink the codelength needed. Qin and Zhang argue the measure shows how parts, observers, and budgets change the leftover uncertainty, offering guidance for judging real agent systems beyond pass-or-fail scores.
Catalog record
| CREATOR | Zihan Qin, Hongrui Zhang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 68 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.25960
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Agentic System as Compressor: Quantifying System Intelligence in Bits Zihan Qin, Hongrui Zhang Abstract Large language models are turning from isolated predictors into agentic systems : they call tools, retrieve evidence, obey environment constraints, use verifiers, and complete tasks through search and multi-turn interaction. We adopts an analytical viewpoint based on “compression is intelligence”: under a fixed task distribution, interface, and compute budget, a stronger agentic system lets a target object be reconstructed with fewer bits . We operationalize the measure with arithmetic coding , seed coding , and a fallback , and evaluate it in five settings: reversed text, chess moves, protein sequences, retrieval-augmented question answering, and semantic story compression; in all of them agentic components reduce codelength. These small, controlled experiments cover component types typical of real agentic systems, show that codelength can analyze how components, observers, and budgets change residual uncertainty, and offer guidance for evaluating real agent systems. 1 Introduction Large language models appear less and less as isolated predictors and more and more embedded in agentic systems that can call tools, retrieve evidence, read feedback, and interact with an environment ( 29 ; 18 ; 13 ; 19 ; 26 ) . In real deployments, intelligent behavior often arises from the whole workflow. That is, the capability deployed and used today is no longer just “model intelligence” but system intelligence formed jointly by the model, tools, environment interfaces, and interaction process. This creates a tension for evaluation. Standard benchmarks usually report accuracy, success rate, pass rate, or an aggregate score ( 14 ; 20 ) ; these metrics indicate whether the system finally completes the task, but say little about where the capability comes from. When a system grows stronger, is it because the model itself predicts better, or because a retriever supplied evidence, a rule-based environment ruled out illegal actions, a verifier filtered out bad trajectories, or a search process found better candidates? How much does a given component actually contribute? When a task is hard, the success rate can stay near zero for a long time; both systems then “almost fail,” yet their reductions of the residual uncertainty over the target space may already differ enormously. The compression-as-intelligence viewpoint offers a finer analytical framework ( 5 ; 25 ; 6 ; 10 ) . Classical information theory shows that a predictive distribution can be turned into a lossless compressor and vice versa ( 5 ; 25 ) ; a language model induces token-level codelengths, making log-loss and bits-per-byte an operational compression rate ( 6 ) ; and cross-model results further show that the compression rate is nearly linearly correlated with downstream capability ( 10 ) . These works, however, mainly measure the bare model itself, without bringing tools, environment, retrieval, and the interaction process into the object of measurement. This paper therefore adopts agentic codelength as a system-level working measure. Intuitively, once the task distribution, observation standard, system interface, and coding protocol are fixed, the encoder need not transmit the target object in full; it sends only a per-instance code that lets a decoder sharing the same system conditions reconstruct the target. The more intelligent a system, the fewer residual bits it should require to reconstruct objects from the same distribution. Bare-model compression is just a special case: a system can choose to call no tools, do no retrieval, and no search, degenerating into ordinary model compression; conversely, when environment interaction provides usable structure, the system can turn that structure into a shorter per-instance code. To estimate this system compression ability, we combine the following three schemes into an executable compress–decompress protocol. First, arithmetic coding accumulates the negative log-likelihood (NLL) on a given output, turning model predictions into codelength, for cases where a specific output must be encoded exactly. Second, seed coding targets tasks in deterministic or replayable environments where qualifying outputs are dense enough within the sampling budget: it transmits a latent description plus the rank of the first successful random seed. Third, when seed coding fails within a finite budget, it falls back to arithmetic coding, so that every sample can be encoded successfully. We validate the framework in five settings: reversed text, chess, protein sequences, retrieval-augmented question answering, and semantic story compression. These settings respectively isolate several types of information source typical of agentic systems: deterministic tools, rule-based environments, scientific priors, verifier feedback, retrieval side information, semantic observers, and the sampling budget. Reversed text provides a minimal sanity check, testing whether a shared transform can move per-instance structure into the decoder; chess tests how environment constraints shrink the space of legal candidates; the protein experiments test both whether codelength can read out the value of template priors when the success rate has almost no resolution, and whether verifier feedback can reduce blind-search cost in functionally equivalent generation; RAG and story compression respectively test how external-evidence relevance, the observation standard, and the compute budget change the residual codelength. In short, the goal of this set of experiments is not to show that some agent achieves the highest score on a single task, but to verify that agentic codelength can serve as a unified accounting tool for residual uncertainty across different information sources and different success criteria. Figure 1 : Overview of agentic compression. Tools, constraints, context, search, and verifiers act as shared decoding resources, reducing the per-instance hint, seed/rank, and residual bits needed for a reconstruction accepted by the observer. Our contributions are as follows. First, we are the first to understand the “compression is intelligence” view from the perspective of agentic systems rather than isolated models. Second, we give a marginal bit value metric for components, quantitatively analyzing the contribution of a specific component in a system via the average codelength reduction. Third, we combine arithmetic coding, seed replay, and fallback coding into an executable compress–decompress protocol. Fourth, we validate the effectiveness of this metric for analyzing system intelligence across five classes of tasks. 2 Related Work 2.1 Compression, intelligence, and “language models as compressors” “Compression measures understanding” is a classical idea: in an appropriate sense, prediction and compression are interconvertible, and the two are essentially equivalent ( 5 ; 25 ) . Recent discussions of large models, including Sutskever’s talk An Observation on Generalization , popularized a sharper operational reading: a predictor that assigns higher probability to a data distribution also induces a shorter code for samples from that distribution ( 21 ) . This idea has a more systematic formalization in MDL (minimum description length) and universal coding. The Kraft–McMillan inequality shows that a uniquely decodable codelength function corresponds to a probability budget; and the log-loss risk -\log q(x) of a probabilistic model q is precisely the ideal description length it induces. Hence, learning within a model class \mathcal{M}=\{q_{\theta}\} is not only a search for parameters with low predictive risk, but also a search for a description language that assigns short codes to data. The MDL principle writes this relationship as a two-part code or universal-coding criterion: choose an explanation by the sum of the model description length L(\theta) and the residual data description length L(x\mid\theta)\simeq-\log q_{\theta}(x) ( 17 ; 2 ; 7 ) . From this perspective, generalization can be understood as structure discovery in the compression