Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models
Andrew J. Kiruluta · 2026
Read Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models by Andrew J. Kiruluta (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-14156.txt.
Introduction
Kiruluta joins two usually distinct ways of shrinking large language models: static pruning and prompt trimming. His compressed-sensing setup probes hidden model use with random measurements, recovers task- and token-adaptive support sets, and turns them into hardware-efficient sparse execution paths. The design adds task-conditioned measurements, token-adaptive recovery, sample-complexity bounds, GPU-friendly limits, and a joint goal joining prompt compression with model reduction.
Catalog record
| CREATOR | Andrew J. Kiruluta |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 79145 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.14156
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Compressed-Sensing-Guided, Inference-Aware Structured Reduction for Large Language Models Andrew J. Kiruluta Abstract Large language models achieve strong generative performance at the cost of extreme parameter counts, large memory footprints, and substantial decoding latency. Existing compression methods have shown that sparse or structured reductions can preserve accuracy under aggressive pruning, and prompt-compression methods have further demonstrated that substantial latency reductions can be obtained by removing redundant input content before inference. However, these two lines of work remain largely disjoint. Most model-compression approaches are static, are optimized offline, and do not explicitly exploit the fact that different prompts and even different decoding steps activate different latent computational pathways. Conversely, prompt-compression methods reduce sequence length but do not directly adapt the executed model subnetwork. In this manuscript we propose a unified compressed-sensing-guided framework in which random measurement operators are used to probe latent model usage, sparse recovery is used to estimate task-conditioned and token-adaptive support sets, and the recovered supports are compiled into hardware-efficient sparse execution paths over blocks, attention heads, channels, and feed-forward substructures. The proposed formulation contributes five coupled novelties. First, it introduces task-conditioned measurements, so that different prompts induce different sparse supports and therefore different computational graphs. Second, it enables token-adaptive recovery, so that active substructures are re-estimated during decoding rather than fixed once offline. Third, it provides a formal sample-complexity analysis showing how many probe measurements are required to recover the active support under restricted isometry or mutual incoherence assumptions. Fourth, it imposes compile-to-hardware constraints so that recovered supports are restricted to structures compatible with efficient GPU kernels and practical runtime speedup. Fifth, it unifies prompt compression and model reduction in a single compressed-sensing objective, thereby coupling input-token selection and subnetwork selection rather than optimizing them independently. The resulting framework defines dynamic large language model execution as a measurement-and-recovery problem, with explicit approximation guarantees and deployment-oriented constraints. 1 Introduction The modern large language model is simultaneously a triumph of scale and a systems bottleneck. Increasing model size has improved reasoning, coding, and instruction-following performance, yet the associated computational costs have made deployment increasingly expensive in memory, bandwidth, energy, and end-to-end latency. As a consequence, a major body of work has investigated pruning, structured compression, sparse execution, and prompt compression as complementary routes toward more efficient inference. One-shot pruning methods such as SparseGPT have shown that large autoregressive transformers can be pruned at substantial sparsity without retraining, and activation-aware methods such as Wanda have shown that weight importance can be estimated effectively from weight–activation interactions. In parallel, inference-aware structured pruning methods such as ZipLM have highlighted the practical distinction between removing parameters and obtaining actual speedups on target hardware. Prompt-compression methods such as LLMLingua and LongLLMLingua have additionally demonstrated that many prompt tokens are redundant for downstream generation and can be removed to reduce cost while preserving utility. These advances establish that both model structure and input context contain substantial redundancy, but they stop short of treating inference itself as a dynamic sparse recovery problem in which the active computational pathway depends on the prompt and evolves across generated tokens. This observation motivates a different perspective. Rather than beginning from the premise that the full dense network must always be executed and then asking which components may be removed statically, we begin from the premise that, for a particular prompt and a particular decoding step, only a sparse subset of latent computational structures is truly necessary. The challenge is then not merely to prune weights, but to identify the active support of blocks, heads, channels, or neurons from a small number of cheap measurements, and to do so quickly enough that the measurement process itself does not dominate inference. This viewpoint is naturally aligned with compressed sensing. In compressed sensing, one does not directly observe a high-dimensional signal in full; instead, one acquires a limited number of linear or approximately linear measurements and then recovers the sparse latent representation under structural assumptions. Translating this idea to large language models suggests a dynamic architecture in which random feature sketches, calibration probes, or low-dimensional activation measurements are used to infer which structured subcomponents should execute for the current task or token. The central thesis of this manuscript is that compressed sensing provides not only a metaphor but also a mathematical and algorithmic framework for dynamic large language model execution. The proposed method uses randomized measurements of prompt-conditioned and token-conditioned features, recovers sparse support sets over structured computational units, and compiles those support sets into hardware-efficient sparse kernels. In contrast to purely static pruning, the support is allowed to vary across prompts. In contrast to conventional routing heuristics, the selection of active substructures is formulated as a sparse inverse problem with explicit assumptions and guarantees. In contrast to model-only compression, the framework also incorporates prompt compression in the same recovery objective, thereby jointly optimizing what information enters the model and which part of the model processes it. The main novelty of the framework lies in the interaction among five components. First, the measurement operators are task-conditioned, capturing the empirical fact that different prompts induce different internal usage patterns. Second, the sparse support can be recovered online during decoding, yielding token-adaptive execution rather than a single static reduced model. Third, the number of measurements required for reliable support recovery can be bounded using standard compressed-sensing tools such as restricted isometry properties and mutual incoherence conditions. Fourth, the admissible supports are restricted to structured sparsity patterns that map to efficient kernels on GPUs or similar accelerators. Fifth, prompt compression and model subnetwork recovery are solved jointly, so the system learns when it is preferable to shorten the sequence, when it is preferable to reduce the executed model, and when both forms of compression should occur together. 2 Relationship to Prior Work and Positioning Large language model pruning is already a mature and rapidly developing area, and any new framework must therefore be positioned carefully. SparseGPT demonstrated that one-shot pruning of very large autoregressive transformers can be posed as a sparse regression problem and solved layerwise with high empirical fidelity. Wanda showed that activation information can be incorporated in a particularly simple form by weighting magnitudes with input activations, leading to strong pruning performance without retraining. ZipLM then sharpened the systems perspective by emphasizing that unstructured sparsity often fails to translate into real speedups and by introducing inference-aware structured pruning targeted to specific runtime environments. These works establish that sparsity is both algorithmically meaningful and practically valuable, but their primary logic is still offline model reduction: they estimate importance, remove parameters, and then deploy a reduced but largely static network. Prompt compression follows a parallel but distinct trajectory. LLMLingua and its extensions show that prompts often contain substantial redundancy and that removing unimportant tokens can reduce cost while preserving semantic integrity and task performance. Yet prompt compression does not typically induce a corresponding dynamic reduction in the executed subnetwork. The model remains dense, and the reduced input merely changes its length. From a systems perspective, prompt compression and model compression therefore attack different terms in the total inference cost, but they are rarely optimized in a coupled fashion. The proposed work differs from these lines in a principled way. It does not merely prune weights, compress