CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression

Morayo Danielle Adeyemi, Ryan A. Rossi, Franck Dernoncourt · 2026

Read CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression by Morayo Danielle Adeyemi, Ryan A. Rossi, Franck Dernoncourt (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-24083.txt.

Introduction

Adeyemi, Rossi, and Dernoncourt test a popular money-saving claim about language models: that writing in a telegraphic caveman style, dropping grammar to save tokens, cuts inference costs. Whether compression helps depends on which side is squeezed, the user's prompt or the model's reply. Their 2026 protocol, Cavewoman, scores every generation on task accuracy, actual per-item cost, and agreement with what the model wrote when left unconstrained.

Eight models face five datasets at five compression levels, with both channels measured on identical items. Squeezing the reply works: output compression lowers realised cost on most commercial models and on all open-weight models tested. Squeezing the prompt backfires completely, raising net cost because models answer at greater length while accuracy collapses. Adeyemi's team also finds the surviving text drifting from the model's own unconstrained wording, with many correct answers no longer matching the reference on reasoning-free models.

Catalog record

CREATORMorayo Danielle Adeyemi, Ryan A. Rossi, Franck Dernoncourt
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE77 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.24083

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression Morayo Danielle Adeyemi, Ryan A. Rossi, Franck Dernoncourt Abstract “Talk short. Drop grammar. Save token.” This caveman style is widely promoted as a way to cut inference cost, but whether it actually saves anything depends on which channel (the user’s prompt or the model’s response) is being compressed. We present Cavewoman , a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement against the model’s unconstrained reference. We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items. Output compression cuts realized cost on most API models ( 1.4–2.4 × \times per model, up to 3 × \times in the best case) and on all four open-weight models under public-tier pricing. Input compression has the opposite effect, a strict lose-lose: it raises net cost rather than lowering it ( ≈ 1.15 × \approx 1.15\times on the five-benchmark mean, up to 1.8 × \times on the worst dataset and 2.7 × \times under stronger compression), because models compensate with longer responses even as accuracy collapses. Under the same setting, surface text diverges from the unconstrained reference: on the non-reasoning models, roughly half of all generations are correct yet their surface text no longer entails the model’s own unconstrained baseline generation. The divergence survives length-controlled re-scoring, multiple-comparisons correction, and replication under complementary semantic measures. Code and data are available at https://github.com/danielle34/cavewoman . Figure 1: Cavewoman framework. The input-compression channel applies a deterministic part-of-speech filter to the user prompt at five reduction levels, leaving the system prompt fixed. The output-compression channel leaves the prompt verbatim and replaces the system prompt with a level-specific instruction that requires the same reduction in the response. Every generation is scored on task accuracy, reference-text agreement against the model’s unconstrained reference under bidirectional NLI (plus eleven complementary measures), and per-item input/output token cost. 1 Introduction Inference cost in large language models scales with both input and output token counts, and output tokens are typically priced 4–8 \times higher than input tokens ( 1 ; 38 ) . Existing compression methods reduce one side or the other, either the prompt ( 24 ; 42 ; 8 ; 43 ) or the response ( 51 ; 45 ) , but the two are studied separately and both are evaluated almost entirely through task accuracy at a reduced token count. Accuracy at a token count is the wrong instrument for two reasons. It does not separate realized cost from prompt-token reduction (a shorter prompt does not reduce realized cost when the model answers at greater length), and it collapses each generation to a binary outcome that cannot distinguish a compressed answer that agrees with the model’s unconstrained reasoning from one that diverges from it. We address both gaps with Cavewoman , a two-channel evaluation protocol that scores every generation on three axes: task accuracy, realized per-item cost on the priced channel, and reference-text agreement against the model’s own unconstrained generation. The protocol measures input compression (the prompt is filtered before the model sees it) and output compression (the model is instructed to answer in a constrained register) at five reduction levels, holding model and item fixed. We evaluate eight models (Qwen2.5-VL-7B, Qwen3.5-9B, DeepSeek-R1-Distill-Qwen-7B, Gemma-4-E4B, GPT-4o, GPT-5.4, Claude Haiku 4.5, Claude Sonnet 4.6) on five benchmarks (GSM8K, BoolQ, ARC-Easy, CommonsenseQA, MMLU-STEM) with complete coverage of both channels at all five levels. Contributions. 1. We propose a two-channel evaluation protocol that scores compression on realized cost with audited answer-extraction rates and a twelve-metric semantic battery (§ 3 ). 2. We measure cost asymmetry between input and output compression on the same items (§ 4.1 ). 3. We measure a surface-text divergence between correct answers and the model’s unconstrained reference under output compression, replicated across complementary semantic measures (§ 4.2 ). 2 Related Work Input compression. Input compression splits along a hard–soft axis ( 34 ) . Hard methods prune tokens by self-information ( 33 ) , perplexity ( 24 ) , question-aware scoring ( 25 ) , or distilled token classification ( 42 ) , with the same logic at the document level via retrieved-context summarization ( 52 ) . Soft methods encode prompts as gist tokens ( 37 ) , recursive summary vectors ( 10 ) , or autoencoder slots ( 15 ) , with rate–distortion bounds in 39 . None measures whether the response says the same thing it would have without compression. Output compression. Output-side methods constrain generation via modified decoding (length controls ( 27 ) , budget-signalling positional encodings ( 47 ) , countdown mechanisms ( 45 ) ) or via post-training and prompting (token-skipping chains ( 51 ) , cognitive-inspired routing ( 4 ) , difficulty-aware prompting ( 19 ) , RL-driven demonstration compression ( 22 ) ). The content/function-word split the caveman register exploits is long-standing in linguistics, but this line of work still reports only task accuracy at a token budget. Semantic fidelity, verbosity, and cost. Bidirectional NLI entailment separates propositional content from lexical form ( 29 ) and has scored summarization faithfulness ( 36 ) and inter-system consistency ( 30 ) ; relatedly, chains of thought can be plausible yet causally disconnected from the prediction ( 49 ; 31 ) . Length itself matters: verbose outputs score lower ( 54 ) , verbosity carries cost ( 6 ) , irrelevant padding degrades reasoning ( 32 ) , and chain-of-thought length tracks accuracy independently of trace correctness ( 26 ; 46 ; 50 ) . The cost–quality frontier is itself a design surface, in selective routing ( 9 ) , quality thresholds ( 14 ) , and preference routers ( 40 ) ; valid benchmarking calls for multi-metric measurement ( 7 ; 35 ; 16 ) . Positioning. The closest prior work compresses one side only: LLMLingua ( 24 ; 42 ; 25 ) on the input; TokenSkip ( 51 ) , Hansel ( 45 ) , and Sketch-of-Thought ( 4 ) on the output, all reporting task accuracy at compressed token budgets. Cavewoman measures both channels on the same items, reports realized per-item cost rather than token reduction, audits answer-extraction rates before any accuracy claim, and scores reference-text agreement under complementary semantic criteria. The input-channel divergence reproduces under LLMLingua-2 (Appendix D ). 3 Methodology 3.1 Experimental Design Setup. Let \mathcal{M} be a fixed language model and let x=(w_{1},\dots,w_{n}) be a question, a sequence of n tokens from the spaCy tokenizer. Each token w carries a Penn Treebank part-of-speech tag g(w) assigned by spaCy with the en_core_web_sm model; g is deterministic given a fixed spaCy version and model. For a set S of part-of-speech tags, write \big(w_{i}\big)_{\,i\,:\,g(w_{i})\in S} for the subsequence of x that retains exactly the tokens whose tag lies in S , with the indices i taken in increasing order. Let \mathrm{trunc}_{k}(z) denote the prefix of a sequence z of length \min(|z|,k) , and let \texttt{NN}{*} and \texttt{VB}{*} denote the Penn Treebank noun and verb tag families. We study a single reduction parameter, the level \ell\in\{0,1,2,3,4\} , named L0 through L4. The level selects one filter from the family \phi_{0},\dots,\phi_{4} defined by \displaystyle\phi_{0}(x) \displaystyle=x, (1) \displaystyle\phi_{1}(x) \displaystyle=\big(w_{i}\big)_{\,i\,:\,g(w_{i})\notin\{\texttt{DT},\texttt{IN},\texttt{CC},\texttt{RP},\texttt{TO},\texttt{MD}\}},

Read the complete text