On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation

Jia Feng, Zhanyue Qin, Cuiyun Gao, Ruiqi Wang, Chaozheng Wang, Yingwei Ma, Xiaoyuan Xie · 2026

Read On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation by Jia Feng, Zhanyue Qin, Cuiyun Gao, Ruiqi Wang, Chaozheng Wang, Yingwei Ma, Xiaoyuan Xie (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13725.txt.

Introduction

Feng and six colleagues test context compression for repository-level coding tasks, where long multi-file inputs bury key context, overflow windows, and slow inference. They sort eight methods into three groups: token sequences, latent vectors, and visual tokens. At fourfold compression, latent-vector methods beat full-context scores by up to 28.3 percent in BLEU, filtering noise rather than merely truncating. Visual and text compression also cut end-to-end latency by up to half at high ratios.

Catalog record

CREATORJia Feng, Zhanyue Qin, Cuiyun Gao, Ruiqi Wang, Chaozheng Wang, Yingwei Ma, Xiaoyuan Xie
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51418
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13725

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

On the Effectiveness of Context Compression for Repository-Level Tasks: An Empirical Investigation Jia Feng, Zhanyue Qin, Cuiyun Gao, Ruiqi Wang, Chaozheng Wang, Yingwei Ma, Xiaoyuan Xie Abstract . Repository-level code intelligence tasks, such as cross-file completion and project-aware code generation, require large language models (LLMs) to process long, multi-file contexts spanning complex dependencies. Such lengthy inputs introduce three practical challenges: crucial context can be obscured by surrounding noise across multiple files, or be truncated due to the limited context windows, and inference latency and memory consumption grow rapidly with context length. Context compression mitigates these risks by condensing long inputs into compact surrogates. While it has received considerable attention in the natural language processing literature , its applicability to code tasks remains largely unexplored; beyond efficiency gains, compression may further benefit LLM performance on code tasks by reducing contextual noise inherent in large, multi-file repositories. Therefore, we present the first systematic empirical study of context compression for repository-level code intelligence, organizing eight representative methods into three paradigms by their output representation space: discrete token sequences, continuous latent vectors, and visual tokens. We evaluate them on code completion and generation, two of the most widely studied tasks in repository-level code intelligence, jointly measuring task performance and deployment efficiency. Our results demonstrate that context compression is effective for code: at 4 × \times compression, methods based on continuous latent vectors surpass full-context performance by up to 28.3% with respect to the BLEU score , indicating that the latent vector compression filters repository noise rather than merely truncating context. On the efficiency side, all three paradigms reduce inference cost relative to full-context decoding. In particular, both visual and text-based compression achieve up to 50% reduction in end-to-end latency at high compression ratios, approaching the cost of inference without repository context. These findings establish context compression as a viable approach for repository-level code intelligence and provide guidance for paradigm selection under different deployment constraints. 1. Introduction Large language models (LLMs) have revolutionized many software engineering tasks, including code generation, program repair, code summarization, and automated testing ( 6 ; 18 ; 38 ; 30 ; 27 ) . State-of-the-art code LLMs such as DeepSeek-Coder ( 13 ) and Qwen2.5-Coder ( 15 ) are increasingly applied to repository-level tasks including cross-file completion, API-aware generation, and codebase question answering, which require reasoning over long, heterogeneous contexts spanning API definitions, dependency files, and historical implementations. Processing such inputs introduces three practical bottlenecks: inference cost and latency grow rapidly with context length due to the quadratic complexity of self-attention ( 31 ; 14 ; 33 ) ; task-critical evidence is frequently obscured by surrounding noise, a phenomenon known as “lost in the middle” ( 23 ) ; and finite context limits force truncation when inputs span large codebases or cross-module dependencies. These challenges are particularly acute for code, where semantic dependencies are dense and long-range, syntactic constraints are strict, and the information-to-noise ratio is considerably lower than in natural language. Context compression attempts to address these bottlenecks by transforming raw context into a compact surrogate representation. Existing methods can be categorized into three paradigms by the representation space of their outputs. Text-to-Text (T2T) methods produce a shorter, discrete token sequence via token-level filtering or rewriting; representative approaches include LLMLingua ( 16 ) , LLMLingua-2 ( 26 ) , and LongLLMLingua ( 17 ) . Text-to-Vector (T2V) methods encode context into a compact set of continuous latent tokens; ICAE ( 10 ) distills long contexts into fixed-size memory slots, while gist-token approaches ( 24 ) and 500xCompressor ( 21 ) extend this paradigm to more aggressive compression ratios. Text-to-Image (T2I) methods render context as images consumed by a vision-language model, exploiting the higher information density of the visual modality; optical encoding has been shown to achieve up to 10\times token reduction while maintaining high reconstruction fidelity ( 36 ; 37 ; 2 ) , with extensions to diverse long-context settings ( 8 ) . Despite this growing body of work, context compression has been studied almost exclusively on NLP tasks in method-specific settings. Whether and how these techniques transfer to code LLMs remains an open question: repository-level code imposes constraints absent in natural language, including long-range cross-file dependencies, strict syntactic structure, and a lower information-to-noise ratio that amplifies the risk of aggressive filtering. We address this gap by presenting, to the best of our knowledge, the first systematic empirical study of context compression for repository-level code LLMs, instantiating eight representative methods across all three paradigms and studying them on code generation and completion, two of the most widely studied repository-level tasks, jointly measuring task performance and deployment efficiency across multiple model scales. We study the following research questions (RQs) and summarize the findings : RQ1: How do different compression paradigms affect task performance? Context compression not only preserves downstream task performance but can actively improve it over full-context inference. T2V surpasses the full-context baseline by up to 28.3% in terms of BLEU on Python completion. We attribute this to the latent vectors acting as a learned filter that suppresses repository boilerplate and reinforces task-relevant signals. T2I tracks full-context performance closely on completion tasks and sustains competitive performance at moderate compression ratios. T2T provides training-free compression that is effective at mild ratios, though performance gaps widen on generation tasks where cross-file relational structure is critical. RQ2: How does compression ratio affect task performance? T2V sustains its performance advantage across the full evaluated range of 4\times to 128\times , with all variants exceeding full-context performance on generation at every ratio, demonstrating that effective compression does not require conservative token budgets. T2I peaks at 4\times on completion and degrades gradually with further downscaling, yet its generation performance remains consistently below full context regardless of ratio, as uniform rendering loses cross-file relational structure at any resolution. T2T degrades most sharply: generation performance converges to the no-context baseline at approximately 7\times to 12\times , establishing a practical upper bound for perplexity-based pruning on code tasks. RQ3: How do compression paradigms and ratios affect throughput and resource usage? All three paradigms deliver measurable efficiency gains over full-context inference, with decoding cost reduction as the primary driver across paradigms. T2I achieves a 33% reduction in end-to-end latency at its performance-optimal 4\times ratio, with resource costs converging to the context-free baseline at higher compression ratios. T2V maintains a stable and predictable overhead profile across all evaluated ratios, remaining well below full-context cost throughout. T2T reduces total inference latency by over 35% relative to full-context inference at moderate compression ratios, with decoding cost and GPU memory converging toward the resource cost of inference without repository context at higher ratios. Collectively, these results demonstrate that context compression is not only viable but, under appropriate paradigm selection, performance-enhancing for repository-level code intelligence, yielding both performance improvements and substantial efficiency gains that make it a practically compelling approach for code LLM deployment. In summary, this paper makes the following contributions: • We introduce context compression to repository-level code intelligence through the first systematic empirical study spanning three paradigms (T2T, T2V, T2I) and eight representative methods, evaluated on code generation and completion across multiple model scales. • We reveal that latent-vector compression can surpass full-context performance on code tasks, establishing that compression

Read the complete text