Beyond Long Context: When Semantics Matter More than Tokens
2025
Read Beyond Long Context: When Semantics Matter More than Tokens (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-25816.txt.
Introduction
Tarun Kumar Chawdhury and Jon D. Duke test a method for answering questions from long clinical notes stored in electronic health records. The notes are stored as coded attachments, so search tools often cut them into chunks and miss links between medical facts. The method they test, called CLEAR, retrieves text by clinical entities, meaning named diseases, drugs, and other medical terms.
Chawdhury and Duke built a test platform and ran it on twelve notes that ranged from ten thousand to sixty five thousand tokens. CLEAR won more often than wide context and chunk based methods, matched meaning well, and used far fewer tokens. Gains were largest on the longest notes.
Catalog record
| YEAR | 2025 |
|---|---|
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 23 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-25816
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Beyond Long Context: When Semantics Matter More than Tokens Abstract Background: Electronic Health Records (EHR) systems store clinical documentation in FHIR DocumentReference resources as base64-encoded attachments, presenting significant challenges for semantic question-answering applications. Traditional approaches using statistical correlation through vector database chunking often fail to capture the nuanced clinical relationships required for accurate medical information extraction. The Clinical Entity Augmented Retrieval (CLEAR) methodology, introduced by Lopez et al. (2025) [ 1 ] , addresses these limitations through entity-aware retrieval strategies and reports improved performance (F1 0.90 vs. 0.86 for embedding RAG; > > 70% fewer tokens and faster inference). Objective: To develop a comprehensive evaluation platform for clinical notes question-answering systems and validate CLEAR against established approaches—including zero-shot large-context processing and traditional chunk-based retrieval-augmented generation—in realistic EHR processing scenarios. Methods: We implemented a Clinical Notes Q&A Evaluation Platform with three retrieval strategies: (1) Wide Context processing for zero-shot inference with large context windows, (2) traditional vector database chunking with semantic search, and (3) entity-aware CLEAR with medical domain knowledge. Evaluation encompassed 12 clinical documents (10K–65K tokens) representing typical EHR DocumentReference content. Results: CLEAR showed a 58.3% win rate across test cases, achieving 0.878 average semantic similarity while requiring 78% fewer tokens than wide-context processing. Gains were most pronounced on large notes (75% win rate for 65K + tokens), consistent with published scalability claims. Conclusions: The Clinical Notes Q&A Evaluation Platform validates CLEAR’s advantages for semantic clinical retrieval in EHR settings where computational efficiency and semantic accuracy are critical, and provides a reusable framework for evaluating clinical NLP approaches in production environments. Tarun Kumar Chawdhury and Jon D. Duke Georgia Institute of Technology [email protected] [email protected] Keywords: Clinical NLP, Entity-Aware Retrieval, Evaluation Platform 1. Introduction Electronic Health Record (EHR) systems in modern healthcare infrastructure store clinical documentation within FHIR DocumentReference resources, typically encoded as base64 attachments containing unstructured clinical notes. These documents, ranging from brief progress notes to comprehensive discharge summaries, present substantial challenges for automated question-answering systems that require semantically accurate information extraction rather than statistical correlation-based retrieval common in traditional vector database approaches. Contemporary approaches to clinical document processing have largely focused on two paradigms: zero-shot inference with large context windows that process entire documents but face computational constraints and the ”lost in the middle” problem, and chunk-based retrieval-augmented generation (RAG) systems that utilize vector databases for semantic similarity search but often fail to capture critical clinical entity relationships and contextual dependencies essential for accurate medical information extraction. A growing body of evidence shows that merely expanding context windows does not guarantee effective use of information: performance often drops when relevant spans occur in the middle of long inputs (“lost in the middle”) [3]. This motivates entity-aware retrieval that selectively centers clinically relevant spans rather than relying on statistically similar but potentially off-target chunks. The CLinical Entity Augmented Retrieval (CLEAR) methodology, published by Lopez et al. in 2025 [ 1 ] , introduced a novel approach that addresses these limitations through entity-aware, entity-centered retrieval strategies. The original study demonstrated significant performance improvements (F1 score of 0.90 vs 0.86 for traditional RAG) with substantial efficiency gains (71% token reduction, 72% faster inference time) on clinical information extraction tasks, positioning CLEAR as a potentially transformative approach for production EHR processing systems. To validate these claims in realistic healthcare scenarios and provide a robust evaluation framework for clinical NLP approaches, we developed the Clinical Notes Q&A Evaluation Platform. This comprehensive validation study implements and compares three fundamental approaches: (1) wide context processing for zero-shot inference with large language models, (2) traditional vector database chunking with embedding-based retrieval, and (3) entity-aware CLEAR methodology adapted for EHR DocumentReference processing. Our contributions include: systematic validation of CLEAR’s performance claims, development of a reusable evaluation platform for clinical NLP research, and empirical analysis of retrieval strategy performance across clinical documents of varying complexity and length. 1.1 Related Work Entity-aware retrieval has gained increasing attention within biomedical NLP and question-answering domains. Early retrieval-augmented methods such as RAG [ 2 ] demonstrated the potential of embedding-based chunk retrieval but lacked domain-specific entity modeling. In the clinical domain, approaches leveraging UMLS concepts and ontology-based retrieval (e.g., Neumann et al., 2019; Johnson et al., 2016) provided partial improvements but often failed to maintain contextual continuity across long clinical narratives. CLEAR [ 1 ] represented a significant advancement by introducing entity-centered retrieval aligned with clinical semantics. Our work extends this line of research by operationalizing CLEAR within an end-to-end evaluation platform, providing reproducible empirical validation across realistic EHR-scale document sets. Recent evaluations of retrieval-augmented models in long-context reasoning (e.g., Karpinska et al., 2023; Xiong et al., 2024) emphasize that retrieval strategies often outperform naive long-context prompting, supporting the need for entity-aware retrieval. Our work contributes a reproducible evaluation framework within this paradigm, focusing on realistic EHR-scale clinical notes. 2. Methods 2.1 Protocol Alignment with CLEAR and Key Differences Our implementation is CLEAR-inspired rather than an exact re-implementation. In contrast to Lopez et al. (2025) [1], we (i) use keyword- and pattern-based entity recognition with vital/lab value extraction instead of model-based NER, (ii) do not apply ontology- or LLM-driven synonym augmentation, and (iii) use fixed-size local windows around entities as an implementation choice (the CLEAR paper does not mandate a specific window size). We evaluate open-ended QA over synthetic clinical notes rather than structured IE. These differences mean our results should be interpreted as consistent with CLEAR’s trend, not a reproduction of its exact metrics. 2.2 Enhanced CLEAR Implementation Our enhanced CLEAR implementation builds upon the original methodology with several key improvements designed for practical clinical deployment. The system incorporates four main components: enhanced entity extraction, section-aware processing, intelligent context selection, and token optimization. 2.2.1 Enhanced Entity Extraction We developed a comprehensive medical entity recognition system using advanced keyword patterns, clinical value recognition for vital signs and laboratory values, and medical domain-specific entity types with confidence-based scoring. The system recognizes six primary entity categories: medications, symptoms, diseases, procedures, laboratory values, and anatomical references. 2.2.2 Section-Aware Processing Clinical documents follow standardized section formats (e.g., ASSESSMENT, PLAN, HISTORY OF PRESENT ILLNESS). Our implementation identifies these sections and applies priority-based weighting, with ASSESSMENT and PLAN sections receiving highest priority (weight = 1.0), followed by HISTORY OF PRESENT ILLNESS (weight = 0.9), and other sections receiving proportional weights. 2.2.3 Context Selection Algorithm We implement fixed-size context windows of \pm 150 words around identified medical entities to bound tokens while preserving local semantics. (CLEAR retrieves windows around entities but does not mandate a specific window size.) Our context selection algorithm incorporates question–entity semantic alignment and medical relationship scoring to prioritize clinically relevant spans. 2.3 Baseline Methods We compared our enhanced CLEAR implementation against two baseline approaches: Wide Context Processing: Complete clinical note processing using full document context. This approach provides comprehensive information access but requires significant computational resources (average 39,173 tokens per query). Retrieval-Augmented Generation (RAG): Semantic chunking with embedding-based retrieval using top-k chunk selection. This approach prioritizes efficiency with minimal token usage (average 544 tokens per query) but may miss critical clinical relationships. 2.4 Evaluation Framework Evaluation was conducted on a dataset of 12 clinical notes ranging from 10,000 to 65,000 tokens, representing diverse clinical scenarios. Each note was accompanied by clinical questions requiring information extraction and