Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal Data

Chandana Cheerla · 2025

Read Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal Data by Chandana Cheerla (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-12425.txt.

Introduction

Chandana Cheerla builds a question-answering system that can search a firm's own records, tables, and reports. The 2025 paper tackles limits of plain chatbots: fixed training, short memory, and weak handling of mixed formats. Standard search-aided designs also stumble on structured data.

The design blends two search styles, dense vectors and keyword match, then filters by metadata and re-ranks hits. Text is chunked by meaning, while tables keep their row and column shape. Human feedback and chat memory help the system adapt. On firm datasets, top-five precision rose from seventy-five to ninety, and recall from seventy-four to eighty-seven. Faithfulness scores also jumped. Code release is planned, with multimodal search as future work.

Catalog record

CREATORChandana Cheerla
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE27 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-12425

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Advancing Retrieval-Augmented Generation for Structured Enterprise and Internal Data Chandana Cheerla Abstract Organizations increasingly rely on proprietary enterprise data—including HR records, structured reports, and tabular documents—for critical decision-making. While Large Language Models (LLMs) exhibit strong generative capabilities, they remain constrained by static pretraining, limited context windows, and challenges in processing heterogeneous data formats. Although conventional Retrieval-Augmented Generation (RAG) frameworks solve some of these constraints, they often fall short in handling structured and semi-structured data effectively. This work introduces an advanced RAG framework that combines hybrid retrieval strategies leveraging dense embeddings (all-mpnet-base-v2) and BM25, enhanced through metadata-aware filtering using SpaCy NER and cross-encoder reranking for improved relevance. The framework employs semantic chunking to preserve textual coherence and explicitly retains the structure of tabular data, ensuring the integrity of row-column relationships. Quantized indexing techniques are integrated to optimize efficiency, while a human-in-the-loop feedback mechanism and conversation memory enhance the system’s adaptability over time. Evaluations on proprietary enterprise datasets demonstrate significant improvements over baseline RAG approaches, with a 15% increase in Precision@5 (90% vs. 75%), a 13% gain in Recall@5 (87% vs. 74%), and a 16% improvement in Mean Reciprocal Rank (0.85 vs. 0.69). Qualitative assessments further validate the framework’s effectiveness, showing higher scores in Faithfulness (4.6 vs. 3.0), Completeness (4.2 vs. 2.5), and Relevance (4.5 vs. 3.2) on a 5-point Likert scale. These findings underscore the framework’s capability to deliver accurate, comprehensive, and contextually relevant responses for enterprise knowledge tasks. Future work will focus on extending the framework to support multimodal data and integrating agent-based retrieval systems. The source code will be progressively released at: GitHub Repository . Keywords: Retrieval-Augmented Generation (RAG), Structured Data Retrieval, Hybrid Retrieval, Tabular Data Processing, Cross-Encoder Reranking, Enterprise Knowledge Augmentation, Metadata Filtering, Semantic Chunking, Query Reformulation, Feedback-Driven Retrieval 1 Introduction Large Language Models (LLMs) such as GPT-4 ( openai2023gpt4 ) , PaLM ( chowdhery2022palm ) , and LLaMA ( touvron2023llama ) have significantly advanced capabilities in natural language understanding and generation, excelling in tasks like question answering, summarization, and knowledge retrieval. Despite these achievements, LLMs remain inherently constrained by static pretraining on fixed corpora and limited context windows ( bommasani2021opportunities ; press2022train ) , which bounds their adaptability to dynamic or proprietary enterprise data. In domains like corporate governance, human resources, and finance, critical information is often encapsulated in structured records, policy documents, and tabular formats that LLMs cannot naturally ingest or reason over post-deployment. Retrieval-Augmented Generation (RAG) frameworks have emerged to address this gap by integrating retrieval mechanisms with LLMs, enabling models to fetch external knowledge at inference time ( lewis2020retrieval ; izacard2020leveraging ) . This paradigm enhances response relevance and factuality by grounding generation in up-to-date, domain-specific data. However, conventional or baseline RAG methods, optimized primarily for unstructured textual data, face notable challenges when applied to enterprise datasets comprising structured, semi-structured, and tabular information ( guu2020realm ) . Key limitations include: • Fragmented Contextual Representation: Baseline chunking strategies, typically using fixed token-length splits, often fracture meaningful contexts, especially in complex documents like policies or manuals ( phang2021clustering ) . • Inadequate Handling of Tabular Data: Flattening tables into linear text formats destroys the intrinsic row-column relationships necessary for precise data retrieval within tables ( zhang2020turl ) . • Limited Retrieval Completeness: Exclusive reliance on dense embeddings or sparse keyword methods like BM25 ( robertson2009probabilistic ) restricts the model’s ability to balance semantic understanding with exact matching. • Absence of Relevance Reordering: Without reranking mechanisms, initial retrieval results may not adequately prioritize the most relevant information ( karpukhin2020dense ) . • Static Query Interpretation: These systems lack the capability to refine or disambiguate queries dynamically, resulting in suboptimal retrieval for vague or incomplete user prompts. 1.1 Contributions To address these failings, we introduce an enterprise-focused RAG framework that systematically enhances the retrieval and generation process across diverse data modalities. The contributions of our system are as follows: 1. Hybrid Retrieval Mechanism: We combine dense semantic retrieval using all-mpnet-base-v2 ( reimers2019sentence ) with BM25-based sparse retrieval, weighted at a 0.6 to 0.4 ratio, to synergize semantic depth with lexical accuracy. 2. Structure-Aware Chunking: For text, we adopt a recursive character-based chunking strategy with empirically tuned chunk sizes (700 tokens) that balance coherence and granularity. For tabular data, we employ Camelot ( camelot ) and Azure Document Intelligence to extract and store tables as structured JSON with metadata capturing file name, row identifiers, and column headers, enabling row-level indexing via FAISS for fine-grained retrieval. 3. Metadata-Driven Filtering: Named Entity Recognition (NER) via spaCy ( spacy ) enriches document metadata, facilitating entity-aligned filtering that sharpens retrieval relevance. 4. Contextual Re-Ranking: We integrate a cross-encoder reranking layer using MS MARCO fine-tuned models ( nogueira2019passage ) , which reorders retrieved candidates based on contextual alignment with the query. 5. Interactive Query Refinement: Leveraging LLaMA and Mistral models on ChatGroq, our system supports query expansion and rephrasing, guided by user feedback and conversational memory to iteratively enhance retrieval and response quality. We evaluated our framework on a corpus of publicly available enterprise policy documents, including the HR Policy Dataset and analogous corporate materials. For tabular data, we experimented with both full-table chunking and row-level indexing, finding the latter significantly enhances precision in row-specific queries while the former suffices for small tables. 1.2 Results Our approach delivers substantial improvements over baseline RAG systems: • Precision@5: 90% vs. 75% • Recall@5: 87% vs. 74% • Mean Reciprocal Rank (MRR): 0.85 vs. 0.69 These results underscore the robustness and applicability of our framework in real-world enterprise contexts, delivering superior retrieval accuracy, comprehensive responses, and higher contextual relevance. We envision extending this framework towards agentic RAG systems ( gao2022precise ; shinn2023reflexion ) , where intelligent agents autonomously select retrieval strategies, adaptively reformulate queries, and integrate multimodal data sources—including images, scanned documents, and audio—thereby broadening the landscape of enterprise knowledge augmentation. 1.3 Paper Organization The rest of the paper is organized as follows: • Section 2: Elaborates on the related work. • Section 3: Presents the methodology. • Section 4: Discusses experiments and results. • Section 5: Concludes with directions for future work, including potential extensions towards agentic retrieval systems. 2 Related Work Retrieval-Augmented Generation (RAG) has emerged as a pivotal paradigm to bridge the limitations of static LLMs by dynamically fetching relevant information at inference time. Lewis et al. ( lewis2020retrieval ) introduced the foundational RAG architecture by coupling dense retrievers with generative models, improving factual grounding in open-domain question answering. Building on this, Izacard and Grave ( izacard2020leveraging ) proposed Fusion-in-Decoder (FiD), integrating multiple retrieved contexts directly into the decoding process, significantly enhancing response accuracy. Dense retrieval techniques such as Dense Passage Retrieval (DPR) ( karpukhin2020dense ) rely on bi-encoder models to map queries and documents into the same vector space, enabling efficient retrieval via vector similarity. However, DPR and similar dense retrievers can struggle with lexical precision, especially when queries are domain-specific or contain rare terminology. Sparse retrieval methods like BM25 ( robertson2009probabilistic ) compensate for this by focusing on exact term matching, which, while precise, often lacks semantic understanding. To overcome the trade-offs between dense and sparse retrieval, hybrid strategies have been explored. Mialon et al. ( mialon2023augmented ) surveyed augmented language models and emphasized that combining dense and sparse signals leads to

Read the complete text