Topic-Enriched Embeddings to Improve Retrieval Precision in RAG Systems for LLMs

Rodrigo Kataishi, Ph.D · 2026

Read Topic-Enriched Embeddings to Improve Retrieval Precision in RAG Systems for LLMs by Rodrigo Kataishi, Ph.D (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-00891.txt.

Introduction

Rodrigo Kataishi is the sole author of a 2026 paper on how to help AI systems find the right texts to cite. Such systems fetch passages from a store, then write answers based on them. If the fetch is poor, the answer is poor. Kataishi enriches each text's code with topic data before the fetch. The code blends three views of the text.

One view counts key terms, in the classic term-weight style. A second view maps latent topics, found by two standard topic tools. A third view is the compact code from a small modern language model. Joined, the code holds both term-level and topic-level sense. Tests on a legal text store show the gains. Texts grouped more cleanly, and fetches scored higher on precision. The method is cheap next to full language model codes. It could make citation-based AI more reliable in fields like law.

Catalog record

CREATORRodrigo Kataishi, Ph.D
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE47 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-00891

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Topic-Enriched Embeddings to Improve Retrieval Precision in RAG Systems for LLMs Rodrigo Kataishi, Ph.D Abstract Retrieval-Augmented Generation (RAG) systems rely on document retrieval to ground large language model outputs, yet retrieval quality often degrades in corpora where topics overlap and relevant evidence is distributed across long, heterogeneous texts. This paper proposes topic-enriched embeddings, a hybrid representation that integrates term-frequency signals (TF-IDF), dimensionality-reduced semantic structure (LSA), and probabilistic topic mixtures (LDA) into a unified vector space anchored by contextual sentence embeddings ( all-MiniLM-L6-v2 ). The approach injects corpus-level thematic information into dense representations through two fusion strategies, concatenation and weighted averaging, while preserving computational tractability via latent-space compression. Empirical evaluation on a legal corpus of 12,436 documents related to Argentina’s Law 19.640 shows that topic enrichment improves both clustering coherence and retrieval effectiveness relative to statistical, probabilistic, and contextual-only baselines, with consistent gains in Precision@k, Recall@k, and F1. The results suggest that explicitly incorporating latent topic structure into embedding construction can reduce redundant or off-topic chunk retrieval, strengthening the evidential grounding of RAG pipelines in knowledge-intensive settings. Keywords: Semantic Analysis Natural Language Processing Machine Learning Computational Social Science Research Methodology 1 Introduction Retrieval-Augmented Generation (RAG) systems have enabled the ability of large language models (LLMs) to perform precise knowledge-intensive tasks by integrating external information through document ingestion ( 13 ; 31 ) . At the heart of these systems lies the retrieval mechanism, which determines the relevance of document chunks to user queries and uses that data as the context for LLM responses. Yet, retrieval precision continues to be a critical bottleneck, particularly when working with large datasets that exhibit heterogeneous topics and high thematic diversity ( 1 ) . These issues often lead to the retrieval of irrelevant or redundant chunks, undermining the precision and reliability of downstream generation tasks ( 8 ) . Despite recent advances in context window expansion, retrieval remains one of the most critical and underexplored challenges in modern LLMs ( 9 ) . Even with larger attention spans and extended context lengths, empirical studies have shown that retrieval systems often suffer from “lost in the middle” effects—where relevant information placed between less salient content is ignored or diluted—ultimately compromising model accuracy ( 17 ) . This phenomenon increases the risk of hallucinations and misaligned answers in RAG applications, particularly when dealing with dense regulatory or technical corpora. As a result, enhancing retrieval fidelity has become a priority for improving factual grounding and minimizing semantic drift in LLM-based pipelines. While high-performing open-source SOTA embedding models—such as nomic-embed-text , mxbai-embed-large , bge-m3 , snowflake-arctic-embed , and especially all-MiniLM-L6-v2 —have significantly advanced semantic retrieval, they tend to operate as stand-alone solutions. In most applications, they prioritize fine-grained semantic similarity, yet overlook thematic coherence and structural ordering across large, heterogeneous corpora. Nonetheless, recent work shows that traditional bag-of-words methods remain competitive across a wide range of classification challenges ( 10 ) , reinforcing our decision to retain TF-IDF in the embedding ensemble. This gap becomes particularly evident in domains that require precise retrievals—such as legal or regulatory texts—and where disambiguation of overlapping topics, hierarchical relationships among concepts, or deeper contextual grounding is essential. In this paper, we argue that a more robust solution lies not in replacing these models, but in reconfiguring them. The paper proposes a hybrid retrieval strategy that integrates the topic-level resolution capacity of statistical and probabilistic methods with the semantic depth of contextual embeddings, specifically leveraging the open-source all-MiniLM model. This layered architecture constitutes the methodological innovation of our approach and provides a functional enhancement to how RAG systems structure, rank, and retrieve complex textual content. This research proposes topic-enriched embeddings, a refined approach that integrates traditional statistical models and probabilistic topic modeling with modern contextual embeddings ( 6 ; 2 ) . This method captures both term-level and topic-level semantics, addressing the retrieval challenges posed by complex datasets ( 14 ) . By enriching embeddings with latent topic structures and leveraging dimensionality reduction, topic-enriched embeddings enable more precise document clustering and retrieval. Dimensionality reduction plays a pivotal role in this approach. Techniques such as LSA preserve critical semantic features while minimizing computational overhead ( 6 ) , ensuring that the method remains scalable for large datasets. This balance of precision and efficiency is particularly valuable for RAG systems, where effective retrieval underpins high-quality response generation ( 31 ) . The practical applicability of topic-enriched embeddings is demonstrated through empirical validation using a legal text dataset, and this approach significantly improves clustering coherence and retrieval precision ( 11 ) . Although the method introduces additional computational steps, the results justify the trade-off: accuracy in retrieval takes precedence over computational cost, as well-defined processing steps enhance the overall quality of the system. By reducing the retrieval of irrelevant or redundant chunks, it addresses core inefficiencies in existing systems, ensuring more accurate grounding for downstream generation tasks ( 15 ) . These results underscore the utility of topic-enriched embeddings in advancing knowledge-intensive RAG systems ( 8 ) . Additionally, the method emphasizes reproducibility and scalability. An open-source framework for embedding enrichment is provided, ensuring adaptability to diverse corpora and retrieval-intensive applications. This flexibility reinforces the relevance of the proposed approach across a wide range of domains, paving the way for future advancements in RAG technologies. The manuscript is organized as follows. Section 2 reviews prior work on topic modeling, embedding techniques, and retrieval-augmented generation (RAG). Section 3 introduces the proposed embedding enrichment method. Section 4 describes the dataset, preprocessing steps, baseline methods, and evaluation framework. Section 5 presents the experimental findings, while Section 6 examines their implications and limitations. Finally, Section 7 summarizes the main contributions and outlines directions for future research. 2 Related Work 2.1 Topic Modeling Approaches Traditional text analysis techniques, such as Term Frequency-Inverse Document Frequency (TF-IDF) ( 27 ) and Latent Semantic Analysis (LSA) ( 6 ) , offer computationally efficient ways to extract meaningful patterns from text. TF-IDF highlights term importance by balancing local and global frequency, while LSA reduces dimensionality to uncover latent semantic relationships. Probabilistic methods like Latent Dirichlet Allocation (LDA) ( 2 ) further enrich semantic representations by identifying document-topic distributions. For instance, 22 introduce a topic specificity metric to guide the choice of topic count and modeling method, and 16 apply contrastive learning to enhance hierarchical topic modeling structures. Despite their individual strengths, these techniques are rarely integrated systematically with modern embedding strategies in the context of RAG systems ( 11 ; 13 ) . Latent Semantic Analysis (LSA) is a dimensionality reduction technique that uses Singular Value Decomposition (SVD) to transform the high-dimensional term-document matrix into a lower-dimensional semantic space ( 6 ) . The decomposition is represented as: A=U\Sigma V^{T}, where A is the term-document matrix, U represents term-topic associations, \Sigma contains singular values, and V^{T} represents document-topic associations. By truncating \Sigma , LSA reduces noise while capturing the most significant semantic relationships. This approach is particularly effective for grouping documents by their underlying themes. Latent Dirichlet Allocation (LDA) , in contrast, adopts a probabilistic framework where each document is represented as a mixture of latent topics, and each topic is characterized as a distribution over terms ( 2 ) . The probability of a word

Read the complete text