Knowledge Retrieval Based on Generative AI
1 st Te-Lun Yang, 2 nd Jyi-Shane Liu, 3 rd Yuen-Hsien Tseng, 4 th Jyh-Shing Roger Jang · 2025
Read Knowledge Retrieval Based on Generative AI by 1 st Te-Lun Yang, 2 nd Jyi-Shane Liu, 3 rd Yuen-Hsien Tseng, 4 th Jyh-Shing Roger Jang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-04635.txt.
Introduction
Te-Lun Yang and co-authors present a 2025 paper on a question-answering system built on retrieval and generation. The system draws facts from Chinese Wikipedia and a legal database called Lawbank. A dense retriever model, BGE-M3, finds passages close in meaning to each question. A reranker then orders those passages by true relevance. The best passages feed a large language model as reference knowledge. Two question sets test the system, one general and one exam-style.
Scoring runs in two stages: fully automatic checks, and assisted scoring with human help on finance-related items. Information retrieval readers get a worked pipeline where retrieval quality visibly drives answer quality. Grounding answers in retrieved law and encyclopedia text fights the confident invention language models are known for. The legal source adds a high-stakes domain where invented answers carry real cost.
Catalog record
| CREATOR | 1 st Te-Lun Yang, 2 nd Jyi-Shane Liu, 3 rd Yuen-Hsien Tseng, 4 th Jyh-Shing Roger Jang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 28 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2501-04635
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Knowledge Retrieval Based on Generative AI 1 st Te-Lun Yang, 2 nd Jyi-Shane Liu, 3 rd Yuen-Hsien Tseng, 4 th Jyh-Shing Roger Jang Abstract This study develops a question-answering system based on Retrieval-Augmented Generation (RAG) using Chinese Wikipedia and Lawbank as retrieval sources. Using TTQA and TMMLU+ as evaluation datasets, the system employs BGE-M3 for dense vector retrieval to obtain highly relevant search results and BGE-reranker to reorder these results based on query relevance. The most pertinent retrieval outcomes serve as reference knowledge for a Large Language Model (LLM), enhancing its ability to answer questions and establishing a knowledge retrieval system grounded in generative AI. The system’s effectiveness is assessed through a two-stage evaluation: automatic and assisted performance evaluations. The automatic evaluation calculates accuracy by comparing the model’s auto-generated labels with ground truth answers, measuring performance under standardized conditions without human intervention. The assisted performance evaluation involves 20 finance-related multiple-choice questions answered by 20 participants without financial backgrounds. Initially, participants answer independently. Later, they receive system-generated reference information to assist in answering, examining whether the system improves accuracy when assistance is provided. The main contributions of this research are: (1) Enhanced LLM Capability: By integrating BGE-M3 and BGE-reranker, the system retrieves and reorders highly relevant results, reduces hallucinations, and dynamically accesses authorized or public knowledge sources. (2) Improved Data Privacy: A customized RAG architecture enables local operation of the LLM, eliminating the need to send private data to external servers. This approach enhances data security, reduces reliance on commercial services, lowers operational costs, and mitigates privacy risks. 1 1 footnotetext: To whom all correspondence should be addressed. E-mail: [email protected] I Introduction The field of Information Retrieval (IR) has evolved significantly, driven by the need for real-time information through systems like Google and Bing. These systems use keyword-based matching and ranking algorithms to provide relevant results. However, traditional IR systems have limitations, such as low relevance or excessively lengthy outputs when query terms differ from indexed terms. This increases the effort required to find the desired information. The rise of Generative AI and Large Language Models (LLMs), such as OpenAI’s ChatGPT [ 1 ] and Google’s Gemini, offers new ways to enhance IR systems by better understanding user intent and generating natural language responses. Despite these advances, LLMs face challenges, including generating false or outdated information when training data is insufficient or obsolete. Additionally, they may struggle with imbalanced training data, leading to incorrect outputs in specialized domains. To address these issues, the Retrieval-Augmented Generation (RAG) framework combines IR and LLMs, allowing LLMs to retrieve specific information and generate accurate answers without retraining. Dense vector retrieval, a key method in this framework, uses deep learning to map text to high-dimensional vectors, improving accuracy by capturing semantic similarity. This approach is crucial for the development of next-generation knowledge retrieval, overcoming the limitations of traditional methods while leveraging the strengths of LLMs. Figure 1 showed basic flow for this study. Fig. 1: Basic Flow for this study II Related Work II-A Information Retrieval Information retrieval (IR) has progressed from basic textual data processing to advanced deep learning techniques, with natural language processing (NLP) playing a key role. Early models, such as the Boolean Retrieval Model, used logical operators (OR, AND, NOT) for document-query matching but struggled with complex queries. The Vector Space Model improved upon this by vectorizing documents and queries, using similarity measures, while enhancements like TF-IDF increased retrieval accuracy by adjusting term importance, though lacking contextual understanding. Probabilistic Retrieval Models ranked documents based on relevance probabilities but required large datasets for accuracy. BM25 further refined this approach by dynamically adjusting term weights, and Latent Dirichlet Allocation (LDA) introduced topic modeling using Bayesian networks. The advent of GPUs and deep learning has popularized Dense Vector Retrieval, which converts text into high-dimensional vectors. Models like BERT and Sentence-BERT leverage these vectors to perform tasks such as sentence classification efficiently, enhancing the semantic understanding of the text and improving IR systems’ accuracy and efficiency. II-B Language Model Language models (LMs) have progressed from Statistical Language Models (SLMs) to Neural Network Models (NNMs), Pre-trained Models, and Large Language Models (LLMs). Early models like N-grams and Markov Models, though foundational, struggled with scalability due to the ”curse of dimensionality.” This limitation led to the adoption of NNMs, such as Recurrent Neural Networks (RNNs), which addressed variable-length sequences but faced challenges with long-term dependencies, later mitigated by Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRUs). The introduction of Distributed Word Representations (DWR), like word2vec, represented a major advancement by mapping words to dense vectors, though it lacked context sensitivity. Pre-trained models, as Figure 2 showed, such as BERT [ 2 ] and Sentence-BERT [ 3 ] , resolved this issue with self-attention mechanisms, allowing context-aware embeddings. The emergence of large language models (LLMs), including GPT-3 and PaLM, has revolutionized NLP by leveraging billions of parameters and exhibiting emergent abilities. These models enable few-shot learning and advance natural language understanding. Fig. 2: Bi-Encoder and Cross-Encoder II-C Retrieval-Augmented Generation Retrieval-Augmented Generation (RAG) [ 4 ] overcomes the limitations of Large Language Models (LLMs), which rely on static, pre-trained datasets that can become outdated and lack domain-specific information. This restricts LLMs’ ability to generate accurate and up-to-date responses. RAG integrates Information Retrieval (IR) systems with LLMs, enabling them to query external knowledge sources and access real-time, domain-relevant data. In a typical RAG framework, a retriever processes user queries and retrieves relevant documents based on semantic similarity. These documents are then combined with the original query and passed to the LLM to generate a more accurate and comprehensive response. RAG also addresses the issue of model obsolescence by dynamically accessing updated information without retraining. However, balancing precision and recall during retrieval can be challenging, and re-ranking is required to prioritize relevant information, ensuring that LLM responses are accurate and contextually appropriate in complex queries. II-D Evaluation The rise of Large Language Models (LLMs) necessitates robust evaluation methods across various tasks. Key evaluation metrics include classification, language modeling, text generation, and question answering. For classification, metrics such as precision, recall, and F1-Score are derived from a confusion matrix that assesses true and false positives and negatives. Language modeling is evaluated using perplexity (PPL), which measures a model’s ability to predict token sequences, with lower PPL values indicating stronger performance. Text generation tasks, including machine translation and summarization, use BLEU for precision, analyzing n-gram overlap, and ROUGE for recall, focusing on the longest common subsequences. Question answering is typically evaluated through accuracy, with Exact Match (EM) and F1-Score commonly applied. For knowledge-based tasks, the MMLU [ 5 ] dataset, which contains multiple-choice questions from 57 domains, is used to assess LLMs’ generalization and knowledge application capabilities across diverse subjects. These metrics are crucial for evaluating LLM performance across different domains. III Approach III-A Embedding Model The BGE-M3 [ 6 ] model is a highly versatile embedding model supporting multilinguality, multifunctionality, and multi-granularity. Capable of converting text into vectors, it supports over 100 languages for cross-lingual retrieval and is designed to handle sentence, passage, and document-level inputs, with a sequence length of up to 8,192 tokens. It enables dense, sparse, and multi-vector retrieval, making it suitable for a range of tasks from basic