IterKey : Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation

Kazuki Hayashi † Hidetaka Kamigaito † Shinya Kouda ‡ Taro Watanabe † † Nara Institute of Science and Technology ‡ TDSE Inc. {hayashi.kazuki.hl4, kamigaito.h, taro}@is.naist.jp · 2025

Read IterKey : Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation by Kazuki Hayashi † Hidetaka Kamigaito † Shinya Kouda ‡ Taro Watanabe † † Nara Institute of Science and Technology ‡ TDSE Inc. {hayashi.kazuki.hl4, kamigaito.h, taro}@is.naist.jp (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-08450.txt.

Introduction

Kazuki Hayashi and co-authors present an information retrieval paper, published in 2025, making search based question answering both accurate and readable. Retrieval augmented generation fetches documents to ground a language model's answers. Dense search is accurate but opaque, while keyword search is transparent yet often misses the user's intent, and their framework, IterKey, aims for both.

A language model drafts keywords, searches with them, answers from what it finds, then checks its own answer. When the check fails, the loop repeats with sharper keywords. Across four question answering tasks the method beat plain keyword search clearly and matched dense search systems, while every step stays visible as plain keywords a reader can judge.

Catalog record

CREATORKazuki Hayashi † Hidetaka Kamigaito † Shinya Kouda ‡ Taro Watanabe † † Nara Institute of Science and Technology ‡ TDSE Inc. {hayashi.kazuki.hl4, kamigaito.h, taro}@is.naist.jp
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE56 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-08450

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

IterKey : Iterative Keyword Generation with LLMs for Enhanced Retrieval Augmented Generation Kazuki Hayashi † Hidetaka Kamigaito † Shinya Kouda ‡ Taro Watanabe † † Nara Institute of Science and Technology ‡ TDSE Inc. {hayashi.kazuki.hl4, kamigaito.h, taro}@is.naist.jp Abstract Retrieval Augmented Generation (RAG) has emerged as a way to complement the in-context knowledge of Large Language Models (LLMs) by integrating external documents. However, real-world applications demand not only accuracy but also interpretability. Dense retrieval methods provide high accuracy but lack interpretability, while sparse retrieval is transparent but often misses query intent due to keyword matching. Thus, balancing accuracy and interpretability remains a challenge. To address these issues, we introduce IterKey , an LLM-driven iterative keyword generation framework that enhances RAG via sparse retrieval. IterKey consists of three LLM-driven stages: generating keywords for retrieval, generating answers based on retrieved documents, and validating the answers. If validation fails, the process iteratively repeats with refined keywords. Across four QA tasks, experimental results show that IterKey achieves 5% to 20% accuracy improvements over BM25-based RAG and simple baselines. Its performance is comparable to dense retrieval based RAG and prior iterative query refinement methods using dense models. In summary, IterKey is a novel BM25-based iterative RAG framework that leverages LLMs to balance accuracy and interpretability. 1 Introduction Large Language Models (LLMs) ( OpenAI, 2023 ; Chiang et al., 2023 ; Dubey et al., 2024 ; Abdin et al., 2024 ) achieve strong performance in various natural language processing tasks but still struggle with issues such as hallucinations, outdated knowledge, and complex queries requiring multi-hop reasoning ( Kandpal et al., 2023 ; Zhang et al., 2023b ; Gao et al., 2024 ) . These issues are especially challenging in knowledge-intensive tasks ( Lee et al., 2019 ; Zellers et al., 2018 ) , where the necessary information may not be fully stored or readily retrieved within the model’s parameters. Retrieval Augmented Generation (RAG) improves the accuracy and relevance of generated responses by integrating external knowledge, making it particularly effective for open domain question answering ( Lewis et al., 2020 ; Izacard & Grave, 2021 ; Shuster et al., 2021 ; Ram et al., 2023 ; Izacard et al., 2024 ) . Although improvements to individual RAG components such as query expansion, document reranking, and retrieval tuning have been extensively studied ( Gao et al., 2024 ; Fan et al., 2024 ) , real-world applications demand both interpretability and accuracy from the retrieval component. Dense retrieval methods are highly accurate but suffer from limited interpretability, while sparse retrieval methods are more transparent but often struggle to capture the underlying intent of user queries ( Kang et al., 2025 ; Cheng et al., 2024 ; Ayoub et al., 2024 ; Llordes et al., 2023 ) . Figure 1: IterKey iteratively generates keywords, produces answers, and validates them using an LLM until the correct response is reached, thereby improving RAG accuracy. To address this, we propose IterKey : Iter ative Key word Generation with LLMs, a BM25-based method that improves RAG through iterative keyword refinement and self evaluation. IterKey operates in three stages: keyword generation, answer generation, and answer validation. In each cycle, an LLM extracts keywords, generates an answer, and checks correctness. If the validation fails, it updates the keywords and repeats the process until a satisfactory answer is obtained. By leveraging sparse BM25 retrieval, the contribution of each keyword to the score is explicit, enabling the tracking of generated keywords, retrieved documents, and validation results, which in turn improves accuracy and traceability. In experiments on four open domain QA datasets, IterKey improved retrieval performance by up to 20% over BM25 baselines and achieved 5% to 20% accuracy gains over vanilla methods and BM25-based RAG, with performance comparable to Dense model based RAG and iterative query refinement methods. These results indicate that LLMs can effectively infer and expand relevant keywords from queries, highlighting the practical potential of LLM-driven RAG refinement that balances performance and interpretability in supporting human search efforts. 2 IterKey : Iterative Keyword Generation with LLMs Sparse retrieval algorithms, such as BM25 ( Robertson & Zaragoza, 2009 ) , efficiently handle large datasets by ranking documents based on keyword frequency and uniqueness. They are interpretable, fast, and require no training, making them suitable for scalable or low resource settings. Their transparency also allows users to understand which keywords influenced the retrieval. However, they often fail to capture nuanced or implicit query intent, limiting their effectiveness compared to dense retrieval methods ( Llordes et al., 2023 ; Izacard et al., 2022 ; Chen et al., 2023 ; Wang et al., 2022b ) . To address this limitation, we propose IterKey, a method that extends query expansion research ( Wang et al., 2023a ; Wang et al., 2023b ; Gao et al., 2023 ) by leveraging LLMs to iteratively refine the retrieval process in RAG. Specifically, IterKey uses the self-validation capabilities of LLMs ( Wang et al., 2024 ) to improve sparse retrieval through iterative keyword generation, document retrieval, answer generation, and validation. By using BM25 for retrieval, all keywords that contribute to the search remain explicit, allowing the generated keywords, retrieved documents, and validation results to be inspected directly. This design aims to preserve the interpretability of sparse retrieval while improving accuracy. Table 1 summarizes the prompts used in each step of IterKey . We further explain the motivation behind each prompt design, with concrete examples provided in Figure 1 to illustrate the full process. Step Prompt Step 1: Initial Keyword Generation System: You are an assistant that generates keywords for information retrieval. User: Generate a list of important keywords related to the Query: { q } . Focus on keywords that are relevant and likely to appear in documents for BM25 search in the RAG framework. Output the keywords as: ["keyword1", "keyword2", "keyword3", …]. Separate each keyword with a comma and do not include any additional text. Step 2: Answer Generation using RAG System: You are an assistant that generates answers based on retrieved documents. User: Here is a question that you need to answer: Query: { q } Below are some documents that may contain information relevant to the question. Consider the information in these documents while combining it with your own knowledge to answer the question accurately. Documents: { D } Provide a clear and concise answer. Do not include any additional text. Step 3: Answer Validation System: You are an assistant that validates whether the provided answer is correct. User: Is the following answer correct? Query: { q } Answer: { a } Previous Retrieval Documents: { Docs } Respond ’True’ or ’False’. Do not provide any additional explanation or text. Step 4: Keyword Regeneration System: You refine keywords to improve document retrieval for BM25 search in the RAG framework. User: Refine the keyword selection process to improve the accuracy of retrieving documents with the correct answer. Query: { q } Previous Keywords: { K } Provide the refined list of keywords in this format: ["keyword1", "keyword2", …]. Separate each keyword with a comma and do not include any additional text. Table 1: IterKey prompts for iterative keyword refinement and answer validation. Step 1: Keyword Generation Given a user query q , the LLM

Read the complete text