GOLFer: Smaller LM-Generated Documents Hallucination Filter & Combiner for Query Expansion in Information Retrieval

Lingyuan Liu, Mengxiang Zhang · 2025

Read GOLFer: Smaller LM-Generated Documents Hallucination Filter & Combiner for Query Expansion in Information Retrieval by Lingyuan Liu, Mengxiang Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-04762.txt.

Introduction

Lingyuan Liu and Mengxiang Zhang expand search queries with documents dreamed up by small language models, filtered for lies by GOLFer. Their paper from twenty twenty-five addresses a cost trap: rival methods need huge models to write trustworthy hypothetical documents. Small open models hallucinate freely, which poisons the expansion. GOLFer splits the fix into two modules.

A hallucination filter strips non-factual and inconsistent sentences from the drafts. A combiner then merges the cleaned text with the query, weighted to balance their pull. Tests span three web search sets and ten low-resource ones. GOLFer beats other small-model methods throughout and stays competitive with large-model rivals. Cheap models, carefully policed, can still sharpen retrieval.

Catalog record

CREATORLingyuan Liu, Mengxiang Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE32 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-04762

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GOLFer: Smaller LM-Generated Documents Hallucination Filter & Combiner for Query Expansion in Information Retrieval Lingyuan Liu, Mengxiang Zhang Abstract Large language models (LLMs)-based query expansion for information retrieval augments queries with generated hypothetical documents with LLMs. However, its performance relies heavily on the scale of the language models (LMs), necessitating larger, more advanced LLMs. This approach is costly, computationally intensive, and often has limited accessibility. To address these limitations, we introduce GOLFer - Smaller LMs- G enerated D o cuments Ha l lucination F ilter & Combin er - a novel method leveraging smaller open-source LMs for query expansion. GOLFer comprises two modules: a hallucination filter and a documents combiner. The former detects and removes non-factual and inconsistent sentences in generated documents, a common issue with smaller LMs, while the latter combines the filtered content with the query using a weight vector to balance their influence. We evaluate GOLFer alongside dominant LLM-based query expansion methods on three web search and ten low-resource datasets. Experimental results demonstrate that GOLFer consistently outperforms other methods using smaller LMs, and maintains competitive performance against methods using large-size LLMs, demonstrating its effectiveness. † † footnotetext: Corresponding author. * * * The code for our method is publicly available at https://github.com/liuliuyuan6/GOLFer . 1 Introduction Information retrieval (IR) is crucial for extracting relevant information from large repositories, serving as a key component in modern search engines [ 27 , 15 ] . Query expansion, a key technique for enhancing IR performance, improves the precision and expressiveness of user queries [ 1 ] . Traditional methods use hand-built knowledge resources like WordNet and Thesaurus ( [ 21 , 11 ] or external text collections [ 24 , 6 ] . However, these methods are limited by the quality of external data sources and show limited success on popular datasets [ 1 ] . More adaptable query expansion approaches are needed to meet diverse requirements across different contexts. Figure 1: Overview of GOLFer. Given a query, GOLFer generates n passages using an Instruct-smaller LM, which are then processed through a hallucination filter to produce filtered hypothetical documents. These filtered documents are combined with the original query using a weight vector by the documents combiner module to create the expanded query embedding for retrieval. LLMs like GPT-4 [ 20 ] and LLaMA 3 [ 7 ] have shown impressive abilities in generating fluent and realistic responses. Pre-trained on extensive corpora, these models excel in natural language understanding and generation. [ 20 ] indicates that LLMs can be fine-tuned with minimal data to align with human intent, enabling them to generalize to diverse instructions in a zero-shot manner. This adaptability has spurred interest in using LLMs for query expansion in IR, where queries are often brief or ambiguous [ 19 , 33 ] . LLMs can generate hypothetical documents based on various prompts to enhance query expansion. For example, HyDE uses zero-shot instructions to generate a hypothetical document [ 10 ] , and Query2Doc employs few-shot examples to create hypothetical-documents [ 29 ] . These methods enhance the performance of retrievers such as Contriever and BM25 across a variety of tasks, including web search, question answering, and fact verification. However, existing LLM-based query expansion methods face several critical challenges. Empirical experiments, such as those involving HyDE, indicate that the performance of LLM-based query expansion heavily depends on the scale of the LLM employed [ 10 ] . [ 29 ] suggest that smaller LMs tend to produce shorter outputs with more factual errors, posing a significant obstacle to building trustworthy systems. This tendency to hallucinate facts has been widely observed, where models can confidently generate fictitious information. Consequently, current LLM-based query expansion methods often advocate for using large-scale, advanced LLMs like GPT-3.5 (175B) and GPT-4 to mitigate these inaccuracies and enhance query expansion performance in IR. However, in practice, employing such large-scale models is costly, computationally intensive, and regionally restricted. For instance, the API costs for GPT-3.5 and GPT-4 are $1.50 and $2.50 per million tokens, respectively † † † Refer to https://openai.com/api/pricing/ . , making widespread use financially prohibitive for small and private institutions. Additionally, quota limits often restrict the practical application of these models, and their token-by-token autoregressive decoding can significantly reduce retrieval efficiency, potentially taking over 2000 ms to generate hypothetical documents [ 29 ] . Furthermore, access to large-scale LLMs like GPT is restricted in certain regions, such as China, further limiting their applicability. These factors collectively constrain the practical deployment of existing LLM-based query expansion methods. To overcome these limitations, we introduce a new method, GOLFer, i.e., smaller LMs- g enerated d o cuments ha l lucination f ilter & combin er for query expansion in IR (See Fig. 1 ). Unlike existing LLM-based query expansion methods, our method focuses on smaller open-source LMs, such as LLaMA-3-8B-Instruct, to assist in query expansion, thereby mitigating some of the limitations associated with using large-scale models. However, smaller LMs often introduce hallucinations, such as non-factual content and inconsistent contexts, when generating hypothetical documents. If these flawed documents are directly utilized by retrievers, they can introduce irrelevant, noisy, or erroneous data, jeopardizing the quality of the expanded query embeddings and thus affecting query expansion performance in IR. To address this issue, GOLFer is designed to detect and filter out non-factual and inconsistent sentences from the hypothetical documents, ensuring that only relevant and accurate information is incorporated. Additionally, to balance the influence of the query and hypothetical documents, our method combines these filtered documents with the original query based on factuality by setting a weight vector. The inner product of the weight vector and the embedding vectors for the original query and filtered documents forms a new, refined query for IR. GOLFer is a lightweight, versatile query expansion method that can be integrated into any Transformer-based LM without requiring additional training, fine-tuning, or prompt engineering. We comprehensively evaluate GOLFer using LLaMA-3-8B-Instruct as the smaller LM, alongside three types of retrievers: sparse retriever (e.g., BM25), dense retriever (e.g., ANCE [ 31 ] ), and advanced dense retriever (e.g., Aggretriever_v1 [ 17 ] ). Under consistent experimental conditions, we compare GOLFer with existing LLM-based query expansion methods, including HyDE [ 10 ] and query2doc [ 29 ] , across three web search datasets from MS MARCO [ 3 ] and ten low-resource datasets from the BEIR benchmark [ 26 ] . The results demonstrate that GOLFer outperforms other methods using smaller LMs on all datasets across nearly all evaluation metrics, highlighting its effectiveness. Notably, GOLFer remains competitive even against methods utilizing large-scale LLMs like GPT-3.5 (175B). In summary, the contributions of our paper are as follows: • We propose a novel smaller LMs-driven query expansion method: GOLFer. It can i) detect and filter out non-factual and inconsistent sentences in the original hypothetical documents generated by smaller LMs, and ii) combine these filtered hypothetical documents with the original query by a weight vector to form a expanded query for IR, enhancing factuality. • We evaluate GOLFer alongside dominant query expansion methods on MS MARCO dev, TREC DL19 and DL20 and nine low-resource datasets form BEIR. Experimental results demonstrate the efficacy of our method. 2 Related Work 2.1 LLM-based Query Expansion Query expansion improves retrieval

Read the complete text