Knowing You Don’t Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy. Proceedings of the 48th International ACM S
Diji Yang, Linda Zeng, Jinmeng Rao, Yi Zhang · 2025
Read Knowing You Don’t Know: Learning When to Continue Search in Multi-round RAG through Self-Practicing Conference: Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 13–18, 2025; Padua, Italy. Proceedings of the 48th International ACM S by Diji Yang, Linda Zeng, Jinmeng Rao, Yi Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02811.txt.
Description
Retrieval augmented generation lets a language model look facts up before answering, and hard questions need several rounds of searching. Diji Yang and co-authors note that such systems judge their own progress poorly: they keep searching when they know enough, or answer boldly when they do not. Their framework, SIM-RAG, teaches a system to know when it does not know. First the system practices alone, trying many search paths for known questions and noting which paths reach the right answer.
From that practice data they train a small critic model that judges, at each round, whether gathered information suffices. During real use the critic tells the system to stop or to search again. Tests on several benchmarks show better answers with this self check in place. The method adds a light component and needs no costly human labels for the middle steps of searching.
Catalog record
| CREATOR | Diji Yang, Linda Zeng, Jinmeng Rao, Yi Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 52 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-02811