CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability

Han Peng, Jinhao Jiang, Zican Dong, Wayne Xin Zhao, Lei Fang · 2025

Read CAFE: Retrieval Head-based Coarse-to-Fine Information Seeking to Enhance Multi-Document QA Capability by Han Peng, Jinhao Jiang, Zican Dong, Wayne Xin Zhao, Lei Fang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-10063.txt.

Description

Han Peng and four co-authors help language models answer questions spread across many documents. Their paper appeared in two thousand twenty-five. Long inputs strain even models with huge context windows. Retrieval blurs and reasoning slips. Prompt tricks and retrieval heads help only partly. Precision and recall fight each other. Their CAFE method works in two passes to settle the fight.

A coarse pass ranks documents using the model's own retrieval heads. Background and distractors get cut early. A fine pass then steers attention to the strongest passages. Answers end up resting on real evidence. Experiments across question sets back the design. Multi-document QA gains a calmer, evidence-first reader.

Catalog record

CREATORHan Peng, Jinhao Jiang, Zican Dong, Wayne Xin Zhao, Lei Fang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-10063

Browse the full library · About the Tower of Babel