Not All Needles Are Found: How Fact Distribution and Don’t Make It Up Prompts Shape Literal Extraction, Logical Inference, and Hallucination Risks in Long-Context LLMs

Amirali Ebrahimzadeh Department of Electrical Engineering & Computer Science University of Michigan, Ann Arbor, MI 48109, USA [email protected] Seyyed M. Salili ∗ Independent Researcher Ann Arbor, MI 48106, USA [email protected] · 2026

Read Not All Needles Are Found: How Fact Distribution and Don’t Make It Up Prompts Shape Literal Extraction, Logical Inference, and Hallucination Risks in Long-Context LLMs by Amirali Ebrahimzadeh Department of Electrical Engineering & Computer Science University of Michigan, Ann Arbor, MI 48109, USA [email protected] Seyyed M. Salili ∗ Independent Researcher Ann Arbor, MI 48106, USA [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-02023.txt.

Description

Long-context language models are being trusted as working memory for autonomous tasks, yet their reliability depends on where the facts sit. Amirali Ebrahimzadeh and Seyyed Salili build an extended needle-in-a-haystack benchmark that separates literal extraction, logical inference, and hallucination, and apply it to four current models.

Two failure modes stand out. Distributional collapse: performance degrades sharply when evidence is scattered across the context instead of concentrated. And a safety tax: prompts telling the model not to make things up cause over-conservative refusals of facts that are actually present, sharply cutting accuracy. The authors blame poor context utilization more than missing knowledge, and call for model-specific robustness work before long-horizon deployment.

Catalog record

CREATORAmirali Ebrahimzadeh Department of Electrical Engineering & Computer Science University of Michigan, Ann Arbor, MI 48109, USA [email protected] Seyyed M. Salili ∗ Independent Researcher Ann Arbor, MI 48106, USA [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-02023

Browse the full library · About the Tower of Babel