Preserving Historical Truth: Detecting Historical Revisionism in Large Language Models

Francesco Ortu, Joeun Yook, Punya Syon Pandey, Keenan Samway, Bernhard Schölkopf, Alberto Cazzaniga, Rada Mihalcea, Zhijing Jin · 2026

Read Preserving Historical Truth: Detecting Historical Revisionism in Large Language Models by Francesco Ortu, Joeun Yook, Punya Syon Pandey, Keenan Samway, Bernhard Schölkopf, Alberto Cazzaniga, Rada Mihalcea, Zhijing Jin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-17433.txt.

Description

Francesco Ortu led an eight-author team auditing how language models handle contested history. Large language models increasingly serve as sources of historical information. That creates a need for scalable audits on contested events and politically charged narratives in real user chats. The team built HistoricalMisinfo: five hundred contested events from forty-five countries, each paired with a factual reference narrative and a documented revisionist one.

Each event appears in eleven prompt settings: questions, textbooks, social posts, and policy briefs. An LLM-as-a-judge protocol compares model outputs with the two references. Tests run under neutral prompts asking for accurate facts, and under robustness prompts where the user requests the revisionist version. Under neutral prompts models mostly track the factual reference, though scores signal reference alignment, not proof of revisionism. Under robustness prompts every model scored sharply higher on revisionism, showing limited resistance or self-correction.

Catalog record

CREATORFrancesco Ortu, Joeun Yook, Punya Syon Pandey, Keenan Samway, Bernhard Schölkopf, Alberto Cazzaniga, Rada Mihalcea, Zhijing Jin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-17433

Browse the full library · About the Tower of Babel