Rectifying Belief Space via Unlearning to Harness LLMs’ Reasoning

Ayana Niwa, Masahiro Kaneko, Kentaro Inui · 2025

Read Rectifying Belief Space via Unlearning to Harness LLMs’ Reasoning by Ayana Niwa, Masahiro Kaneko, Kentaro Inui (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-20620.txt.

Description

Ayana Niwa, Masahiro Kaneko, and Kentaro Inui fix wrong answers from language models by editing what the models believe. Their paper is from two thousand twenty-five. Models can reason well yet still land on false answers. The authors blame hidden false beliefs: claims the model treats as true inside, which quietly steer its reasoning off course.

Their method first asks the model to explain its answers, hunting for the beliefs behind right and wrong replies. A beam search works forward and backward to surface those beliefs. Then a forgetting technique dims the false beliefs while boosting the true ones. On several question sets, past mistakes were corrected without hurting overall scores. The fix also carried over to unseen data, which hints at a general cure.

Catalog record

CREATORAyana Niwa, Masahiro Kaneko, Kentaro Inui
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-20620

Browse the full library · About the Tower of Babel