Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation

Hongxiang Zhang, Hao Chen, Muhao Chen, Tianyi Zhang · 2025

Read Active Layer-Contrastive Decoding Reduces Hallucination in Large Language Model Generation by Hongxiang Zhang, Hao Chen, Muhao Chen, Tianyi Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-23657.txt.

Description

Hongxiang Zhang and three co-authors present a decoding method that cuts made-up content in language model answers. Their method appeared in twenty twenty-five. Even strong models invent facts, above all over long passages, and earlier fixes mostly worked one word at a time.

Their approach, called ActLCD, contrasts signals from different layers of the model while it writes. The twist is that it actively chooses when to switch that contrast on. A learning policy, guided by a reward-aware judge, treats each choice as a step in a sequence. Across five test sets, the method beat the best known rivals. The authors argue that timing the fix, not just applying it, is what lifts factual accuracy.

Catalog record

CREATORHongxiang Zhang, Hao Chen, Muhao Chen, Tianyi Zhang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-23657

Browse the full library · About the Tower of Babel