HALT: Hallucination Assessment via Log-probs as Time series

Ahmad Shapiro, Karan Taneja, Ashok Goel · 2026

Read HALT: Hallucination Assessment via Log-probs as Time series by Ahmad Shapiro, Karan Taneja, Ashok Goel (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-02888.txt.

Description

Spotting made-up answers from language models is the aim of Ahmad Shapiro and 2 co-authors in a 2026 paper. Their detector, HALT, reads only the top token probabilities produced during generation, treated as a time series. It needs no access to hidden states, attention maps, or model weights.

A small recurrent model plus entropy-based features learns each model's calibration bias. The authors also built HUB, a unified benchmark covering ten abilities from math to summarization. HALT beat a fine-tuned encoder baseline while being far smaller. It also ran many times faster. Because it uses only output probabilities, HALT can work with closed commercial models too.

Catalog record

CREATORAhmad Shapiro, Karan Taneja, Ashok Goel
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602-02888

Browse the full library · About the Tower of Babel