Inference Time Causal Probing in LLMs

Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Matthias Grossglauser · 2026

Read Inference Time Causal Probing in LLMs by Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Matthias Grossglauser (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07631.txt.

Description

Sadegh Khorasani and three co-authors improve causal probing, a family of methods that test how a language model's internal states drive its behavior by intervening on them. Earlier interventions needed a separately trained probe classifier, which ties the method to one task and can clash with the model's own geometry.

Their method, HDMI, skips probes entirely: a gradient-based margin objective steers hidden states directly, raising the chance of a target continuation while lowering the source one. A lookahead variant supports text editing while preserving fluency. Judged by completeness and selectivity on two benchmarks and two model families, HDMI achieved higher overall reliability than prior probing methods.

Catalog record

CREATORSadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Matthias Grossglauser
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE53 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.07631

Browse the full library · About the Tower of Babel