Inference Time Causal Probing in LLMs
Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Matthias Grossglauser · 2026
Read Inference Time Causal Probing in LLMs by Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Matthias Grossglauser (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-07631.txt.
Description
Sadegh Khorasani and three co-authors improve causal probing, a family of methods that test how a language model's internal states drive its behavior by intervening on them. Earlier interventions needed a separately trained probe classifier, which ties the method to one task and can clash with the model's own geometry.
Their method, HDMI, skips probes entirely: a gradient-based margin objective steers hidden states directly, raising the chance of a target continuation while lowering the source one. A lookahead variant supports text editing while preserving fluency. Judged by completeness and selectivity on two benchmarks and two model families, HDMI achieved higher overall reliability than prior probing methods.
Catalog record
| CREATOR | Sadegh Khorasani, Saber Salehkaleybar, Negar Kiyavash, Matthias Grossglauser |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 53 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.07631