Estimating Tail Risks in Language Model Output Distributions
Rico Angell, Raghav Singhal, Zachary Horvitz, Zhou Yu, Rajesh Ranganath, Kathleen McKeown, He He · 2026
Read Estimating Tail Risks in Language Model Output Distributions by Rico Angell, Raghav Singhal, Zachary Horvitz, Zhou Yu, Rajesh Ranganath, Kathleen McKeown, He He (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-22167.txt.
Description
Angell and six colleagues propose a sample-efficient way to estimate harmful-output probabilities for any query. They build unsafe model versions, making harmful outputs more probable through importance sampling. On misuse and misalignment benchmarks, their estimates match Monte Carlo estimates using 10 to 20 times fewer samples. A probability of 10^-4 can be estimated with just 500 samples. Harm scores also reveal input response and predict deployment risks, showing rare-event estimation is critical and feasible for safety evaluations.
Catalog record
| CREATOR | Rico Angell, Raghav Singhal, Zachary Horvitz, Zhou Yu, Rajesh Ranganath, Kathleen McKeown, He He |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66997 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.22167