Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models
Qi Cao, Andrew Gambardella, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa · 2026
Read Semantic Token Clustering for Efficient Uncertainty Quantification in Large Language Models by Qi Cao, Andrew Gambardella, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-20161.txt.
Description
Cao and co-authors measure uncertainty in language model output by clustering tokens with like meanings. Most uncertainty methods resample many times or call helper models, which costs a lot. Semantic Token Clustering needs just one generation pass. It groups tokens by embedding similarity and shared prefixes. Uncertainty is then scored from the total probability mass of each meaning cluster. No auxiliary model is involved at any point. Results matched leading baselines on their tests. Compute overhead fell sharply compared with sampling-based rivals.
Catalog record
| CREATOR | Qi Cao, Andrew Gambardella, Takeshi Kojima, Yutaka Matsuo, Yusuke Iwasawa |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 40618 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.20161