Confidence Elicitation: A New Attack Vector for Large Language Models
Brian Formento, Chuan Sheng Foo, See-Kiong Ng · 2025
Read Confidence Elicitation: A New Attack Vector for Large Language Models by Brian Formento, Chuan Sheng Foo, See-Kiong Ng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-04643.txt.
Description
Brian Formento, Chuan Sheng Foo, and See-Kiong Ng show a new way to attack large language models in a 2025 paper: ask them how confident they are. Modern closed models reveal nothing but their final text, so classic attacks that read internal scores no longer work. Attackers can only see the final label. The team test whether a model's stated confidence can stand in for those hidden scores.
Across current models, the stated confidence proved well calibrated rather than made up. Lowering that confidence by tweaking the input raised the chance of a wrong classification. Their method, which swaps words to push confidence down, beat existing black box attack methods on three data sets and two open models. The work names confidence elicitation as an attack vector that defenders of closed systems should weigh.
Catalog record
| CREATOR | Brian Formento, Chuan Sheng Foo, See-Kiong Ng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-04643