Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models
Wei-Ping, Chee-En, Guan-Ting, Hung-yi · 2026
Read Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models by Wei-Ping, Chee-En, Guan-Ting, Hung-yi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08186.txt.
Description
Wei-Ping Huang, Chee-En Yu, Guan-Ting Lin, and Hung-yi Lee unify a scattered practice: adapting generative speech and language models at test time by minimizing entropy. For classification the trick is well understood, but for autoregressive models, methods relied on separate heuristics such as teacher forcing with pseudo labels or policy-gradient reinforcement learning.
The authors derive the exact entropy objective for autoregressive models and show it splits into a token-level policy gradient term plus a token-level entropy term. Earlier methods appear as partial versions of that single formula. Using the Whisper speech recognizer as a testbed, the unified method improved results across more than twenty domains, including noise, accents, and multiple languages.
Catalog record
| CREATOR | Wei-Ping, Chee-En, Guan-Ting, Hung-yi |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.08186