Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models

Wei-Ping, Chee-En, Guan-Ting, Hung-yi · 2026

Read Rethinking Entropy Minimization in Test-Time Adaptation for Autoregressive Models by Wei-Ping, Chee-En, Guan-Ting, Hung-yi (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-08186.txt.

Description

Wei-Ping Huang, Chee-En Yu, Guan-Ting Lin, and Hung-yi Lee unify a scattered practice: adapting generative speech and language models at test time by minimizing entropy. For classification the trick is well understood, but for autoregressive models, methods relied on separate heuristics such as teacher forcing with pseudo labels or policy-gradient reinforcement learning.

The authors derive the exact entropy objective for autoregressive models and show it splits into a token-level policy gradient term plus a token-level entropy term. Earlier methods appear as partial versions of that single formula. Using the Whisper speech recognizer as a testbed, the unified method improved results across more than twenty domains, including noise, accents, and multiple languages.

Catalog record

CREATORWei-Ping, Chee-En, Guan-Ting, Hung-yi
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE46 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.08186

Browse the full library · About the Tower of Babel