The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning
Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng · 2025
Read The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning by Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15134.txt.
Description
Shivam Agarwal and co-authors find that a push to be sure lifts word tools for free in a 2025 paper. The push is plain: train the tool to heap its odds on the calls it holds most sure, with no told marks at all.
Three forms of the push are tried. One trains on the tool's own raw takes, as school with no keys. One pays the tool by a prize of sure-ness lone. One bends the tool's calls at the ask, with no train at all. On a mid tool the prize form matched prize ways schooled on scores of told sums. The ask form let a big tool match paid kin tools on a hard code test, at a third of the sums of the slow ways.
Catalog record
| CREATOR | Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-15134