The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning

Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng · 2025

Read The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning by Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-15134.txt.

Description

Shivam Agarwal and co-authors find that a push to be sure lifts word tools for free in a 2025 paper. The push is plain: train the tool to heap its odds on the calls it holds most sure, with no told marks at all.

Three forms of the push are tried. One trains on the tool's own raw takes, as school with no keys. One pays the tool by a prize of sure-ness lone. One bends the tool's calls at the ask, with no train at all. On a mid tool the prize form matched prize ways schooled on scores of told sums. The ask form let a big tool match paid kin tools on a hard code test, at a third of the sums of the slow ways.

Catalog record

CREATORShivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han, Hao Peng
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE64 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-15134

Browse the full library · About the Tower of Babel