LLMs Encode How Difficult Problems Are
William Lugoloobi, Chris Russell · 2025
Read LLMs Encode How Difficult Problems Are by William Lugoloobi, Chris Russell (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-18147.txt.
Description
William Lugoloobi and Chris Russell probe whether language models know, inside, how hard a problem is. Models often solve tough questions yet trip on easy ones, which makes their sense of difficulty worth checking. Linear probes read difficulty signals across layers in sixty models, scored on math and coding sets with human labels.
Human-rated difficulty decoded strongly and grew with model size. Model-derived difficulty decoded weakly and scaled poorly. Steering a model toward feeling a problem as easier cut made-up answers and raised accuracy. During one reinforcement run, the human-difficulty signal strengthened alongside test scores. Interpretability readers get a usable internal gauge.
Catalog record
| CREATOR | William Lugoloobi, Chris Russell |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2510-18147