Sample, Don’t Search: Rethinking Test-Time Alignment for Language Models
Gonçalo Faria, Noah A. Smith · 2025
Read Sample, Don’t Search: Rethinking Test-Time Alignment for Language Models by Gonçalo Faria, Noah A. Smith (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03790.txt.
Description
Goncalo Faria and Noah Smith wrote a machine learning paper in two thousand twenty-five on improving chatbot answers at reply time. Spending more compute while generating can lift quality when retraining is impossible or the model is private. But search methods guided by a reward model often overfit that imperfect guide as compute grows.
Their method, QAlign, samples replies instead of searching for a single best one. As compute scales, its output converges to the ideal aligned distribution for each prompt. It uses sampling methods borrowed from statistics and needs no access to the model's internals. Tests on math word problems show steady gains over common rivals.
Catalog record
| CREATOR | Gonçalo Faria, Noah A. Smith |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 49 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-03790