Sample, Don’t Search: Rethinking Test-Time Alignment for Language Models

Gonçalo Faria, Noah A. Smith · 2025

Read Sample, Don’t Search: Rethinking Test-Time Alignment for Language Models by Gonçalo Faria, Noah A. Smith (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-03790.txt.

Description

Goncalo Faria and Noah Smith wrote a machine learning paper in two thousand twenty-five on improving chatbot answers at reply time. Spending more compute while generating can lift quality when retraining is impossible or the model is private. But search methods guided by a reward model often overfit that imperfect guide as compute grows.

Their method, QAlign, samples replies instead of searching for a single best one. As compute scales, its output converges to the ideal aligned distribution for each prompt. It uses sampling methods borrowed from statistics and needs no access to the model's internals. Tests on math word problems show steady gains over common rivals.

Catalog record

CREATORGonçalo Faria, Noah A. Smith
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE49 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-03790

Browse the full library · About the Tower of Babel