ConSol: Sequential Probability Ratio Testing to Find Consistent LLM Reasoning Paths Efficiently

Jaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, Hyun-Hwan Jeong · 2025

Read ConSol: Sequential Probability Ratio Testing to Find Consistent LLM Reasoning Paths Efficiently by Jaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, Hyun-Hwan Jeong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17587.txt.

Description

Sampling many reasoning paths and taking the majority answer makes language models more reliable, but it burns tokens when the first few paths already agree. Jaeyeon Lee and co-authors import a classic statistics tool, the sequential probability ratio test, to stop the sampling early. Their system, ConSol, watches the answers as they arrive and halts once consistency is statistically convincing.

The test's knobs are tuned for this job, especially for spotting the leading answer reliably. Across their experiments, early stopping matched the accuracy of full self-consistency at a fraction of the cost. That saving matters in money and in energy. The authors release the code as a public package so other teams can bolt the brake onto their own samplers.

Catalog record

CREATORJaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, Hyun-Hwan Jeong
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE38 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-17587

Browse the full library · About the Tower of Babel