ConSol: Sequential Probability Ratio Testing to Find Consistent LLM Reasoning Paths Efficiently
Jaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, Hyun-Hwan Jeong · 2025
Read ConSol: Sequential Probability Ratio Testing to Find Consistent LLM Reasoning Paths Efficiently by Jaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, Hyun-Hwan Jeong (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-17587.txt.
Description
Sampling many reasoning paths and taking the majority answer makes language models more reliable, but it burns tokens when the first few paths already agree. Jaeyeon Lee and co-authors import a classic statistics tool, the sequential probability ratio test, to stop the sampling early. Their system, ConSol, watches the answers as they arrive and halts once consistency is statistically convincing.
The test's knobs are tuned for this job, especially for spotting the leading answer reliably. Across their experiments, early stopping matched the accuracy of full self-consistency at a fraction of the cost. That saving matters in money and in energy. The authors release the code as a public package so other teams can bolt the brake onto their own samplers.
Catalog record
| CREATOR | Jaeyeon Lee, Guantong Qi, Matthew Brady Neeley, Zhandong Liu, Hyun-Hwan Jeong |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 38 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-17587