LLM-42 : Enabling Determinism in LLM Inference with Verified Speculation
Raja Gond, Aditya K Kamath, Ramachandran Ramjee, and Ashish Panwar, [1ex] Microsoft Research · 2026
Read LLM-42 : Enabling Determinism in LLM Inference with Verified Speculation by Raja Gond, Aditya K Kamath, Ramachandran Ramjee, and Ashish Panwar, [1ex] Microsoft Research (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-17768.txt.
Description
Raja Gond, Aditya K Kamath, Ramachandran Ramjee, and Ashish Panwar of Microsoft Research study a quirk of large language models. The same prompt can give a different reply each time the model runs. Their 2026 preprint traces the cause to math inside the machine. Adding numbers in a changed order can change the final sum, and batch sizes shift that order.
One fix is to turn off dynamic batching, but that slows the system down a lot. The team instead works on a way to keep results stable while the speed stays high. Their title calls the method verified speculation. Steady output matters for tests, audits, and any use where the same input must give the same output.
Catalog record
| CREATOR | Raja Gond, Aditya K Kamath, Ramachandran Ramjee, and Ashish Panwar, [1ex] Microsoft Research |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 55 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-17768