BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation

Ankur Samanta, Akshayaa Magesh, Tal Lancewicki, Ayush Jain, Youliang Yu, Paul Sajda, Kaveh Hassani, Aditya Modi, Daniel R. Jiang, Yonathan Efroni · 2026

Read BayesBench: Evaluating LLM Belief Trajectories Under Multi-Turn Evidence Accumulation by Ankur Samanta, Akshayaa Magesh, Tal Lancewicki, Ayush Jain, Youliang Yu, Paul Sajda, Kaveh Hassani, Aditya Modi, Daniel R. Jiang, Yonathan Efroni (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-30850.txt.

Introduction

Ankur Samanta and nine colleagues introduce BayesBench, a benchmark for how language models update beliefs as evidence arrives turn by turn. Rational behaviour in a long conversation means inferring hidden quantities. It also means revising beliefs as each new observation lands. Most tests score only a final answer, given everything at once. BayesBench instead simulates three tasks of rising difficulty. The first is estimating an unknown parameter from sequential evidence. The second is forecasting outcomes from inferred beliefs. The third adds a filter: observations arrive shaped by a user's persona.

Across seven models from 3 to 70 billion parameters, larger models inferred hidden structure better. They also accumulated evidence more faithfully, sometimes matching the exact Bayesian posterior. Yet those gains did not reliably improve the downstream forecasts. Knowing the hidden state and using it to update predictions proved to be separate skills.

Catalog record

CREATORAnkur Samanta, Akshayaa Magesh, Tal Lancewicki, Ayush Jain, Youliang Yu, Paul Sajda, Kaveh Hassani, Aditya Modi, Daniel R. Jiang, Yonathan Efroni
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE166 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.30850

Browse the full library · About the Tower of Babel