Can Transformers Learn Full Bayesian Inference In Context?

Arik Reuter, Tim G. J. Rudner, Vincent Fortuin, David Rügamer · 2025

Read Can Transformers Learn Full Bayesian Inference In Context? by Arik Reuter, Tim G. J. Rudner, Vincent Fortuin, David Rügamer (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-16825.txt.

Introduction

Transformers, the engines behind chat tools, can learn a task from the examples in a prompt, with no new training. Arik Reuter and co-authors ask in 2025 if such engines can do full Bayesian maths in that in context way. Bayesian work weighs fresh data against prior views to give a spread of likely answers.

The team builds a frame that joins set fit networks with smooth flow tools. On common models, such as line fits and hidden factor models, the frame draws answer spreads in context. Tests on real data show those spreads match the quality of slow, step by step methods that need fresh runs for each task. The work widens what in context learning can mean.

Catalog record

CREATORArik Reuter, Tim G. J. Rudner, Vincent Fortuin, David Rügamer
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE197 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-16825

Browse the full library · About the Tower of Babel