Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs
Gabriel Freedman, Francesca Toni · 2025
Read Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs by Gabriel Freedman, Francesca Toni (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-13644.txt.
Introduction
Gabriel Freedman and Francesca Toni (2025) test whether large language models hold rational beliefs about uncertain claims. Models now feed search tools and automated decision systems, so their handling of probability matters for trust. Earlier work hinted that models can reason and report well-calibrated confidence. The authors doubted that picture. They built a data set of claims whose truth is not settled either way, so a careful thinker should give middle-range chances rather than firm answers.
They then applied standard methods for drawing out stated probabilities from the models. The answers broke basic rules of coherent belief. Chances for a claim and its opposite failed to add up as they should, and related claims received inconsistent values. The authors conclude that current models do not represent probabilistic beliefs in a rational, coherent way. That gap, they argue, should temper how much weight decision systems place on model confidence.
Catalog record
| CREATOR | Gabriel Freedman, Francesca Toni |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 26 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-13644
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Exploring the Potential for Large Language Models to Demonstrate Rational Probabilistic Beliefs Gabriel Freedman, Francesca Toni Abstract Advances in the general capabilities of large language models (LLMs) have led to their use for information retrieval, and as components in automated decision systems. A faithful representation of probabilistic reasoning in these models may be essential to ensure trustworthy, explainable and effective performance in these tasks. Despite previous work suggesting that LLMs can perform complex reasoning and well-calibrated uncertainty quantification, we find that current versions of this class of model lack the ability to provide rational and coherent representations of probabilistic beliefs. To demonstrate this, we introduce a novel dataset of claims with indeterminate truth values and apply a number of well-established techniques for uncertainty quantification to measure the ability of LLM’s to adhere to fundamental properties of probabilistic reasoning. 1 Introduction In order for an agent to be an effective probabilistic reasoner, it must not violate the axioms of probability [ 2019 ] . If an agent violates any of these axioms, it implies that it lacks the capacity to perform robust probabilistic reasoning, including uncertainty quantification. Three of the most fundamental properties of probabilistic reasoning, derived as corollaries from Kolmogorov’s three axioms of probability theory [ 1963 ] , are as follows. These properties clarify essential relationships between events, their complements, and their probabilities (for examples of each see Figure 1 ): • Complementarity : Consider any event A within the sample space \Omega , where \Omega represents the set of all possible outcomes. Let A^{c} denote the complement of A , defined as the set of all outcomes in \Omega that do not belong to A . The property of complementarity states that: P(A)+P(A^{c})=1. This result arises directly from the axioms of normalisation (the probability of the entire sample space is 1) and finite additivity (the probability of the union of mutually exclusive events is the sum of their probabilities). • Monotonicity (Specialisation) : For two events A and A^{\prime} such that A^{\prime}\subset A (i.e. A^{\prime} is more specific than A ), the probability of A^{\prime} is less than or equal to the probability of A . Formally: P(A^{\prime})<P(A). This reflects the principle that the probability of a more specific or refined event (a subset of another event) cannot exceed the probability of the broader event it is contained within. • Monotonicity (Generalisation) : For two events A and A^{\prime} such that A\subset A^{\prime} (i.e. A^{\prime} is more general than A ), the probability of A is less than or equal to the probability of A^{\prime} . Formally: P(A)<P(A^{\prime}). This is the inverse of the specialisation relation. Generative LLMs have demonstrated impressive performance in many reasoning tasks - including tasks which they have not been specifically trained for [ 2020 , 2023 ] . This has led to the incorporation of LLMs into automated decision systems (ADSs) [ 2023 , 2023 , 2023 ] . In order for ADSs to be trustworthy and contestable [ 2021 , 2021 ] , they should be accompanied by a faithful representation of their reasoning. In the majority of real-world settings, in order for this to be an effective representation, it would need to include probabilistic uncertainty estimates. Unlike some previous work, we are attempting to measure something distinct from ‘subjective’ uncertainty estimates [ 2023 ] , which aim to assess the uncertainty intrinsic to the model. This is measured by comparing the uncertainty estimate with the veracity of the model’s output. Instead, we are interested in what may be called ‘objective’ uncertainty, as described in [ 2019 ] : “The objective probability of A at time t is the subjective probability that a perfectly rational agent would assign to A, if she had perfect information about the world at times \leq t and no information about the world at times >t.” . Thus, this is concerning the probability of a state of the world, regardless of the knowledge, or lack thereof, of the agent assigning the probability. This class of statements are at the core of academic disciplines and event forecasting. An example of a decision which requires ‘objective’ uncertainty estimations is determining the diets of the first inhabitants of America. The main sources of uncertainty in this inference come from imprecision in the measurement of carbon isotope levels in bone samples, as well as in auxiliary evidence about the climate [ 2011 ] . Having a consistent and rational model of the uncertainties involved in this scenario allows for the effective integration of any new evidence or theories that come to light, and the possibility of overhauling existing conclusions. Figure 1: Examples of LLMs violating three principles of probabilistic reasoning. The image above the line shows the original claim with the corresponding probability assignment by an LLM, and below the line the images demonstrate the violation of the principles of: complementarity (top), specialisation (middle) and generalisation (bottom). In this paper we demonstrate that the current generation of state-of-the-art LLMs, both open and closed source, frequently violate the basic principles of probabilistic reasoning, even when applying techniques for enhancing the ability of LLMs to perform uncertainty quantification. This undermines a corpus of existing work that argues that LLMs exhibit sophisticated decision-making capabilities [ 2023 ] , including the ability to effectively quantify uncertainty [ 2023 , 2023 , 2023 ] . In Section 2 we detail the construction of a novel dataset we use for evaluating LLMs’ probabilistic beliefs. In Section 3 we describe our method for evaluating LLMs’ adherence to the principles of complementarity and monotonicity and in Section 4 we outline our methodologies for measuring uncertainty. In Section 5 we report the results of our experiments using these methods. In Section 6 we briefly highlight relevant related work, and in the final Section we discuss the implications of our results and propose possible directions for future work. Code and data can be found at https://github.com/GIFRN/Rational-Probabilistic-Beliefs . 2 Novel Dataset A contribution of this work is the construction of a novel, synthetically generated dataset of 517 claims with indeterminate truth values. Each claim is accompanied by its complement, as well as a more specific and more general version of the claim, amounting to 2068 samples in total. We name this the Rational Probabilistic Belief (RPB) dataset. The initial claims were generated by prompting GPT-4o to generate claims that had an n% probability of being true, with n ranging between 10% and 90% in 10% intervals. To promote variety in the types of claims, we generated three variations of the basic prompt: asking the LLM to pose as an expert, and asking for historical claims and scientific claims. 200 claims were generated using each version of the prompt, resulting in 800 total. In order to further ensure sufficient diversity of claims, we filtered out similar claims. We did this with simple string matching, removing any claim that overlapped with any other claim in the dataset by at least 90%. While this naive method does not capture semantically similar claims that are expressed differently, we note that the exact formulation of a statement sometimes has an impact on the uncertainty estimations of the LLMs, therefore making it desirable to have some samples where