Reasoning aligns language models to human cognition Gonçalo Guiomar, Elia Torre, Pehuen Moure, Victoria Shavina, Mario Giulianelli, Shih-Chii Liu, Valerio Mante Abstract Do language models make decisions under uncertainty like humans do? And if so, what role does chain-of-thought (CoT) reasoning play in the underlying decision process? We answer this question by introducing an active probabilistic reasoning task that cleanly separates sampling (actively acquiring evidence) from inference (integrating evidence towards a decision). Benchmarking humans and a broad set of contemporary LLMs against near-optimal reference policies reveals a consistent pattern: extended reasoning is the key determinant of strong performance, driving large gains in inference and producing belief trajectories that become strikingly human-like, while yielding only modest improvements in active sampling. To explain these differences, we fit a mechanistic model that captures systematic deviations from optimal behavior via four interpretable latent variables, placing humans and models in a shared low-dimensional cognitive space . The resulting fits show how CoT shifts models toward human-like regimes of evidence accumulation and belief-to-choice mapping, tightening alignment in inference while leaving a persistent gap in information acquisition. † † affiliationnotice: Equal contribution 1 Introduction Figure 1: From task performance to latent cognitive variables. A: Task. We introduce an active probabilistic reasoning task in which agents sequentially sample from up to four buttons (A–D), each revealing a binary outcome (RED/GREEN). One button is biased toward RED, while the others are unbiased. During N sampling rounds, agents actively choose what evidence to sample among the buttons available on a given round. In a final inference round, agents indicate which button they believe is biased. An equivalent text-based version is used for LLMs (Appendix B ). B: Behavior. We compare human and LLM behavior by quantifying overall performance, sampling/inference quality, and invalid choices. These metrics reveal a broad spectrum of performance, with extended chain-of-thought reasoning improving overall success via enhanced inference, while gains in sampling remain limited. C: Mechanisms. To move beyond behavioral scores, we fit a mechanistic model that captures deviations from optimal Bayesian inference using four interpretable latent variables: Memory ( \beta ), Strategy ( \kappa ), Choice Bias ( \omega ), and Occlusion Awareness ( \theta ). D: Cognitive space. These latent variables define a shared low-dimensional cognitive space in which humans and models can be positioned. Reasoning shifts LLMs toward human-like inference strategies, and tightens, but does not fully close, the gap in sampling strategies. Chain-of-thought ( CoT ) prompting can substantially improve LLM performance on challenging benchmarks, often reaching near-human accuracy, by eliciting intermediate, step-by-step reasoning ( 65 ; 78 ; 98 ; 42 ; 96 ; 16 ; 38 ; 31 ; 24 ) . However, it remains unclear why CoT helps and how these reasoning traces relate to the computations that actually produce the answer. Indeed, CoT narratives can be unreliable and can diverge from the model’s internal processes, implying that they do not necessarily describe the true algorithm that a model applied in solving a task ( 92 ; 47 ; 5 ; 17 ; 3 ) . More generally, algorithmic-level descriptions that capture the abstract representations and processes explaining the behavior of LLMs are arguably missing for most current benchmarks ( 62 ) . Yet, a long line of work in cognitive neuroscience suggests that such descriptions could be critically important for any attempts to link a behavior to specific parameters and activations in the underlying circuits ( 57 ; 51 ) . Making explicit the algorithms and representations that lead to a particular behavior can yield interpretable latent variables that are often more directly tied to activations than the observable behavior, and thus help bridge what in most tasks is a wide conceptual gap between behavior and the underlying very complex, high-dimensional activations ( 51 ; 44 ) . Such algorithmic descriptions are also needed to evaluate human–LLM alignment when LLMs are treated as candidate models of human cognition , and to assess how CoT shapes that alignment. Fine-tuning on human feedback and behavioral datasets can make LLMs reproduce human-like choices and signatures of classic cognitive models ( 46 ; 74 ; 97 ; 4 ; 70 ; 10 ; 60 ; 7 ) . Yet, performance and behavioral similarity alone may not be sufficient to conclude that LLMs rely on the same latent computations and decision strategies as humans. In this work, we take inspiration from cognitive neuroscience by developing a behavioral task that isolates two core elements of decision-making under uncertainty: sampling and inference . Despite its simple structure, our task is demanding, making it a sensitive benchmark for both humans and frontier LLMs. Crucially, behavior in our task can be explained in terms of the dynamics of a few, interpretable latent variables, which capture the agents’ strategies and how they are shaped by CoT . Concretely, we make the three following contributions: • We introduce an active probabilistic reasoning task that disentangles sampling (evidence acquisition) from inference (evidence integration) and is well-suited for algorithmic, process-level analysis of behavior (Section 3 , Fig. 1 A ). • We evaluate a broad set of LLMs and human participants under identical instructions, and use near-optimal reference policies to assess sampling and inference quality separately. These analyses reveal a broad spectrum of performance with several models reaching human-level. Some models exceed humans in inference quality, but consistently under-perform in sampling quality (Sections 4 , 5 , 6 , Fig. 1 B ; Fig. 2 ). • We develop a mechanistic model that captures how agents sample evidence and update their beliefs in our task. The model places humans and LLMs in a shared cognitive space of four latent cognitive variables: Memory- \beta , Strategy- \kappa , Choice Bias- \omega , Occlusion Awareness- \theta . Fits of this model reveal that chain-of-thought reasoning improves LLM performance by shifting their sampling and inference strategies closer to those of humans (Section 7 , 8 ; Fig. 1 C–D ; Fig. 3 and 4 ). 2 Related Work In recent literature, LLMs are increasingly studied as generalist decision-making agents that act in dynamic environments and adapt via in-context learning (ICL) and in-context reinforcement learning (ICRL) ( 13 ; 103 ; 48 ; 85 ; 53 ; 52 ; 22 ; 59 ) . The same domain-general competence motivates a complementary line of work that treats LLMs not only as capable agents, but as candidate models of human cognition. Since a single fixed model can be evaluated across diverse tasks, it provides a substrate for which observable behaviors could potentially be precisely linked to internal representations ( 80 ; 26 ; 9 ; 1 ; 106 ; 54 ) . These two lines of work intersect in sequential decision-making settings, such as classical multi-armed bandits (MABs) and their variants ( 88 ; 49 ) . In these settings, LLMs can leverage bandit feedback to improve their choices in-context ( 50 ; 18 ; 43 ; 61 ; 81 ) . Moreover, alignment and fine-tuning on human feedback and behavioral datasets can substantially increase human-likeness, in some cases matching the decision dynamics of traditional cognitive models ( 4 ; 70 ; 8 ; 7 ; 60 ; 90 ) . However, most evaluations in these settings emphasize the exploration–exploitation trade-off or overall performance, providing limited leverage for distinguishing between underlying strategies that could generate similar behavioral outcomes ( 72 ; 77 ; 64 ; 25 ) . Our task departs from classical stochastic MABs by design: instead of conflating information gathering with reward maximization, it separates sampling from inference, which we find to be key to precisely assess alignment between LLMs and humans. 3 Active Probabilistic Reasoning Task We introduce an active probabilistic reasoning task (Fig . 1 A ) inspired by classical decision-making and K -armed bandit paradigms ( 45 ; 20 ; 63 ; 88 ; 49 ) , but explicitly designed to disentangle sampling from inference . The task is well-suited for modeling at the algorithmic level, and its structure is ”simple” in comparison to many current benchmarks ( 40 ; 89 ; 52 ; 107 ; 58 ; 75 ) . This simplicity, however, is deceiving, as below we find that CoT reasoning is required for LLMs to achieve high performance in this task, and that even leading models still under-perform compared to humans in key aspects of the task. Each trial is an independent game with a variable number of sampling rounds N\in\{2,\dots,15\} followed by a single inference round in which the agent must commit to a final decision. On each sampling round, the agent selects one of four buttons (A–D) and observes a binary outcome (RED/GREEN); at the start of game g , exactly one button is biased towards RED, emitting RED with probability \alpha_{B}=0.9 , while the remaining buttons are unbiased with \alpha_{U}=0.5 . The agent’s objective is to identify the biased button from the sampled evidence and report it in the inference round. Formally, the biased button is a latent variable z_{g}\in\{1,\dots,4\} ; at round t the agent chooses a_{t}\in\{1,\dots,4\} and observes x_{t}\in\{0,1\} (with x=1 for RED). To promote active information acquisition, between 0 and 3 buttons are temporarily occluded on each round and thus unavailable. Although this task resembles a K -armed bandit in its sequential action structure, it differs in computational goal and information structure: unlike standard bandits where choices yield rewards and agents maximize cumulative return via exploration–exploitation, here performance is determined solely by a one-shot final inference choice. Sampling choices carry no reward but only provide evidence about the single latent hypothesis (which button is biased). This separation enables independent assessment of sampling quality and inference quality. 4 Data Collection We collect behavioral data from both human participants and language models. Human participants played an interactive, graphical version of the task 1 1 1 https://ai.trt-bench.org . For language models, we designed an equivalent text-based version of the task, in which LLMs interact via token-defined choices under the same instructions provided to the human players (see Appendix B ). Here we describe in detail the experimental procedures used for humans and language models. Humans. We recruited 50 human participants for a 1-hour live in-person competition. All participants provided written informed consent prior to the task and received instructions matching those used to prompt the LLMs (Appendix B ). Participants played multiple independent games with trial lengths sampled uniformly between 2 and 15 rounds. Of the 50 participants, 46 completed the full protocol of 100 games each, yielding 4,600 games in total (see Appendix E for individual participant performance profiles). Large language models. We evaluate a broad set of models spanning the current LLM landscape, including both state-of-the-art closed-source systems and competitive open-weight models. The models range from dense to Mixture-of-Experts (MoE) architectures across different sizes and training paradigms: pre-trained, instruction fine-tuned, reasoning, and hybrid-reasoning models ( 94 ; 82 ; 86 ; 97 ; 70 ; 98 ; 87 ; 83 ; 15 ) . Our model selection includes: OpenAI’s gpt 4o mini ( 37 ) , gpt 4.1 mini ( 66 ) , gpt 5 mini ( 67 ) , and the gpt oss open architectures in both the 20b and 120b parameter variants ( 68 ) ; several llama models ( 91 ; 23 ) , including variants fine-tuned on human behavioral data ( 7 ) ; a distilled version of deepseek ( 32 ) ; Anthropic’s claude sonnet 4 and claude haiku 3.5 ( 2 ) ; Google’s gemini 2.5 pro/flash and the smaller gemma models ( 19 ; 27 ) ; the qwen family, including the 235B Mixture-of-Experts (MoE) model and earlier dense variants ( 76 ; 102 ) ; and the fully open-source apertus model ( 35 ) , grok 3 mini ( 100 ) , and glm 4.5 ( 104 ) . Some of the included reasoning models allow for control over reasoning effort (resulting in longer or shorter chain-of-thoughts token sequences); we evaluate their performance for both low - and high-reasoning effort. In all figures, results for the high-reasoning condition are shown with gray bars (see, e.g., Fig. 2 A ) and denoted as Extended Reasoning . For every LLM and reasoning condition, we evaluate a minimum of 1,400 individual games spanning uniformly the range of 2 to 15 rounds, amounting to more than 55,000 games in total. Figure 2: Comparing human and LLM behavior. A: Task performance. Average success rate across trial lengths N\in\{2,\dots,15\} . We report human performance (green), split into lower 75\% and top 25\% of participants and the near-optimal reference agent (PPO sampling + MAP inference) (light blue). For models that support increased reasoning effort, gray overlays indicate the Extended Reasoning condition. Error bars represent standard deviations, computed across trial-cluster means with a uniform distribution over the number of rounds. The vertical line marks chance performance (25%). B: Sampling and inference quality. We quantify sampling quality (left) and inference quality (right) as performance loss with respect to the near-optimal agent (lower is better): inference loss is the gap between the agent and a counterfactual agent that preserves the same sampled evidence but applies MAP at the inference round for the final decision; sampling loss is the gap between this counterfactual MAP agent and the reference agent (PPO + MAP), isolating suboptimal evidence acquisition. Reasoning primarily reduces inference loss, with only modest effects on sampling loss. C: Invalid choices. Fraction of invalid choices during sampling (left) and at the final, inference decision (right). Invalid choices include selecting occluded options, producing tokens outside the valid choice set (A–D), or failing to respond; humans cannot produce invalid choices in the graphical interface. Invalid choices occur more frequently during sampling than at the final decision and are reduced by reasoning. 5 Optimal Agent To evaluate and compare the performance of LLMs and humans, we introduce an optimal agent that provides an upper bound on task performance. In our task, where inference and sampling are explicitly separated, establishing the optimal strategy for inference is straightforward: an agent should select the button with the highest posterior probability of being biased, following a Maximum-A-Posteriori (MAP) decision rule ( 30 ) . This optimal inference strategy is defined as follows; button outcomes are generated by Bernoulli emissions, with the emission probabilities \{\alpha_{B},\alpha_{U}\} for the biased and unbiased buttons respectively. A single game g of N rounds consists of a sequence of choice-evidence pairs (a_{1:N},x_{1:N}) together with a latent context variable k\in\{1,\dots,K\} . We represent beliefs by the posterior vector (\bm{p}_{t})_{k}\in\Delta^{K} . We also define the likelihoods over the latent variable k and emission rates \{\alpha_{B},\alpha_{U}\} by (\bm{L}_{t})_{k}\in\Delta^{K} . By assuming a uniform prior, we can write down the recursive normative posterior via Bayes’ rule as 2 2 2 See Appendix Section H.2.2 for derivation details. : \bm{p}_{t}=\frac{\bm{p}_{t-1}\odot\bm{L}_{t}}{\mathbf{1}^{\top}(\bm{p}_{t-1}\odot\bm{L}_{t})},\hskip 20.00003pt(\bm{p}_{0})_{k}=\tfrac{1}{4}. (1) where \odot denotes the Hadamard product. The MAP inference policy at round t selects the hypothesis with maximum posterior probability: a^{\mathrm{MAP}}_{t}:=\arg\!\max_{k\in\{1,\dots,4\}}(\bm{p}_{N})_{k}. (2) Determining the optimal sampling strategy is substantially more complex. The agent faces two sources of uncertainty: (1) buttons are randomly occluded at each round, restricting the available choice set at time t , A_{t}\subseteq\{1,\dots,4\} in unpredictable ways, and (2) the total number of sampling rounds N\in\{2,\dots,15\} is unknown to the agent until the inference round begins. These factors render analytical derivation of the optimal sampling policy non-trivial. Consequently, we employ reinforcement learning to obtain a near-optimal sampling strategy. We train an RL agent using Proximal Policy Optimization (PPO) ( 82 ) ; see Appendix C for full details. The resulting optimal agent combines the PPO-trained sampling policy with the MAP inference rule at the final decision round. We assume that this agent achieves near-ceiling performance and provides normative benchmarks for both sampling and inference quality . 6 Comparing Human & LLM Behavior To compare human and LLM performance, we employ four complementary metrics that capture distinct aspects of behavior: (1) overall task performance, (2) sampling quality, (3) inference quality, and (4) invalid choice rates. Together, these metrics reveal a broad range of behaviors across LLMs, as well as systematic differences between LLM and human performance. Overall task performance. In Figure 2 A , we quantify overall task performance as the average success rate across all trial lengths ( N\in\{2,\dots,15\} ), i.e., the fraction of games in which the agent correctly identifies the biased button at the inference round. Despite the task’s simple structure, it induces a broad distribution of performance across both humans and models, and is sensitive to explicit reasoning, with clear gains from extended chain-of-thought (gray vs. colored bars). On average, human participants achieve \simeq 61\% success, comparable to the best-performing LLM ( gpt 5 mini ); however, the top quartile of humans exceeds this level by \sim 7\% , highlighting wide human variability and a remaining gap between top models and skilled human players. Across models, reasoning LLMs outperform non-reasoning counterparts, attaining higher success rates overall. Success rates increase with game-length only for reasoning models; success rates in low/no-reasoning models remain largely flat, suggesting they largely fail to leverage the additional evidence provided by long games (Appendix D ). Sampling & inference quality. In Figure 2 B , we quantify inference quality by computing the performance gap between each agent and a counterfactual agent that retains the same sampling strategy but applies the MAP inference rule at the final choice. This difference quantifies the performance loss attributable solely to suboptimal inference (right-hand side of Figure 2 B , lower is better). We also evaluate Sampling quality , by comparing this counterfactual agent (which combines the agent’s sampling strategy with MAP inference) against the optimal agent defined above (which combines the RL-trained sampling policy with MAP inference). Since both agents share the optimal MAP inference rule, any remaining performance gap reflects the loss due to suboptimal sampling strategy (left-hand side of Fig. 2 B , lower is better). We find that inference quality varies broadly across LLMs and is strongly correlated (negatively, because the metric is a loss) with overall performance ( r=-0.99 ), whereas sampling quality is more homogeneous across LLMs and a weaker predictor of performance ( r=-0.47 ; r=-0.70 if we include extended reasoning variants). Performance gains from extended reasoning are driven mainly by improved inference: for any given LLM, chain-of-thought reduces inference loss by an amount that largely mirrors the corresponding gain in overall performance. Notably, frontier reasoning models match, and sometimes exceed, humans in inference quality. In comparison, extended reasoning has only modest effects on sampling quality, and while the best reasoning models approach the performance of the lower 75% of human participants, skilled humans (top 25%) still substantially outperform LLMs in their sampling strategies (left-hand side of Fig. 2 B ; skilled humans shown in light green). Invalid choice rates. Finally, in Figure 2 C , we quantify invalid choices as the proportion of choices in which agents produce invalid responses during either sampling or inference. Invalid choices include selecting occluded buttons, generating output tokens outside the valid button vocabulary (A–D), or failing to produce a response altogether. By design, human participants cannot produce invalid choices in the graphical interface, and therefore invalid-choice rates are computed only for LLMs. Overall, we do not observe a statistically significant correlation between invalid choice rates and LLM performance; however, invalid choices are consistently reduced under extended reasoning. Invalid responses tend to occur more often during sampling than during inference, potentially reflecting the changing availability of choices during the sampling rounds. 7 A mechanistic model of latent cognitive variables Figure 3: Model parameters explain human and LLM behavior A : Memory parameter \beta governs non-optimal evidence-integration, spanning stubborn ( \beta<0 ) to forgetful ( \beta>0 ) regimes. B : Strategy parameter \kappa controls choice stochasticity ( random near \kappa=0 , increasingly rational for large \kappa ), fit separately for sampling (left) and inference (right). C : Entropy of Choice-bias vector \omega^{x} captures deviations from internal posterior-driven decisions, fit separately for sampling and inference (more biased for smaller entropy). D : Occlusion Awareness parameter (shown as \log\theta ) captures sensitivity to occlusions (invalid choices). E : Model schematic: choices generate evidence that updates a Bayesian posterior p_{t} ; memory ( \beta ) produces a non-optimal information accumulation process h_{t} ; strategy ( \kappa ) applies an inverse-temperature transformation; bias ( \omega ) and awareness ( \theta ) modulate the resulting policy \pi_{t} . The behavioral metrics above reveal systematic differences across LLMs, but do not identify the processes responsible for them. For instance, reasoning might improve inference by changing a single computation shared across models (e.g., evidence accumulation), by jointly affecting multiple computations within each model (e.g., evidence accumulation and internal biases), or by altering different computations in different models. To resolve these possibilities, we develop and fit a mechanistic model of sampling and inference for both humans and LLMs. The model draws on Bayesian ideal-observer and probabilistic cognition frameworks ( 41 ; 79 ) and parallels recent views of in-context learning as implicit inference over latent context variables ( 101 ; 71 ; 105 ) , but allows for deviations from an ideal-observer in terms of memory accuracy, choice selection strategy and relative choice bias. The fitted parameters define four interpretable latent cognitive variables: Memory, Strategy, Choice Bias , and Occlusion Awareness placing agents in a cognitive space (Fig. 4 A and schematically in Fig. 1 D ) spanned by the key parameters influencing decision strategy. This approach allows us to interpret key differences across LLMs and humans (Figs. 3 , 4 A ). Memory ( \beta ). As in traditional accounts of probabilistic inference in neuroscience and sequential sampling models ( 14 ; 93 ) , we assume that an agent’s choices are a consequence of a non-optimal memory \bm{h}_{t}\in\mathbb{R}^{K} ( 29 ; 36 ; 55 ) and is updated via the mean subtracted likelihood: \Delta\bm{h}_{t}:=\log\bm{L}_{t}-\left\langle\log\bm{L}_{t}\right\rangle (3) (See Appendix H.4.3 for details). After each sampling round, \bm{h}_{t} is given by \bm{h}_{t}=(1-\beta)\,\bm{h}_{t-1}+\Delta\bm{h}_{t}. (4) Deviations from optimal recollection of evidence are controlled by the memory parameter \beta\in[-1,1] . For \beta=0 accumulation of evidence is perfect ; for \beta>0 the agent is forgetful , leading to late evidence weighing more on choices than early one; and for \beta<0 the agent is stubborn , leading to early evidence weighing more on choice than late one. Strategy ( \kappa ). After updating its memory state \bm{h}_{t} based on new evidence, the agent converts \bm{h}_{t} into an internal posterior \bar{\bm{p}}_{t}\in\Delta^{K} over which button is biased. We model this memory-to-belief transformation by multiplying \bm{h}_{t} with a strategy parameter \kappa_{x} and re-normalizing, fit separately for sampling and inference ( x\in\{s,f\} ). This yields a continuum from near-random behavior ( \kappa_{x}\approx 0 ) to increasingly MAP-like choices (large \kappa_{x} ) ( 69 ) : \bar{\bm{p}}^{\,x}_{t}=\sigma(\kappa_{x}\bm{h}_{t}), (5) For \kappa_{x}>1 , differences in accumulated evidence are amplified, producing sharper, more decisive behavior; for \kappa_{x}<1 they are compressed, yielding a more diffuse posterior. In the limit \kappa_{x}\gg 1 , the policy becomes effectively MAP-like , whereas \kappa_{x}\approx 0 yields near-uniform ( random ) choices. The policy for both sampling and inference choices is derived directly from \bar{\bm{p}}^{\,x}_{t} . Choice Bias ( \bm{\omega} ). To dissociate evidence-driven choices from systematic preferences, we include choice biases \{\bm{\omega}_{s},\bm{\omega}_{f}\}\in\Delta^{K} for the sampling and final choices, respectively ( 28 ; 99 ; 39 ) . These normalized bias vectors act multiplicatively on the internal posterior to rescale the propensity for a choice independently of the sampled evidence. To quantify the strength of the bias in an agent, we compute the entropy of the bias vectors H^{x}(\bm{\omega})=-\sum^{x}_{j}\omega^{x}_{j}\log(\omega^{x}_{j}) . Unbiased agents distribute their choices uniformly in the absence of evidence, corresponding to large bias entropy. While, biased agents systematically favor one or more choices over others, corresponding to small bias entropy. Occlusion Awareness ( \theta ). To account for invalid sampling choices, we included the occlusion awareness parameter \theta\in\mathbb{R}^{>0} , which like the choice biases acts multiplicatively on the internal posterior (see ‘Policy’ below). We define a multiplicative mask m^{s}_{t,j} as: m^{s}_{t,j}=\theta^{-\mathbf{1}\{j\notin\mathcal{A}_{t}\}}, (6) where \mathcal{A}_{t} are the available choices at (t) . Large \theta values result in an agent that is more occlusion‑aware and avoids choosing occluded buttons while small \theta values make the agent more prone to selecting occluded buttons. Policy. Finally, the stochastic policy is obtained by combining the internal posterior , the choices biases , and occlusion awareness . We first define an un-normalized choice preference: \bm{u}^{\,x}_{t}:=\bar{\bm{p}}^{\,x}_{t}\odot\;\bm{w}_{x}\;\odot\;\bm{m}^{\,x}_{t}, (7) The preferences are then normalized to obtain the full policy \bm{\pi}^{\,x}_{t}=\frac{\bm{u}^{\,x}_{t}}{\mathbf{1}^{\top}\bm{u}^{\,x}_{t}} from which choices are sampled. The details of the model evaluation are described in Appendix E and full model derivations in Appendix H . 8 Reasoning aligns humans and LLMs The parameters of the mechanistic model provide a compact description of the algorithms implemented by the agents we evaluated on the active probabilistic reasoning task (Fig. 3 ; parameter values are also shown in Appendix J ). Critically, this description is also accurate, as simulations of the mechanistic model based on the fitted parameters reproduce all main features of LLM and human behavior in our task (see Appendix F , Fig. 7 ). We find that two model parameters are strongly related to overall LLM performance. First, the memory parameter \beta has values close to zero for all the highest performing models, indicating close to perfect accumulation of evidence. Poorly performing models instead tend to have large positive or negative \beta values, indicating either forgetful or stubborn accumulation of evidence (Fig. 3 A ). Second, the inference strategy parameter, \kappa_{f} , has larger values for better models (Fig. 3 B ). For the best models, we find \kappa_{f}\gg 1 , implying a strategy that approximates a maximum-a-posteriori (MAP) policy. For models with intermediate performance, \kappa_{f} is closer to 1, meaning that information from the internal posterior is influencing final choices, but less so than expected from an optimal agent. Finally, for models with poor performance, \kappa_{f} progressively approaches zero, implying random choices and an inability to act on the observed evidence. Together, these trends in the memory and strategy parameters contribute to the improvements in inference quality observed in high-performing models (Fig. 2 B ). However, other parameters also show substantial variation across LLMs, but weaker correlations with overall performance. Many poorly performing models, but also some high-performing ones, show substantial choice biases (Fig. 3 C ), which necessarily reduce sampling and/or inference quality. For any given LLM, the strength of sampling and inference biases is typically similar, suggesting a common underlying mechanism. LLMs also vary substantially in their occlusion awareness (Fig. 3 D ), resulting in widely different numbers of invalid choices across LLMs (Fig. 2 C ). Across LLMs, CoT reasoning affects all four model parameters, albeit to different extents (Fig. 3 A-D , gray vs. color bars). In most reasoning LLMs, a higher reasoning effort leads to \beta values closer to zero, indicating more accurate accumulation of evidence, and larger \kappa_{f} values, indicating a more rational policy on the final choice. Higher reasoning effort also reduces choice biases, during both sampling and inference, and increases occlusion awareness. Nonetheless, none of the tested LLMs can fully reproduce the algorithms employed by the optimal agent (Fig. 3 A-D , blue bars). In particular, the inferred sampling strategy parameters \kappa_{s} are substantially smaller than those for the optimal agent or humans (Fig. 3 B , left). This finding implies that sampling choices in LLMs are consistently less influenced by the observed evidence than the optimal agent, which explains the resulting gap in sampling quality discussed above (Fig. 2 B , left). Overall, CoT reasoning results in an increased alignment between the choice strategies used by LLMs and humans. This increased alignment is best appreciated when mapping each agent onto a location in a Cognitive Space spanned by \beta and \kappa_{f} (Fig. 4 A , and schematically in Fig. 1 D ). This summary representation shows how CoT pushes LLMs towards strategies that are closer to those used by skilled humans (Fig. 4 A , light green dot) and that result in higher overall performance (Fig. 4 A , background color and contours). Figure 4: Reasoning aligns language models to human cognition. A: Cognitive space. Agents embedded by fitted memory \beta ( stubborn \beta<0 to forgetful \beta>0 ) and inference strategy \kappa_{f} (more rational for larger \kappa_{f} ). Near-optimal agent not shown ( \kappa_{f} beyond y-axis limits). Shading/contours show predicted success across fitted model (\beta,\kappa_{f}) ; markers show humans (green/light green) and LLMs under low/no reasoning (black border) versus extended reasoning (white border). Reasoning shifts models toward the high-success, human-like regime. B: Latent-variable dynamics. Shows a specific evidence sequence (top row) and resulting round-by-round dynamics of latent variables (lower rows) during a game by gpt oss 20b under high - (left) vs. low -reasoning effort (right). Game-dependent terms (log-likelihood increments \Delta h_{t} and the posterior p_{t} ) are identical across efforts, while agent-dependent quantities differ: low-reasoning yields a stubborn memory state trajectory h_{t} (4th row; \beta<0 ) and diffuse policy \pi_{t} (5th row; smaller \kappa_{f} ), whereas high-reasoning shows near-optimal memory updates ( \beta\!\approx\!0 ) and a sharper, more decisive belief-to-choice mapping (larger \kappa_{f} ). 9 Reasoning shapes latent cognitive variables Beyond summarizing the key features of the algorithms implemented by LLMs and humans, our mechanistic model also reveals latent cognitive variables whose dynamics reflect the operation of the distinct processes driving behavior (Fig. 3 E ). For example, our model implies that the final choice generated by an LLM reflects a policy that is obtained by accumulating all the sampled evidence into the memory , producing a strategy-sensitive internal posterior , and finally combining it with systematic biases. These latent variables provide another view into the effects of CoT reasoning, as exemplified in Fig. 4 B by the round-by-round dynamics of some of the inferred variables for a single game played by gpt oss 20b , under high - (left) versus low -reasoning effort (right). We consider how the latent variables are updated on each round, during a game based on a specific sequence of RED and GREEN outcomes sampled across the four buttons (1st-row from top). Each outcome is transformed into a log-likelihood-ratio increment \Delta\bm{h}_{t} (2nd-row) that can be accumulated into the optimal Bayesian posterior \bm{p}_{t} (3rd-row). In our model, the \Delta\bm{h}_{t} contribute to the memory state \mathbf{h}_{t} (4th-row), which is then transformed into a policy for the final choice \bm{\pi}_{t} (5th-row). The dynamics of the resulting variables differ prominently across reasoning efforts. First, the memory state ( \mathbf{h}_{t} ) is much more similar to the Bayesian posterior ( \mathbf{p}_{t} ) for high- vs. low-reasoning effort. Midway through the example game, the Bayesian posterior switches from favoring button C to favoring button A , a dynamic reproduced by the memory state for high-reasoning (because \beta\approx 0 ). For low-reasoning , the memory state is stubborn ( \beta<0 ), and continues favoring the initially best option ( button C ) even in light of new evidence towards A . Second, reasoning affects the transformation of the memory into the internal posterior : high-reasoning sharpens this mapping, by favoring the best option on each round and ultimately resulting in a more rational policy ( \kappa_{f}\gg 1 ). In contrast, low-reasoning weakens the differences between options in the internal posterior , which then results in a more diffuse policy ( \kappa_{f} close to 0 ). Such latent variables also amount to concrete, testable hypotheses for mechanistic interpretability, which aims to explain behavior at the implementation level by identifying the circuits and causal pathways that give rise to it ( 95 ; 34 ; 33 ; 6 ; 73 ; 84 ) . Past efforts to understand CoT at the implementation level have yielded limited insights, in part because long reasoning traces expand the space of candidate computations and complicate causal attribution ( 12 ; 56 ) . Latent variables as in Fig. 4 B (and Appendix Fig. 8 B ), have structure and dynamics that is much richer than the categorical (A–D) outputs required by the task, and as such motivate targeted questions about their representation: whether and where these latent variables are encoded in activations and circuits, which layers instantiate them, and how they evolve over the course of a reasoning chain . 10 Discussion Our goal in this work was to understand whether language models form beliefs and make decisions under uncertainty in a human-like way, and how this alignment is affected by chain-of-thought ( CoT ) reasoning. We argued that answering these questions requires approaches that go beyond evaluating task performance alone, and instead rely on algorithmic descriptions that can isolate similarities and differences in the strategies implemented by different agents. We applied this approach in a novel, active probabilistic reasoning task that disentangles sampling (active evidence acquisition) from inference (evidence integration) and employed a mechanistic model to infer interpretable, latent variables explaining each agent’s behavior. Across LLMs, CoT reasoning is the key determinant of strong performance, but its effects are selective: reasoning leads to better performance primarily by improving inference rather than sampling . Mechanistically, improved inference is explained by (i) reduced choice biases ( \omega ), (ii) a more MAP-like belief-to-choice mapping (larger inference strategy parameters \kappa_{f} ) and, in strong models (iii) more accurate evidence-integration ( \beta\!\approx\!0 ). Together, these effects of reasoning move LLMs toward human-like regimes in cognitive space (Fig. 4 B ); sampling strategies, however, remain sub-optimal, leaving a persistent performance gap relative to skilled human sampling. Surprisingly, CoT reasoning in frontier LLMs leads to better alignment between humans and LLMs on our task than what is achieved by non-reasoning LLMs that were fine-tuned on human behavior from large corpora of cognitive tasks 7 . Our work suggests a path toward LLM evaluations that measure capability while constraining underlying mechanisms, by asking not only whether models solve a task but how they do so. The resulting compact, algorithmic descriptions of how reasoning changes LLM strategies provides concrete hypotheses for analyses at the implementation-level. 11 Reproducibility Statement To ensure reproducibility of our results, all code used in our experiments, along with detailed instructions for setup and execution, is available at: https://drive.google.com/drive/folders/17tQxO02lLN1VpwbOF_IIiM9oSm8DmRik . Additionally, the active probabilistic reasoning task used for data collection is accessible at https://ai.trt-bench.org . 12 Impact Statement This paper presents work whose goal is to advance our understanding of the field of Machine Learning through the lens of Cognitive Science. There are many potential societal consequences of our work, none which we feel must be specifically highlighted here. Appendix A Limitations and Future Work Our task and mechanistic model provide a step toward evaluations that move beyond final accuracy, which here allowed us to disentangle sampling from inference and to map both humans and LLMs into a shared space of interpretable latent variables. This framing yields a compact, process-level account of behavior and makes concrete, testable predictions. At the same time, some modeling and benchmarking choices trade expressivity for interpretability, suggesting clear directions for improvement. (1) Coarse belief dynamics via a single leak parameter \beta . We summarize memory with a single leak parameter \beta , which imposes one global timescale for evidence retention across all rounds and both task phases (sampling vs. final choices). This constraint improves interpretability and comparability, but cannot capture richer dynamics such as nonstationary integration of evidence, context-dependent leakage across rounds, or within-game regime switches. Future work could extend the model to include different internal memory \bm{h}_{t} integrators for sampling and inference , testing whether the observed stubborn and forgetful regimes reflect a single computation or a mixture of mechanisms that CoT modulates selectively. (2) Round-invariant parameters, especially the sampling strategy \kappa_{s} . Several parameters are assumed constant within a game, most notably the sampling strategy \kappa_{s} . Empirically, posterior trajectories for inference are well captured by the fitted model, whereas sampling posteriors in very specific cases are less accurately predicted, with deviations suggestive of within-game adaptation. The sampling curves shown in Appendix Figs. 9 – 12 show sampling posterior trajectories with different curvature dynamics than the ones observed in the inference posteriors (note in particular the trajectories for the top models). One plausible interpretation is in-context learning : as evidence accumulates, the belief-to-sampling mapping becomes sharper. Allowing a round-dependent \kappa_{s}(t) (or a simple update rule for \kappa_{s} ) would directly test this hypothesis and may improve the sampling posterior fits. This can be intuitively understood as the model changing its propensity for MAP-like sampling as it gathers evidence. Importantly, even though posterior trajectories are in some cases imperfectly fit, they provide additional, direct support for our main conclusions: sampling and inference dissociate, and extended reasoning primarily improves inference while leaving a systematic gap in active information acquisition. Our hypothesis is that the task mode switching that happens when transitioning from sampling to inference leads to a different computation to be performed on the evidence, as in the inference round all the evidence has been accumulated and so a more clear decision can be made, leading to higher \kappa ’s. In contrast, in settings when more exploration is needed, like early in sampling, lower \kappa ’s dominate. (3) No closed-form optimal sampler under occlusions and unknown horizon. Our benchmark relies on a near-optimal PPO policy as a reference sampler because the analytically optimal sampling policy is non-trivial under stochastic occlusions and an unknown number of rounds. Stronger optimality guarantees would further sharpen the decomposition into sampling versus inference losses. (4) A single set of task meta-parameters. Because collecting statistically reliable human data and running large-scale evaluations across a broad set of LLMs is expensive and time-consuming, we focus on a single regime of task meta-parameters (number of buttons, emission probabilities, occlusion process, and horizon distribution). Different regimes may induce qualitatively different optimal strategies and could change both the magnitude and the nature of CoT effects, motivating systematic sweeps over these task parameters. (5) Reasoning traces are not analyzed. We focus on behavior and latent-variable fits, but do not analyze the reasoning traces themselves. Combining trace-level analyses with latent variables could test whether (and when ) explicit CoT content tracks inferred belief states versus serving as a post-hoc narrative, and could identify trace features predictive of latent parameters (e.g., signatures of evidence accumulation or policy sharpening). More broadly, this would open a natural path for future mechanistic interpretability work: using trace-linked latent variables as targeted probes to ask where (and how ) quantities like p_{t} , h_{t} , or \kappa -like belief-to-policy transformations are instantiated in model activations and circuits. Overall, our framework points toward a next generation of evaluations: benchmarks that measure capability while constraining the computations that produce it, and mechanistic models that translate behavior into algorithmic variables, setting the stage for implementation-level tests of how reasoning is realized in language-model circuits. Appendix B Prompting Structure used for the LLMs in the Task We evaluate language models using a text-based version of the task that mirrors the instructions and feedback given to human participants. The prompt specifies the task mechanics (four cues, one biased at 90/10 while the others are 50/50), the round-by-round interaction protocol (available cues may change across rounds, but at least one cue is always available), and a strict response format (one letter per round, no additional text). After the sampling rounds, models are asked to report which cue had the highest proportion of RED outcomes. The prompt also defines the feedback and scoring scheme (correct/incorrect/invalid; +100 for a correct final report and -100 otherwise), explicitly framing success as identifying the biased cue. Models interact with the environment across full trials and receive round- and trial-specific updates conditional on their previous choices. ⬇ Task - You will play a game with \{ n \ _rounds \} rounds . - In each round , some cues are available : A , B , C , D . - One cue is biased : 90\% one color / 10\% the other . The others are 50/50. - Available cues may disappear at random , but at least one is always active . - Each round , respond with exactly one letter ( A , B , C , or D ). No markup or punctuation . - After \{ n \ _rounds \} rounds , identify the biased cue . - Scoring : Correct +100 points , Wrong -100 points . Round prompt - “ Trial X , Round \{ current \ _round \}: Available cues \{ available \ _cues \}. Which do you choose ? Respond with exactly one letter : A , B , C , or D .” Round feedback - “ Trial X , Round \{ current \ _round \}: Available cues \{ available \ _cues \}. You chose “ CURRENT \ _ANSWER ” and saw “ RESULT ”.” Final decision prompt - “ Trial X : Based on all observed colors , which cue \{ letters \} had the highest ratio of RED ? Respond with exactly one letter : A , B , C , or D .” Final feedback - “ Trial X : Based on all observed colors , which cue \{ letters \} had the highest ratio of RED ? You chose “ CURRENT \ _ANSWER ” which was “ FEEDBACK - CORRECT / INCORRECT ”. You received \{ score \} points .” Feedback labels - Correct : “ the biased cue ” - Incorrect : “ not the biased cue . The biased cue was “ BIASED \ _QUADRANT ”.” - Invalid : “ an invalid choice ” Appendix C Reinforcement Learning Agent Details We train a reinforcement-learning agent to obtain a near-optimal sampling policy for our task. Training is performed with Proximal Policy Optimization (PPO) [ 82 ] . At each round, the agent observes a 12-dimensional state vector consisting of (i) cue–color counts for each button (8 dimensions: 4 cues \times 2 colors) and (ii) a 4-dimensional binary mask indicating which cues are currently available. Environment dynamics (cue bias, occlusions, and trial length) match the experimental task, with the number of rounds N sampled uniformly from \{2,\dots,15\} . During the sampling phase, the PPO policy selects which cue to sample. At the end of the trial, the agent makes a final decision using a Maximum-A-Posteriori (MAP) rule given the accumulated evidence. Rewards are sparse and tied to task success: the agent receives +100 for a correct final report, 0 for valid sampling choices, and -10 for selecting an unavailable cue. Unavailable selections advance the round without revealing evidence, encouraging the agent to avoid invalid choices and sample efficiently. We select optimization and architecture hyperparameters via a grid search that maximizes correct cue identification. The policy network is a two-layer MLP with 64 hidden units per layer and ReLU activations. We train for 5\times 10^{6} environment steps with learning rate 2\times 10^{-5} and minibatch size 128. Each PPO iteration collects T=2048 on-policy steps and performs 10 optimization epochs. We use discount \gamma=0.99 , generalized advantage estimation \lambda_{\mathrm{GAE}}=0.95 , and clipping parameter \varepsilon=0.2 , with entropy and value loss coefficients set to 0.01 and 0.5 , respectively. Appendix D Task Performance across Rounds Figure 5: Evolution of success rate across rounds. For each agent, we report the mean success rate (fraction of trials in which the final reported button is the true biased one) as a function of the number of sampling rounds N\in\{2,\dots,15\} . Humans are shown in green, split into the top 25\% and lower 75\% of participants (same split as in Fig. 2 A ); LLMs are shown as individual traces (legend), with colors indicating each model’s overall average success rate (lighter/orange = lower, darker/purple = higher). Across models, this view reveals a qualitative separation around \sim 45\% average success: below this regime, curves remain approximately flat with increasing N , suggesting limited ability to convert additional evidence into improved final decisions; above it, curves exhibit a clear positive slope (“lift-off”), indicating effective inference in longer games. Claude haiku 3.5 is the first model in the performance ranking to show this lift-off, and from DeepSeek R1 0528 Qwen3 8B onward, many models track the round-by-round improvement profile of the lower 75\% human cohort. Overall, models that display lift-off are predominantly those with extended reasoning. Appendix E Human Participants We collected human data in a 1-hour, live in-person session. Fifty participants took part and provided written informed consent. Participants played independent games of our task with trial length sampled uniformly from N\in\{2,\dots,15\} under the same instructions used throughout the paper (Fig. 1 A). Forty-six participants completed the full protocol of 100 games each; all analyses in the main text therefore use these 46\times 100=4{,}600 trials. Figure 6 reports participant-level performance and decomposes it into inference versus sampling components, complementing the aggregate human summary in Fig. 2 . Human success rates span a wide range (roughly 37\% – 77\% ; chance =25\% ), with most participants clustered around \sim 55\% – 67\% . This heterogeneity is informative for our central comparison: the human distribution overlaps the best LLMs in overall success, while the strongest participants remain a meaningful reference point for what robust inference and effective sampling look like in this task. Figure 6: Individual human performance. Each row corresponds to one of the 46 participants who completed 100 games (trial lengths uniformly sampled from N\in\{2,\dots,15\} ). Participants are ordered by mean success rate. Mean success rate (correct identification of the biased option at the final decision); the vertical line marks chance performance (25%). The cohort exhibits substantial heterogeneity, spanning \approx 37\% to \approx 77\% success, with most participants concentrated around \sim 55 – 67\% . Appendix F Mechanistic Model reproduces Behavioral Metrics Model evaluation is performed by simulating the fitted model policy on the Active Probabilistic Reasoning Task . For each model and reasoning condition parameters, we roll out full games with the same underlying \{\alpha_{B},\alpha_{U}\} parameters as in the Human and LLM experiment. At each round, the internal memory representation is updated from the initial posterior and a sampling policy is formed from the choice bias and internal posterior . The comparative posterior/bias plots can be seen in Fig. 7 and represents the average evolution of the underlying posteriors across games with \{2,...,15\} rounds, split by sampling and inference components. Figure 7: Behavioral metrics are recapitulated by the fitted mechanistic model. A–C : Empirical behavioral metrics computed directly from human and LLM trajectories. A : Overall task performance (success rate) across agents. B : Decomposition of performance loss into sampling versus inference deficits, measured by comparing each agent to counterfactual and optimal reference policies (loss w.r.t. optimal sampling on the left; loss w.r.t. optimal inference on the right). C : Invalid-choice rates during sampling and final report. D–F : The same metrics computed from trajectories generated by the mechanistic model after fitting (\beta,\kappa,\omega,\theta) to each agent. D : Model-predicted success rates closely match empirical performance ordering. E : Model reproduces the sampling–inference loss decomposition, capturing that reasoning primarily reduces inference loss while leaving a residual sampling gap. F : Model reproduces invalid-choice profiles via the occlusion-awareness component. Green markers denote human quartiles; grey indicates extended-reasoning conditions. Appendix G Latent variable dynamics: Stubborn and Forgetful Examples Figure 8: Stubborn vs. forgetful latent dynamics under reasoning. For both models we show how the latent variables are updated on each round, during a game based on a specific sequence of red and green outcomes sampled across the four buttons under high -reasoning (left) and low -reasoning (right). The top block ( game-dependent ) shows the sampled evidence sequence and the corresponding log-likelihood increments \Delta h_{t} and normative posterior p_{t} (computed from Eq. 1 ), which are identical across reasoning conditions for the same game. The bottom block ( agent-dependent ) shows the fitted internal belief state h_{t} and the induced policy \pi_{t} (rows: buttons A–D; columns: rounds and final decision). A: Stubborn ( gpt oss 20b ). Under low-reasoning , gpt oss 20b exhibits a stubborn regime ( \beta<0 ), with h_{t} dominated by early evidence and a comparatively diffuse belief-to-choice mapping; higher reasoning shifts h_{t} toward cleaner integration ( \beta\approx 0 ) and sharpens \pi_{t} (larger \kappa_{f} ). B: Forgetful (Gemini-2.5 Flash). In contrast, Gemini-2.5 Flash is forgetful under low reasoning ( \beta>0 ), over-weighting recent evidence; extended reasoning reduces leakage and sharpens the final policy, yielding more stable integration and more MAP-like choices. Appendix H Model Formulation Details H.1 Introduction We define this model under a set of cognitive assumptions: agents can be biased, forgetful or stubborn, can ignore the occlusions thus making invalid choices and can exploit in a MAP-like way or explore randomly. The goal is to parameterize these departures from an ideal observer, so that each fitted agent can be characterized in terms of (i) how evidence is integrated over time, and (ii) how beliefs are translated into choices. We start by defining the normative Bayesian model and the quantities needed to build a stepwise belief integrator from observations generated by the Active Probabilistic Reasoning Task . This provides a reference point (the ideal observer) against which non-optimal integration and biased choice rules can be measured against. H.1.1 Active Probabilistic Reasoning Task We consider independent games of each with t\in\{1,...,N\} rounds consisting of t sampling rounds followed by one final choice where K choices are potentially available; in our version of the task, K=4 . Latent target and observations. Each game has a latent target z\in\{1,\dots,K\} with prior \Pr(z=k)=p_{0}(k),\hskip 20.00003ptp_{0}(k)>0. (8) At sampling round t\in\{1,\dots,N\} , the agent chooses an arm a_{g,t}\in\{1,\dots,K\} and observes a binary outcome x_{t}\in\{0,1\} (e.g. x=1 = RED, x=0 = GREEN). Shared-rate Bernoulli emission model. We assume a shared-rate Bernoulli model with parameters 0<\alpha_{U}<\alpha_{B}<1 . Defining the Bernoulli pmf \mathrm{Ber}(x;\theta):=\theta^{x}(1-\theta)^{1-x} , the emission model is \Pr(x_{t}=1\mid a_{t}=i,z=k)=\begin{cases}\alpha_{B}&\text{if }k=i,\\ \alpha_{U}&\text{if }k\neq i,\end{cases} (9) \Pr(x_{t}=0\mid a_{t}=i,z=k)=1-\Pr(x_{t}=1\mid a_{t}=i,z=k). (10) Equivalently, \Pr(x_{t}\mid a_{t}=i,z=k)=\begin{cases}\mathrm{Ber}(x_{t};\alpha_{B})&k=i,\\ \mathrm{Ber}(x_{t};\alpha_{U})&k\neq i.\end{cases} (11) Conditional independence across rounds. Given z and the choices, outcomes are conditionally independent across rounds: p(x_{1:N}\mid a_{1:N},z_{n}=k)=\prod_{t=1}^{N}p(x_{t}\mid a_{t},z=k). (12) H.2 Normative Bayes Posterior Let the latent context be z\in\{1,\dots,K\} with prior \bm{p}_{0}\in\Delta^{K},\hskip 20.00003pt(\bm{p}_{0})_{k}=\Pr(z=k). (13) On each game g , the agent makes a sequence of choices a_{1:t} and observes evidence x_{1:t} , so if we assume conditional independence across trials given z : p(x_{1:t}\mid a_{1:t},z=k)=\prod_{r=1}^{t}p(x_{r}\mid a_{r},z=k). (14) Equation ( 14 ) is the (factorized) likelihood of the data up to round t and evidence sequence x_{1:t} under hypothesis z=k , conditional on the choices a_{1:t} . H.2.1 Bayes posterior in scalar form For a single hypothesis k\in\{1,\dots,K\} , define the scalar posterior p_{t}(k)\;:=\;\Pr(z=k\mid a_{1:t},x_{1:t}). (15) Applying Bayes’ rule and using the factorization ( 14 ), \displaystyle p_{t}(k) \displaystyle=\frac{\Pr(z=k)\,p(x_{1:t}\mid a_{1:t},z=k)}{\sum_{j=1}^{K}\Pr(z=j)\,p(x_{1:t}\mid a_{1:t},z=j)} \displaystyle=\frac{(\bm{p}_{0})_{k}\prod_{r=1}^{t}p(x_{r}\mid a_{r},z=k)}{\sum_{j=1}^{K}(\bm{p}_{0})_{j}\prod_{r=1}^{t}p(x_{r}\mid a_{r},z=j)}. (16) Defining the per-round likelihood L_{r}(k)\;:=\;p(x_{r}\mid a_{r},z=k), (17) equation ( 16 ) can be written compactly as p_{t}(k)=\frac{(\bm{p}_{0})_{k}\prod_{r=1}^{t}L_{r}(k)}{\sum_{j=1}^{K}(\bm{p}_{0})_{j}\prod_{r=1}^{t}L_{r}(j)}. (18) The denominator is the normalization over all hypotheses. Equivalently, for any fixed k , \sum_{j=1}^{K}(\bm{p}_{0})_{j}\prod_{r=1}^{t}L_{r}(j)=(\bm{p}_{0})_{k}\prod_{r=1}^{t}L_{r}(k)+\sum_{j\neq k}(\bm{p}_{0})_{j}\prod_{r=1}^{t}L_{r}(j). (19) H.2.2 Stepwise Bayes update The expression ( 16 ) is a “batch” update using the entire likelihood up to time t . Equivalently, we can update step-by-step by incorporating the new observation x_{t} on top of the history H_{t}:=\{(a_{1},x_{1}),\dots,(a_{t},x_{t})\} , so by Bayes’ rule, \displaystyle\Pr(z=k\mid H_{t})=\frac{p(x_{t}\mid H_{t-1},a_{t},z=k)\;\Pr(z=k\mid H_{t-1})}{p(x_{t}\mid H_{t-1},a_{t})}. (20) Under conditional independence, the past does not affect the distribution of the new observation once we condition on the current action and z : p(x_{t}\mid a_{1:t},x_{1:t-1},z=k)=p(x_{t}\mid a_{t},z=k)=:L_{t}(k). (21) Moreover, since z is static and choices are treated as exogenous (i.e. we condition on a_{1:t} as given), the belief about z before observing x_{t} is just the previous posterior: \Pr(z=k\mid a_{1:t},x_{1:t-1})=\Pr(z=k\mid a_{1:t-1},x_{1:t-1})=p_{t-1}(k). (22) Using the law of total probability and the chain rule p(A,B\mid C)=p(A\mid B,C)\,p(B\mid C), we can expand as \displaystyle p(x_{t}\mid a_{1:t},x_{1:t-1})=\sum_{j=1}^{K}p(x_{t},z=j\mid a_{1:t},x_{1:t-1}) \displaystyle=\sum_{j=1}^{K}p(x_{t}\mid a_{1:t},x_{1:t-1},z=j)\;\Pr(z=j\mid a_{1:t},x_{1:t-1}) \displaystyle=\sum_{j=1}^{K}p(x_{t}\mid a_{t},z=j)\;p_{t-1}(j) \displaystyle=\sum_{j=1}^{K}L_{t}(j)\,p_{t-1}(j). (23) Substituting ( 21 ), ( 22 ), and ( 23 ) into ( 20 ) gives p_{t}(k)=\frac{p_{t-1}(k)\,L_{t}(k)}{\sum_{j=1}^{K}p_{t-1}(j)\,L_{t}(j)}. (24) Given that the likelihoods of a mismatch with the true latent variable are shared, we can also write down the step-wise integrator with choice a_{t} as depending on L_{t}^{+}:=p(x_{t}\mid a_{t},z=a_{t}) (25) L_{t}^{-}:=p(x_{t}\mid a_{t},z\neq a_{t}), (26) which leads to the update equation ( 24 ) being rewritten as p_{t}(a_{t})=\frac{p_{t-1}(a_{t})\,L_{t}^{+}}{p_{t-1}(a_{t})\,L_{t}^{+}+\sum_{j\neq a_{t}}p_{t-1}(j)\,L_{t}^{-}}. (27) The definition of L_{t}^{+},L_{t}^{-} will come useful later when we derive log-ratio accumulator model. H.2.3 Bayes update in probability space Define the likelihood vector \bm{L}_{t}\in\mathbf{R}_{+}^{K} by (\bm{L}_{t})_{k}:=p(x_{t}\mid a_{t},z=k), (28) so that (\bm{L}_{t})_{k}=L_{t}(k) in ( 24 ). Then Bayes’ rule is the elementwise update plus renormalization gives us Eq. 1 \bm{p}_{t}=\frac{\bm{p}_{t-1}\odot\bm{L}_{t}}{\mathbf{1}^{\top}(\bm{p}_{t-1}\odot\bm{L}_{t})}, (29) where \odot denotes componentwise multiplication. H.2.4 Log-integrator equation Define the (unnormalized) log-posterior vector at round t \bm{\ell}_{t}:=\log\bm{p}_{0}+\sum_{r=1}^{t}\log\bm{L}_{r},\hskip 20.00003pt\text{equivalently}\hskip 20.00003pt\bm{\ell}_{t}=\bm{\ell}_{t-1}+\log\bm{L}_{t}. (30) Lemma (softmax invariance to common shifts). For any \bm{v}\in\mathbf{R}^{K} and any \alpha\in\mathbf{R} , \sigma(\bm{v}+\alpha\mathbf{1})=\sigma(\bm{v}). (31) Proposition 1. Define \sigma(\bm{v}):=\frac{\exp(\bm{v})}{\mathbf{1}^{\top}\exp(\bm{v})}. (32) Then the Bayesian posterior is \bm{p}_{t}=\sigma(\bm{\ell}_{t}). (33) where \sigma(.) is the softmax operator. Proof. By construction, \exp(\bm{\ell}_{t})=\exp(\log\bm{p}_{0})\odot\prod_{r=1}^{t}\exp(\log\bm{L}_{r})=\bm{p}_{0}\odot\prod_{r=1}^{t}\bm{L}_{r}, whose k th component is p_{0}(k)\prod_{r=1}^{t}p(x_{r}\mid a_{r},z=k) , i.e. Bayes’ numerator. Normalizing across k yields ( 33 ). \square H.3 Log-ratio accumulator In this section we first derive the log-ratio accumulator that allows for step-wise information gathering and then prove that it leads to the Normative Bayes Integrator defined in Appendix Section H.2.4 . H.3.1 Log-ratio evidence Assume binary outcomes x_{t}\in\{0,1\} . Let R\equiv\{x_{t}=1\}\hskip 20.00003ptG\equiv\{x_{t}=0\} (34) Under a Bernoulli likelihood with parameter \theta we can write explicitely \displaystyle\Pr(R\mid\theta)=\Pr(x_{t}=1\mid\theta)=\theta, (35) \displaystyle\Pr(G\mid\theta)=\Pr(x_{t}=0\mid\theta)=1-\theta. (36) We can also define the following equivalences: T:\ z=a_{t}\hskip 10.00002pt\text{(match)},\hskip 20.00003ptF:\ z\neq a_{t}\hskip 10.00002pt\text{(mismatch)}. (37) and assume the Bernoulli rate is \alpha_{B} under T and \alpha_{U} under F , with 0<\alpha_{U}<\alpha_{B}<1 , or more concretely p_{H}(x)=\mathrm{Ber}(x;\alpha_{B}),\hskip 20.00003ptp_{L}(x)=\mathrm{Ber}(x;\alpha_{U}). (38) The two outcome-specific log-evidence increments (r,g) are then \displaystyle r:=\log\frac{\Pr(R\mid T)}{\Pr(R\mid F)}=\log\frac{\alpha_{B}}{\alpha_{U}}, (39) \displaystyle g:=\log\frac{\Pr(G\mid T)}{\Pr(G\mid F)}=\log\frac{1-\alpha_{B}}{1-\alpha_{U}}. (40) A single evidence step is the log-likelihood ratio between T and F : \displaystyle\log\frac{p_{H}(x_{t})}{p_{L}(x_{t})}=x_{t}\, (41) \displaystyle r+(1-x_{t})\,g,r:=\log\frac{\alpha_{B}}{\alpha_{U}}, (42) \displaystyle g:=\log\frac{1-\alpha_{B}}{1-\alpha_{U}}. (43) With these components we can now write in full the (r,g) counters that lead to the accumulator we are looking for, concretely if we let e_{a_{t}}\in\mathbf{R}^{K} be the one-hot action vector, (e_{a_{t}})_{k}=\mathbf{I}[k=a_{t}] we can define a vector counts of successes and failures per arm by \bm{N}_{t}^{(1)}:=\sum_{r=1}^{t}x_{r}\,e_{a_{r}},\hskip 20.00003pt\bm{N}_{t}^{(0)}:=\sum_{r=1}^{t}(1-x_{r})\,e_{a_{r}}, (44) which allows us to reach the accumulator state in closed form as \bm{q}_{t}=\log\bm{p}_{0}+r\,\bm{N}_{t}^{(1)}+g\,\bm{N}_{t}^{(0)}. (45) H.3.2 \delta_{t} evidence recursive equation With the above derivation in mind, we can simplify the integrator equation a bit more so as to make the comparisons between the different formalisms easier. If we recall likelihood terms defined previously in 25 , 26 L_{t}^{+}:=p(x_{t}\mid a_{t},z=a_{t}),\hskip 10.00002ptL_{t}^{-}:=p(x_{t}\mid a_{t},z\neq a_{t}) (46) we can see that the likelihood vector satisfies (\bm{L}_{t})_{k}=p(x_{t}\mid a_{t},z=k)=\begin{cases}L_{t}^{+},&k=a_{t},\\ L_{t}^{-},&k\neq a_{t}.\end{cases} (47) This is equivalent to the initial the Bernoulli dependent definition L_{t}^{+}=p_{H}(x_{t})=\alpha_{B}^{x_{t}}(1-\alpha_{B})^{1-x_{t}}, (48) L_{t}^{-}=p_{L}(x_{t})=\alpha_{U}^{x_{t}}(1-\alpha_{U})^{1-x_{t}}, (49) which implies the equivalence \log\frac{L_{t}^{+}}{L_{t}^{-}}=x_{t}\log\frac{\alpha_{B}}{\alpha_{U}}+(1-x_{t})\log\frac{1-\alpha_{B}}{1-\alpha_{U}}=x_{t}\,r+(1-x_{t})\,g. (50) Substituting into ( 45 ) yields the likelihood-ratio form \bm{q}_{t}=\log\bm{p}_{0}+\sum_{r=1}^{t}\log\frac{L_{r}^{+}}{L_{r}^{-}}\;e_{a_{r}}. (51) Define the scalar log-likelihood ratio (single-step evidence) \delta_{t}:=\log\frac{L_{t}^{+}}{L_{t}^{-}}=\log\frac{p(x_{t}\mid a_{t},z=a_{t})}{p(x_{t}\mid a_{t},z\neq a_{t})}. (52) Then ( 51 ) becomes the simplified accumulator definition \bm{q}_{t}:=\log\bm{p}_{0}+\sum_{r=1}^{t}\delta_{r}\,e_{a_{r}}, (53) or more simply \bm{q}_{t}=\bm{q}_{t-1}+\delta_{t}\,e_{a_{t}}. (54) It is also convenient to write the likelihood vector itself as \bm{L}_{t}=L_{t}^{-}\mathbf{1}+(L_{t}^{+}-L_{t}^{-})\,e_{a_{t}}, and in log space (component-wise) \log\bm{L}_{t}=\log(L_{t}^{-})\,\mathbf{1}+\delta_{t}\,e_{a_{t}}. (55) H.3.3 Recovering the original log-integrator Recall from Section 1 that the unnormalized log-posterior (log-integrator) is \bm{\ell}_{t}=\log\bm{p}_{0}+\sum_{r=1}^{t}\log\bm{L}_{r} ( 30 ). Plugging ( 55 ) into this definition gives \bm{\ell}_{t}=c_{t}\,\mathbf{1}+\bm{q}_{t},\hskip 20.00003ptc_{t}:=\sum_{r=1}^{t}\log(L_{r}^{-}). (56) Corollary 2.1 (accumulator posterior is Bayesian). Combining ( 33 ), ( 56 ), and ( 31 ), \bm{p}_{t}=(\bm{\ell}_{t})=\sigma(c_{t}\mathbf{1}+\bm{q}_{t})=\sigma(\bm{q}_{t}). (57) H.4 Memory Bayesian Integrator Up to this point we derived several equivalent representations of the Bayesian posterior: the log-integrator \bm{\ell}_{t} ( 30 ), log-ratio accumulator \bm{q}_{t} ( 54 ), and their relation \bm{\ell}_{t}=c_{t}\mathbf{1}+\bm{q}_{t} ( 56 ). These representations are not unique, because the posterior depends only on relative log-beliefs: by Lemma ( 31 ), \sigma(\bm{v}+\alpha\mathbf{1})=\sigma(\bm{v})\hskip 20.00003pt(\alpha\in\mathbf{R}), so any global shift along \mathbf{1} is a non-informative degree of freedom. Concretely, the scalar term c_{t}\mathbf{1} in ( 56 ) carries no information about z and cancels under \sigma(\cdot) . A natural way to remove this invariance is to project log-beliefs onto the (K-1) -dimensional subspace orthogonal to \mathbf{1} . We therefore introduce a centered log state \bm{h}_{t} defined as the unique representative of the equivalence class \bm{\ell}_{t}+\mathrm{span}\{\mathbf{1}\} with zero mean across components. Concretely, \bm{h}_{t}=\bm{\ell}_{t}-\tfrac{1}{K}(\mathbf{1}^{\top}\bm{\ell}_{t})\mathbf{1}, which can be written compactly as \bm{h}_{t}=C\bm{\ell}_{t} for an idempotent centering projection C . Working in centered space has two practical advantages for the modeling that follows: (i) it removes unidentifiable global offsets that would otherwise interact with forgetting parameters, and (ii) under the shared-negative-evidence structure ( 47 ), the evidence update becomes a simple state-dependent impulse proportional to the \delta_{t} integrator form. We now can formalize this centering operator and derive the centered (state-dependent) recursion that forms the base model for the memory defined in Section 7 . Considering the centering projection as C:=I-\frac{1}{K}\mathbf{1}\mathbf{1}^{\top},\hskip 20.00003ptC\mathbf{1}=\bm{0},\hskip 20.00003ptC^{2}=C. (58) Define the centered log state \bm{h}_{t}:=C\,\bm{\ell}_{t}. (59) From ( 30 ), \bm{h}_{t}=C\bm{\ell}_{t}=C(\bm{\ell}_{t-1}+\log\bm{L}_{t})=\bm{h}_{t-1}+C\log\bm{L}_{t}. (60) H.4.1 Equivalence with the accumulator \bm{q}_{t} Using ( 55 ) and C\mathbf{1}=\bm{0} , C\log\bm{L}_{t}=C\big(\log(L_{t}^{-})\mathbf{1}+\delta_{t}e_{a_{t}}\big)=\delta_{t}\,Ce_{a_{t}}=\delta_{t}\Big(e_{a_{t}}-\tfrac{1}{K}\mathbf{1}\Big). (61) Therefore the centered (state-dependent) integrator is \bm{h}_{t}=\bm{h}_{t-1}+\delta_{t}\Big(e_{a_{t}}-\tfrac{1}{K}\mathbf{1}\Big). (62) From ( 56 ) and C\mathbf{1}=\bm{0} , \bm{h}_{t}=C\bm{\ell}_{t}=C(c_{t}\mathbf{1}+\bm{q}_{t})=C\bm{q}_{t}. (63) So \bm{h}_{t} is exactly the centered version of the accumulator. Conversely, because \mathrm{null}(C)=\mathrm{span}\{\mathbf{1}\} , \bm{h}_{t} determines \bm{q}_{t} only up to a common shift: \bm{h}_{t}=C\bm{q}_{t}\hskip 10.00002pt\Longleftrightarrow\hskip 10.00002pt\exists\alpha_{t}\in\mathbf{R}:\ \bm{q}_{t}=\bm{h}_{t}+\alpha_{t}\mathbf{1}. (64) Using ( 57 ), ( 64 ), and ( 31 ), \bm{p}_{t}=\sigma(\bm{q}_{t})=\sigma(\bm{h}_{t}+\alpha_{t}\mathbf{1})=\sigma(\bm{h}_{t}). (65) Since C\bm{q}_{t}=\bm{h}_{t} and \mathrm{null}(C)=\mathrm{span}\{\mathbf{1}\} , we have \bm{q}_{t}=\bm{h}_{t}+\alpha_{t}\mathbf{1} for some scalar \alpha_{t} . Choosing the canonical representative induced by C (mean-zero \bm{h}_{t} ) gives \alpha_{t}=\frac{1}{K}\mathbf{1}^{\top}\bm{q}_{t}. (66) Using the accumulator form \bm{q}_{t}=\log\bm{p}_{0}+\sum_{r=1}^{t}\delta_{r}\,e_{a_{r}} and \mathbf{1}^{\top}_{a_{r}}=1 , \displaystyle\alpha_{t}=\frac{1}{K}\mathbf{1}^{\top}\log\bm{p}_{0}+\frac{1}{K}\sum_{r=1}^{t}\delta_{r}, (67) \displaystyle\alpha_{t}=\alpha_{t-1}+\frac{\delta_{t}}{K}. (68) \alpha_{t}\mathbf{1} is a common shift applied to all coordinates of \bm{q}_{t} and therefore carries no information about relative odds; it is removed by centering and cancels under \sigma ( 31 ). Hence \bm{h}_{t} differs from \bm{q}_{t} (and \bm{\ell}_{t} ) only by an additive term in \mathrm{span}\{\mathbf{1}\} : it retains exactly the evidence that changes the posterior, while discarding a global baseline drift that does not affect \bm{p}_{t} . We can thus show that, under the shared-negative-evidence assumption ( 47 ), \bm{p}_{t}=\sigma(\bm{\ell}_{t})=\sigma(\bm{q}_{t})=\sigma(\bm{h}_{t}) (69) where \bm{q}_{t}=\bm{q}_{t-1}+\delta_{t}e_{a_{t}},\hskip 20.00003pt\bm{h}_{t}=\bm{h}_{t-1}+\delta_{t}\Big(e_{a_{t}}-\tfrac{1}{K}\mathbf{1}\Big). Thus starting from the accumulator or starting from the centered integrator yields the same Bayesian posterior. H.4.2 Evidence increments in centered log space Within game g , the latent target z\in\{1,\dots,K\} is fixed and the agent produces a sequence of choices and outcomes H_{g}:=(a_{g,1},x_{g,1}),\dots,(a_{g,t},x_{g,t}) . All belief states and policies below are defined per game. For a given game and round t , define the likelihood vector exactly as in Section 2: (\bm{L}_{t})_{k}:=p(x_{t}\mid a_{t},z_{n}=k),\hskip 20.00003pt\bm{L}_{t}\in\mathbf{R}_{+}^{K}. (70) Define the Bayes-centered log state \bm{h}_{t}:=C\bm{\ell}_{t} as in ( 59 ), and its increment \Delta\bm{h}_{t}:=\bm{h}_{t}-\bm{h}_{t-1}. (71) From the centered recursion ( 60 ), this increment is just the centered log-likelihood: \Delta\bm{h}_{t}=C\log\bm{L}_{t}. (72) Under the shared-negative-evidence structure ( 47 ), introduce the match/mismatch likelihoods (identical to ( 25 )–( 26 ) but with indices t ) \displaystyle L_{t}^{+}:=p(x_{t}\mid a_{t},z=a_{t}), (73) \displaystyle L_{t}^{-}:=p(x_{t}\mid a_{t},z\neq a_{t}), (74) and the single-step log-likelihood ratio (LLR) ( 52 ) \delta_{t}:=\log\frac{L_{t}^{+}}{L_{t}^{-}}=\log\frac{p(x_{t}\mid a_{t},z=a_{t})}{p(x_{t}\mid a_{t},z\neq a_{t})}. (75) Then the decomposition ( 55 ) together with C\mathbf{1}=\bm{0} implies that the increment in centered space depends only on the discriminative LLR term: \Delta\bm{h}_{t}=\delta_{t}\,Ce_{a_{t}}=\delta_{t}\Big(e_{a_{t}}-\tfrac{1}{K}\mathbf{1}\Big). (76) This is exactly the game-indexed version of the centered update ( 61 ). H.4.3 Memory state update The exact Bayesian centered state satisfies \bm{h}_{t}=\bm{h}_{t-1}+\Delta\bm{h}_{t} . To model forgetting, we replace this perfect accumulation by a recursion on an internal state \bm{h}_{t}\in\mathbf{R}^{K} : \displaystyle\bm{h}_{0}:=C\log\bm{p}_{0}, (77) \displaystyle\bm{h}_{t}:=(1-\beta)\,\bm{h}_{t-1}+\Delta\bm{h}_{t}, (78) with t=\{1,\dots,t\},\beta\in[0,1] . Here \beta controls memory decay: \beta=0 recovers the exact Bayesian centered integrator (because then \bm{h}_{t}=\bm{h}_{t} ), while larger \beta discounts older evidence more strongly. Since both \bm{h}_{n,0} and \Delta\bm{h}_{t} lie in the centered subspace (orthogonal to \mathbf{1} ), \bm{h}_{t} remains centered for all t . The internal posterior used for decision-making is the softmax map applied to the memory state: \bar{\bm{p}}_{t}:=\sigma(\bm{h}_{t}). (79) When \beta=0 , this coincides with the Bayesian posterior \bm{p}_{t} (cf. ( 65 )). This same update can be also written as an exponential kernel \rho:=1-\beta\in[0,1] where by unrolling ( 78 ) yields the causal exponential-kernel representation \bm{h}_{t}=\rho^{t}\bm{h}_{0}+\sum_{r=1}^{t}\rho^{t-r}\,\Delta\bm{h}_{r}. (80) H.5 Internal posterior and Policy Following bounded-rationality formulations with information-processing costs [ 69 ] , we model choice as a Gibbs/Boltzmann policy \pi_{x}(j)\;\propto\;\pi_{0,x}(j)\,\exp\!\big(\kappa_{x}\,U_{x}(j)\big),\hskip 20.00003pt\pi_{x}(j)=\frac{1}{Z_{x}}\,\pi_{0,x}(j)\,\exp\!\big(\kappa_{x}\,U_{x}(j)\big), (81) where \pi_{0,x} is a reference (prior) policy, U_{x} is a utility, and \kappa_{x} is an inverse-temperature controlling the strength of exploitation (random as \kappa_{x}\!\to\!0 , increasingly greedy as \kappa_{x} grows). H.5.1 Internal posterior In our setting, the memory state \bm{h}_{t} is a centered log-belief (defined up to an additive constant). We choose the bounded-rational utility to be the log-belief coordinate itself, U_{x}(j):=\big(\bm{h}_{\tau_{x}(t)}\big)_{j}, (82) so that evidence accumulated in \bm{h} directly determines action values. With a uniform reference policy, ( 81 ) yields the internal posterior \bar{\bm{p}}^{\,x}_{t}:=\sigma\!\big(\kappa_{x}\,\bm{h}_{\tau_{x}(t)}\big)=\frac{\exp(\kappa_{x}\,\bm{h}_{\tau_{x}(t)})}{\mathbf{1}^{\top}\exp(\kappa_{x}\,\bm{h}_{\tau_{x}(t)})}\;\in\;\Delta^{K}, (83) which is equivalent to \bar{\bm{p}}^{\,x}_{t}=\sigma(\kappa_{x}\bm{h}_{t}) (up to the phase/time index \tau_{x}(t) ). This makes clear that \kappa_{x} interpolates between near-uniform behavior ( \kappa_{x}\!\to\!0 ), posterior-proportional behavior ( \kappa_{x}\!=\!1 ), and MAP-like concentration as \kappa_{x}\!\to\!\infty . (Equivalently, since \sigma(\bm{h})\propto\exp(\bm{h}) , ( 83 ) is identical to raising \sigma(\bm{h}_{\tau_{x}(t)}) to a power and renormalizing.) H.5.2 Occlusion mask During sampling, not all actions are available: let \mathcal{A}_{g,t}\subseteq\{1,\dots,K\} denote the set of visible (valid) buttons at round t in game g . We encode availability by a multiplicative mask \bm{m}^{\,s}_{g,t}\in\mathbb{R}_{>0}^{K} with entries m^{s}_{g,t,j}:=\theta_{\mathrm{occ}}^{-\mathbf{1}\{j\notin\mathcal{A}_{g,t}\}},\hskip 20.00003ptj\in\{1,\dots,K\}, (84) so visible actions have weight 1 and occluded actions are down-weighted by 1/\theta_{\mathrm{occ}} . Larger \theta_{\mathrm{occ}} therefore corresponds to stronger occlusion awareness (greater avoidance of invalid actions). At the final decision round, all actions are available and we set \bm{m}^{\,f}_{g,t}\equiv\mathbf{1} . Bias and policy. We incorporate systematic preferences and availability constraints through a phase-specific reference policy \pi_{0,x}(j)\propto\omega_{x}(j)\,m^{x}_{t}(j) , where \bm{\omega}_{x}\in\Delta^{K} is a choice-bias vector and m^{x}_{t} is an occlusion mask (with m^{f}_{t}\equiv\mathbf{1} ). Combining ( 81 ) and ( 83 ) yields the full policy \bm{\pi}^{\,x}_{t}=\frac{\bar{\bm{p}}^{\,x}_{t}\odot\bm{\omega}_{x}\odot\bm{m}^{x}_{t}}{\mathbf{1}^{\top}\!\big(\bar{\bm{p}}^{\,x}_{t}\odot\bm{\omega}_{x}\odot\bm{m}^{x}_{t}\big)}. (85) H.6 Model loss function For each agent (human or LLM condition), the dataset consists of G games. Game g provides sampling choices \{a_{g,t}\}_{t=1}^{T_{g}} , observed outcomes \{x_{g,t}\}_{t=1}^{T_{g}} , occlusion sets \{\mathcal{A}_{g,t}\}_{t=1}^{T_{g}} (sampling only), and a final inference choice f_{g} . Given parameters \Theta:=(\bm{\omega}_{s},\bm{\omega}_{f},\kappa_{s},\kappa_{f},\beta,\theta), (86) we unroll the memory trajectory \{\bm{h}_{g,t}\} deterministically from the observed history using the memory update (Eq. ( 4 )) and the corresponding evidence injection \Delta\bm{h}_{g,t} . At each sampling round t we compute the internal posterior \bar{\bm{p}}^{\,s}_{g,t} from the pre-observation state \bm{h}_{g,t-1} (Eq. ( 83 )) and form the sampling policy \bm{\pi}^{\,s}_{g,t} via Eq. ( 85 ) using the occlusion mask \bm{m}^{s}_{g,t} (derived from \mathcal{A}_{g,t} and \theta ). After the last update, we compute the final policy \bm{\pi}^{\,f}_{g} from \bm{h}_{g,T_{g}} with \bm{m}^{f}\equiv\mathbf{1} . The conditional log-likelihood of the observed choices under the induced policies is \mathcal{L}(\Theta)=\sum_{g=1}^{G}\Bigg[\sum_{t=1}^{T_{g}}\log\big(\bm{\pi}^{\,s}_{g,t}\big)_{a_{g,t}}\;+\;\log\big(\bm{\pi}^{\,f}_{g}\big)_{f_{g}}\Bigg]+\Omega(\Theta) (87) and we estimate \Theta by minimizing a regularized negative log-likelihood [ 21 , 11 ] with a simple quadratic regularizer (one choice among many) \Omega(\Theta)=\lambda_{\omega}\!\sum_{x\in\{s,f\}}\Big\|\bm{\omega}_{x}-\tfrac{1}{K}\mathbf{1}\Big\|_{2}^{2}+\lambda_{\kappa}(\kappa_{s}^{2}+\kappa_{f}^{2})+\lambda_{\beta}(\beta-\beta_{0})^{2}+\lambda_{\mathrm{occ}}\big(\log\theta_{\mathrm{occ}}\big)^{2}, (88) where \lambda_{\omega},\lambda_{\kappa},\lambda_{\beta},\lambda_{\mathrm{occ}}\geq 0 and \beta_{0} is a chosen reference value. Appendix I Fitting Results For each agent (humans and each LLM) and reasoning mode ( Base / Extended ), Appendix Figs. 9 – 12 visualize (i) posterior trajectories during inference and sampling , and (ii) choice biases alongside the fitted bias weights \{\bm{\omega}_{s},\bm{\omega}_{f}\} . The same quantities are summarized as heatmaps in Appendix Fig. 13 . Full fitted parameter values are reported in Table J (corresponding to Fig. 3 in the main text). Posterior curves. All posterior panels compress the full K -dimensional belief vector into a single scalar by reading out the posterior mass assigned to the choice that was actually taken. Concretely, let \bm{p}_{g,t}\in\Delta^{K} denote the (normative) Bayesian posterior after t samples in game g . Then for inference panels (columns 1–2), the plotted quantity is the posterior probability of the final chosen option , evaluated as evidence accumulates across rounds. While for sampling panels (columns 3–4), the plotted quantity is the posterior probability of the sampling action chosen on round t , evaluated just before the sample is taken on that round (so that it reflects the beliefs that informed the sampling decision). Curves are grouped by trial length N\in\{2,\dots,15\} . For a fixed N , we compute the scalar trajectory above for every game of length N and then average across games, producing one mean trajectory per N (shown as a family of curves, ordered by N ). Thus, each line summarizes the average posterior dynamics conditional on game length for inference and sampling . True posteriors are computed from the observed trajectories of each agent, and fitted posteriors from trajectories simulated by the fitted model, in both cases using the same normative Bayesian update (Eq. 1 ). Bias panels. The bias panels compare empirical choice frequencies against the fitted bias weights. Empirical bars report how often each option is chosen (separately for sampling and inference), while the fitted bars show the learned bias vectors \bm{\omega}_{s} and \bm{\omega}_{f} (or their summary statistic, depending on the plot), which capture systematic preferences not explained by the posterior. Important note. The “fitted posterior” curves are not directly optimized by the likelihood. They are emergent summaries obtained by running the fitted model, collecting its actions, and then reconstructing the same posterior-readout trajectories from those actions (analogous to the empirical construction). Figure 9: Posterior evolution by rounds and bias across humans and language models. In each row the first two panels represent the true and model generated posterior evolution of the final choice inference followed by the true and model generated posterior for the sampling choices sampling . The remaining panels show the true bias for sampling and inference and their corresponding fitted bias weights. Figure 10: Posterior evolution by rounds and bias across humans and language models. In each row the first two panels represent the true and model generated posterior evolution of the final choice inference followed by the true and model generated posterior for the sampling choices sampling . The remaining panels show the true bias for sampling and inference and their corresponding fitted bias weights. Figure 11: Posterior evolution by rounds and bias across humans and language models. In each row the first two panels represent the true and model generated posterior evolution of the final choice inference followed by the true and model generated posterior for the sampling choices sampling . The remaining panels show the true bias for sampling and inference and their corresponding fitted bias weights. Figure 12: Posterior evolution by rounds and bias across humans and language models. In each row the first two panels represent the true and model generated posterior evolution of the final choice inference followed by the true and model generated posterior for the sampling choices sampling . The remaining panels show the true bias for sampling and inference and their corresponding fitted bias weights. Figure 13: Posterior and bias structure is captured by the mechanistic model. For each agent (rows), we compute phase-specific summaries during sampling (left block) and inference (middle block), and compare empirical estimates ( True ) to quantities generated by the fitted mechanistic model ( Model ). Sampling : heatmaps show the agent’s True Posterior signal (normative posterior mass assigned to the agent’s eventual choice under the game history) alongside the corresponding Model Posterior obtained by unrolling the fitted belief dynamics. Inference : analogous comparison for the final inference phase. Bias : right block contrasts True Bias (empirical action preference beyond belief) with Model Bias recovered from the multiplicative bias parameters. Across agents, the model reproduces both the magnitude and qualitative structure of posterior dynamics and systematic biases, supporting the claim that (\beta,\kappa,\omega,\theta) provide a sufficient explanation of the behavioral signatures. Appendix J Model Fitting Parameters Table 1: Fitted parameters of the model Model Reasoning \omega_{s} \omega_{f} \beta \kappa_{s} \kappa_{f} \log\theta PPO MAP Base [0.304, 0.168, 0.367, 0.161] [0.273, 0.215, 0.282, 0.231] -0.016 0.607 9.971 7.449 Humans Base [0.275, 0.264, 0.235, 0.227] [0.251, 0.259, 0.249, 0.240] -0.028 0.378 1.179 6.977 Humans Extended [0.271, 0.284, 0.215, 0.230] [0.243, 0.259, 0.243, 0.254] -0.061 0.616 1.697 7.166 gpt-5-mini Base [0.564, 0.300, 0.109, 0.028] [0.489, 0.308, 0.152, 0.052] -0.036 0.000 0.706 2.336 gpt-5-mini Extended [0.350, 0.283, 0.241, 0.127] [0.363, 0.294, 0.216, 0.126] 0.043 0.000 2.249 3.226 grok-3-mini Base [0.283, 0.330, 0.203, 0.183] [0.388, 0.314, 0.168, 0.130] 0.014 0.000 1.971 3.668 grok-3-mini Extended [0.310, 0.293, 0.224, 0.173] [0.358, 0.333, 0.188, 0.121] 0.002 0.063 2.147 4.153 gemini-2.5-flash Base [0.727, 0.165, 0.078, 0.030] [0.632, 0.197, 0.105, 0.066] 0.140 0.000 0.357 3.672 gemini-2.5-flash Extended [0.384, 0.270, 0.199, 0.147] [0.412, 0.290, 0.172, 0.127] 0.003 0.046 1.755 3.344 gemini-2.5-pro Base [0.580, 0.245, 0.132, 0.043] [0.473, 0.251, 0.148, 0.128] -0.032 0.000 0.228 7.261 gemini-2.5-pro Extended [0.482, 0.248, 0.167, 0.103] [0.530, 0.203, 0.148, 0.119] -0.063 0.121 1.377 6.788 gpt-oss-120b Base [0.418, 0.179, 0.375, 0.029] [0.462, 0.227, 0.283, 0.027] 0.047 0.000 0.983 1.197 gpt-oss-120b Extended [0.299, 0.275, 0.209, 0.217] [0.427, 0.232, 0.171, 0.170] 0.004 0.041 1.660 1.827 gpt-5 Base [0.526, 0.283, 0.123, 0.068] [0.410, 0.306, 0.162, 0.123] 0.044 0.000 0.565 2.024 gpt-5 Extended [0.302, 0.279, 0.231, 0.188] [0.423, 0.244, 0.188, 0.144] -0.036 0.081 2.336 6.828 gpt-oss-20b Base [0.359, 0.296, 0.206, 0.139] [0.340, 0.268, 0.218, 0.174] -0.401 0.000 0.026 1.222 gpt-oss-20b Extended [0.331, 0.293, 0.216, 0.160] [0.429, 0.265, 0.185, 0.122] 0.020 0.000 1.821 1.429 claude-sonnet-4 Base [0.377, 0.270, 0.199, 0.155] [0.558, 0.205, 0.139, 0.098] -0.013 0.000 2.413 3.696 glm-4.5 Base [0.270, 0.415, 0.228, 0.086] [0.418, 0.323, 0.174, 0.085] 0.028 0.000 0.287 1.595 glm-4.5 Extended [0.334, 0.270, 0.254, 0.142] [0.401, 0.276, 0.228, 0.095] -0.013 0.010 1.988 2.685 DeepSeek-R1-Qwen3-8B Base [0.250, 0.319, 0.304, 0.127] [0.210, 0.298, 0.337, 0.155] -0.004 0.000 0.787 2.202 gpt4.1-mini Base [0.575, 0.276, 0.081, 0.069] [0.548, 0.230, 0.104, 0.119] -0.044 0.039 0.697 2.080 claude-haiku-3.5 Base [0.179, 0.366, 0.362, 0.093] [0.224, 0.412, 0.269, 0.095] 0.039 0.000 0.618 2.333 Qwen3 235B Instruct Base [0.444, 0.340, 0.143, 0.073] [0.410, 0.322, 0.158, 0.110] 0.673 0.000 1.572 3.506 gpt4o-mini Base [0.574, 0.305, 0.094, 0.027] [0.390, 0.403, 0.139, 0.068] -0.438 0.011 0.043 3.419 Qwen 14B Instruct Base [0.466, 0.261, 0.141, 0.132] [0.386, 0.223, 0.130, 0.261] 0.492 0.000 1.126 3.773 Llama-4-Scout-Instruct Base [0.588, 0.174, 0.123, 0.116] [0.512, 0.228, 0.107, 0.152] 0.404 0.000 0.738 1.967 Qwen 7B Instruct Base [0.356, 0.290, 0.211, 0.143] [0.535, 0.225, 0.135, 0.105] 0.316 0.000 0.316 1.555 Llama-3.1-405b-Instruct Base [0.472, 0.309, 0.143, 0.076] [0.481, 0.282, 0.132, 0.104] 0.912 0.000 0.408 3.435 Llama-4-Maverick-Instruct Base [0.498, 0.237, 0.181, 0.084] [0.419, 0.248, 0.206, 0.126] 1.000 0.000 0.206 2.152 Llama 3 3 70B Instruct Base [0.396, 0.374, 0.142, 0.088] [0.376, 0.354, 0.146, 0.123] 0.971 0.000 0.000 1.824 Gemma 2B Instruct Base [0.351, 0.212, 0.323, 0.115] [0.268, 0.231, 0.374, 0.128] 0.009 0.000 0.060 1.756 Centaur 70B Base [0.799, 0.179, 0.018, 0.005] [0.726, 0.264, 0.005, 0.005] 0.992 0.000 0.000 2.420 Centaur 8B Base [0.346, 0.250, 0.289, 0.115] [0.294, 0.175, 0.324, 0.207] -1.000 0.003 0.016 2.349 Qwen3 32B Base [0.502, 0.295, 0.157, 0.047] [0.412, 0.322, 0.201, 0.065] 0.996 0.000 0.000 2.435 Apertus 70B Instruct Base [0.304, 0.289, 0.193, 0.214] [0.231, 0.307, 0.188, 0.274] -0.939 0.000 0.006 1.752 Apertus 8B Instruct Base [0.389, 0.275, 0.187, 0.149] [0.225, 0.331, 0.161, 0.283] 0.999 0.000 0.000 0.948 Qwen 3B Instruct Base [0.283, 0.241, 0.284, 0.192] [0.256, 0.172, 0.290, 0.282] -0.317 0.000 0.000 0.879 Llama-3.2-3B-Instruct Base [0.353, 0.224, 0.219, 0.203] [0.162, 0.172, 0.308, 0.358] 0.279 0.000 0.253 1.330 Gemma 2B Base [0.284, 0.379, 0.272, 0.065] [0.334, 0.337, 0.242, 0.087] 0.993 0.000 0.000 0.603 Llama-3.1-8B-Instruct Base [0.921, 0.048, 0.026, 0.005] [0.754, 0.107, 0.110, 0.029] 0.985 0.000 0.000 2.302 Table 2: Negative log-likelihoods of the mechanistic model fitsnot . Model Reasoning Train NLL Test NLL NLL PPO MAP Base 0.576 0.580 0.580 Humans Base 0.724 0.724 0.724 Humans Extended 0.627 0.630 0.630 gpt-5-mini Base 0.540 0.569 0.569 gpt-5-mini Extended 0.593 0.634 0.634 grok-3-mini Base 0.605 0.605 0.605 grok-3-mini Extended 0.641 0.631 0.631 gemini-2.5-flash Base 0.466 0.495 0.495 gemini-2.5-flash Extended 0.613 0.612 0.612 gemini-2.5-pro Base 0.534 0.518 0.518 gemini-2.5-pro Extended 0.551 0.574 0.574 gpt-oss-120b Base 0.869 0.874 0.874 gpt-oss-120b Extended 0.737 0.734 0.734 gpt-5 Base 0.651 0.676 0.676 gpt-5 Extended 0.586 0.592 0.592 gpt-oss-20b Base 0.976 0.960 0.960 gpt-oss-20b Extended 0.825 0.833 0.833 claude-sonnet-4 Base 0.583 0.583 0.583 glm-4.5 Base 0.845 0.849 0.849 glm-4.5 Extended 0.779 0.786 0.786 DeepSeek-R1-0528-Qwen3-8B Base 0.765 0.772 0.772 gpt4.1-mini Base 0.595 0.600 0.600 claude-haiku-3.5 Base 0.657 0.679 0.679 Qwen3 235B Instruct Base 0.585 0.602 0.602 gpt4o-mini Base 0.495 0.444 0.444 Qwen 14B Instruct Base 0.625 0.642 0.642 Llama-4-Scout-Instruct Base 0.728 0.759 0.759 Qwen 7B Instruct Base 0.825 0.848 0.848 Llama-3.1-405b-Instruct Base 0.628 0.626 0.626 Llama-4-Maverick-Instruct Base 0.748 0.742 0.742 Llama 3 3 70B Instruct Base 0.722 0.723 0.723 Gemma 2B Instruct Base 0.774 0.799 0.799 Centaur 70B Base 0.427 0.469 0.469 Centaur 8B Base 0.726 0.725 0.725 Qwen3 32B Base 0.607 0.589 0.589 Apertus 70B Instruct Base 0.804 0.790 0.790 Apertus 8B Instruct Base 1.045 1.048 1.048 Qwen 3B Instruct Base 1.113 1.153 1.153 Llama-3.2-3B-Instruct Base 1.030 1.026 1.026 Gemma 2B Base 1.171 1.177 1.177 Llama-3.1-8B-Instruct Base 0.500 0.546 0.546 Table 2: Negative log-likelihoods (fit summary).