Large Language Models are Near-Optimal Decision-Makers with a Non-Human Learning Behavior

Hao Li, Gengrui Zhang, Petter Holme, Shuyue Hu, Zhen Wang · 2025

Read Large Language Models are Near-Optimal Decision-Makers with a Non-Human Learning Behavior by Hao Li, Gengrui Zhang, Petter Holme, Shuyue Hu, Zhen Wang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-16163.txt.

Introduction

Hao Li and co-authors (2025) compared how language models and people learn to decide. Five leading models took three classic psychology tasks. The tasks probe handling of uncertainty, risk, and shifting rules. Three hundred sixty newly recruited people took the same tasks. Across all three, the models often beat the humans and came close to the best possible scores. They managed uncertainty, sized up risk, and adapted when rules changed.

The path to those scores, though, was not human. The models' learning patterns diverged from the people's in basic ways. The authors read this as a double-edged result. Skill at the outcome level does not mean judgment like ours underneath. Handing decisions to such systems, they warn, needs care and more study. The paper offers a benchmark for comparing machine and human decision learning as delegation grows.

Catalog record

CREATORHao Li, Gengrui Zhang, Petter Holme, Shuyue Hu, Zhen Wang
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE79 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-16163

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Large Language Models are Near-Optimal Decision-Makers with a Non-Human Learning Behavior Hao Li, Gengrui Zhang, Petter Holme, Shuyue Hu, Zhen Wang Abstract Human decision-making belongs to the foundation of our society and civilization, but we are on the verge of a future where much of it will be delegated to artificial intelligence. The arrival of Large Language Models (LLMs) has transformed the nature and scope of AI-supported decision-making; however, the process by which they learn to make decisions, compared to humans, remains poorly understood. In this study, we examined the decision-making behavior of five leading LLMs across three core dimensions of real-world decision-making: uncertainty, risk, and set-shifting. Using three well-established experimental psychology tasks designed to probe these dimensions, we benchmarked LLMs against 360 newly recruited human participants. Across all tasks, LLMs often outperformed humans, approaching near-optimal performance. Moreover, the processes underlying their decisions diverged fundamentally from those of humans. On the one hand, our finding demonstrates the ability of LLMs to manage uncertainty, calibrate risk, and adapt to changes. On the other hand, this disparity highlights the risks of relying on them as substitutes for human judgment, calling for further inquiry. † † footnotetext: These authors contributed equally; this work was partly done during their internship at Shanghai Artificial Intelligence Laboratory. ‡ ‡ footnotetext: Corresponding authors. keywords Decision-making, Artificial intelligence, Large language models, Experimental psychology Decision-making is a unifying theme of almost all the social and behavioral sciences. Moreover, it has been closely tied to artificial intelligence (AI), ever since the inception of the latter. It is no coincidence that one of AI’s founders, Herbert Simon, began his career studying organizational decision-making Simon 1947 . Simon once pointed out that “the capacity of the human mind for formulating and solving complex problems is very small compared with the size of the problems whose solution is required for objectively rational behavior in the real world” Simon 1947 . And so the first generation of AI built decision systems Turban and Watkins 1986 by implementing logic and circumstantial knowledge. In contrast, the last decade of AI development has had a very different driving force—to imitate human intellectual output in broad generality. By training self-supervised artificial neural networks on immense textual corpora—so-called Large Language Models (LLMs)—it is commonly accepted that AI has now passed the Turing test Jones and Bergen 2025 ; Mei et al. 2024 . LLMs do make decisions if we prompt them to do so, but are those decisions made with a capacity as “very small” as the humans they try to emulate? That is the central question we pursue in this paper. LLMs are remarkably versatile, capable of generating meaningful output across a wide range of tasks, and are, unsurprisingly, already being deployed in real-world settings to make or influence decisions Bommasani et al. 2021 ; Handler et al. 2024 ; Chen et al. 2024 . For a concrete example, in a legal case in Colombia concerning the medical expenses of an autistic boy, the judge not only queried ChatGPT, asking “Is an autistic minor exonerated from paying fees for their therapies?”, but also cited both the prompt and the answer in the official ruling Taylor 2023 . Likewise, LinkedIn, a major professional networking platform, has deployed an LLM-based chatbot to assist recruiters in automating critical hiring decisions, such as the shortlisting of job candidates, thereby directly shaping individuals’ career opportunities Alba and Yin 2024 . Globally, surveys estimate that 71% of major organizations have already regularly used generative AI in at least one business function Singla et al. 2025 . For better or worse, generative AI is rapidly becoming integral to decision-making infrastructures, where each decision made, like those made by humans, has the power to shape individual lives and, potentially, ripple outward to influence entire societies Noy and Zhang 2023 ; Extance 2023 ; Williams et al. 2024 ; Goh et al. 2025 . However, are generative AI systems truly ready to assist or even replace humans in decision-making? Recent studies have reported strong performance for the decision-making of these systems in domains such as healthcare Singhal et al. 2023 ; Hager et al. 2024 , finance Chen et al. 2021 ; Xie et al. 2024 , and strategic games Wang et al. 2024 ; Tan et al. 2024 ; Akata et al. 2025 ; Wang et al. 2024 . Yet, arguably more critical—but far less explored—is how they arrive at these decisions, particularly when operating outside the narrowly defined scenarios. Domain-specific tasks, though valuable, often introduce numerous confounds, such as background knowledge, ethical considerations, and cultural biases, which can obscure the working mechanisms that drive system behaviors. Moreover, although these tasks capture real-world complexity, they often conflate conceptually distinct dimensions of decision-making, such as risk and uncertainty. For instance, a medical diagnosis benchmark may fold epistemic uncertainty about a patient’s true condition into the known risks of various treatments Hager et al. 2024 . Consequently, even strong performance on these tasks offers little diagnostic insight into whether the system effectively excels at managing uncertainty, calibrating risk, or both, let alone how these dimensions are reasoned about. In contrast, what fundamentally characterizes human decision-making capacity and their agency is not merely the correctness of a task-specific choice, but the ability to navigate different decision dimensions while simultaneously aligning their choices with goals Johnson and Busemeyer 2010 ; Hastie and Dawes 2010 ; Einhorn and Hogarth 1981 . In this study, we compare the decision-making of generative AI systems and humans, particularly under conditions of uncertainty, risk, or set-shifting. Here, decision-making refers to the process by which individuals assess multiple alternatives and choose actions that align with defined goals. Uncertainty involves acting with incomplete information and ambiguous future outcomes, requiring a balance between short-term and long-term consequences Knight 1921 ; Camerer and Weber 1992 . Risk entails evaluating potential gains and losses based on known probabilities, demanding careful judgment of outcome likelihoods Machina 1987 ; Kahneman and Tversky 2013 . Set-shifting refers to dynamic environments where conditions evolve over time, requiring the ability to adapt strategies as new information emerges Brehmer 1992 ; Payne et al. 1993 . Uncertainty, risk, and set-shifting are three key dimensions that dominate most real-world decisions Kochenderfer 2015 ; Ruggeri et al. 2020 ; Uddin 2021 . They each represent a conceptually distinct construct, yield their own behavioral patterns, and engage their own cognitive mechanism Strategy 1997 ; Sacré et al. 2019 ; Konishi et al. 1998 . Our methodology for investigating if and how generative AI systems navigate these decision dimensions aligns with recent calls for machine psychology Binz and Schulz 2023 ; Hagendorff 2023 ; Kosinski 2024 ; Hagendorff 2024 ; Chen et al. 2023 ; Lehr et al. 2025 , which advocates the use of experimental psychology paradigms. Unlike domain-specific tasks, psychological tests, originally designed for humans, strip away contextual confounds and isolate distinct decision dimensions. As a result, they can provide simplified yet tight experimental control, enable precise theory testing, and reveal general cognitive mechanisms that may extend across diverse domains. We adapted three well-established psychological tests: the Iowa Gambling Task for uncertainty Bechara et al. 1994 , the Cambridge Gambling Task for risk Rogers et al. 1999 ,

Read the complete text