Replicating Human Motivated Reasoning Studies with LLMs
Neeley Pate, Adiba Mahbub Proma, Hangfeng he, James N. Druckman, Daniel Molden, Gourab Ghoshal, Ehsan Hoque · 2026
Read Replicating Human Motivated Reasoning Studies with LLMs by Neeley Pate, Adiba Mahbub Proma, Hangfeng he, James N. Druckman, Daniel Molden, Gourab Ghoshal, Ehsan Hoque (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16130.txt.
Introduction
Neeley Pate and coauthors test if language models copy slanted thought in humans. Their 2026 preprint deals with driven thought, where a wish bends how facts are read. Human tests show this bend in clear ways. It is not known if plain language models show the same bend. The team reruns human tests with models in the place of people.
They check if model picks shift with the wish as human picks do. Like shifts would mean models pick up human bias from text. Un like shifts would mark a gap in how models think. The work helps teams judge model acts in charged topics. It ties model tests to known human test designs.
Catalog record
| CREATOR | Neeley Pate, Adiba Mahbub Proma, Hangfeng he, James N. Druckman, Daniel Molden, Gourab Ghoshal, Ehsan Hoque |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 59 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16130
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Replicating Human Motivated Reasoning Studies with LLMs Neeley Pate, Adiba Mahbub Proma, Hangfeng he, James N. Druckman, Daniel Molden, Gourab Ghoshal, Ehsan Hoque Abstract Motivated reasoning – the idea that individuals processing information may be motivated to reach a certain conclusion, whether it be accurate or predetermined – has been well-explored as a human phenomenon. However, it is unclear whether base LLMs mimic these motivational changes. Replicating 4 prior political motivated reasoning studies, we find that base LLM behavior does not align with expected human behavior. Furthermore, base LLM behavior across models shares some similarities, such as smaller standard deviations and inaccurate argument strength assessments. We emphasize the importance of these findings for researchers using LLMs to automate tasks such as survey data collection and argument assessment. 1 Introduction Motivated reasoning, a concept that describes how individuals process information Druckman and McGrath 2019 , is often portrayed as involving either the pursuit of a directional (predetermined) conclusion or the pursuit of an accurate (non-directional) conclusion Kunda 1990 . For example, individuals endorse a policy proposed by their party because they desire holding positions that align with their party leaders (a directional goal). Alternatively, they may instead consider the strength of the arguments for the policy in an effort to form an accurate opinion. A particular motivation can ultimately affect individuals’ opinions and actions Bolsen et al. 2014 ; Bolsen and Druckman 2015 ; Druckman et al. 2013 ; Mullinix 2016 or even facets of reality Kahan et al. 2013 ; Druckman et al. 2013 . Figure 1: Graphic Describing the Procedure Followed in the Study. An example of different types of prompts (accuracy vs. directional) is shown to highlight potential differences in reasoning. Motivations affect an individual’s opinion formation through how they process information about policies, issues, and stances, particularly within the political domain. The manner in which such information is processed can affect both opinion formation and perceptions of how strong presented arguments are Druckman et al. 2013 . When motivated by accuracy, an individual receives information in a relatively unbiased fashion, focusing on the perceived argument quality rather than external factors Druckman and McGrath 2019 . When motivated directionally, an individual interprets the information in a way to align with prior beliefs or group expectations. Given the direction of motivated reasoning is often dependent on an individual’s identity or experiences, such as partisanship Bolsen et al. 2014 , recent work has begun to explore whether LLMs embody motivated reasoning when given particular aspects of demographics or personality. Studies have shown providing LLMs with traits such as political affiliation and gender Dash et al. 2025 or occupation and goal Cho et al. 2025 do mimic motivated reasoning based on their specified role. However, it is well-established that even base LLMs (with no provided personality or identity) hold certain biases and preferences, particularly skewing to left-leaning perspectives Motoki et al. 2024 ; Proma et al. 2025 . This raises the question of whether base LLMs inherently mimic motivated reasoning when provided with information. Thus, this paper aims to answer the following research questions: • RQ1 : Do base LLMs (without persona induction) mimic motivated reasoning similar to human responses? • RQ2 : Do similarities in behavior exist between groups of LLMs under different motivations and domains? In order to answer these questions, we completed a comprehensive literature review to select four prior works on political motivated reasoning to replicate with LLMs (Figure 1 ). We report results from seven models spanning five families. The contributions of this work include: • Base LLMs do not appear to mimic human-like motivated reasoning. • Whether different LLMs perform similarly in opinion formation depends on given topics. • LLMs may systematically fail to accurately assess information, leading inaccuracies in natural language processing (NLP) tasks such as argument strength assessment. These results have implications for social scientists and computer scientists who utilize LLMs as survey proxies or predicting public opinion, as well as the artificial intelligence (AI) and NLP communities interested in LLM-human alignment or using LLMs for tasks such as argument assessment and opinion formation. We offer considerations for researchers using base LLMs for predicting how individuals will respond to arguments in motivated reasoning contexts (such as surveys) and anticipating how arguments resonate with humans. 2 Related Works 2.1 Motivated Reasoning Motivated reasoning emphasizes two main avenues of reaching conclusions: pursuing an accuracy (non-directional) goal or pursuing a directional (predetermined) goal Kunda 1990 . Reasoning is often induced by altering content around or within information. For example, accuracy motivation is often primed by asking participants to consider information provided in an evenhanded fashion Bolsen et al. 2014 ; Bolsen and Druckman 2015 or informing participants they will be asked to later justify opinions Bolsen et al. 2014 ; Bolsen and Druckman 2015 . To induce directional motivation, there are two broad categories such inductions can fall into: authority/community and personality. Calls on authority or community often include information about who approves (or disapproves), such as party leaders Bolsen et al. 2014 ; Druckman et al. 2013 or scientific experts Bolsen and Druckman 2015 . For example, including information about polarization around a given issue can make group identity more salient than message content, pushing individuals to align their opinion with the perceived opinion of their party leaders, while not including such information makes these tendencies less prevalent Druckman et al. 2013 . Personality techniques call on the individual to reflect on particular cues, such as their prior beliefs and core values Bayes et al. 2020 . These cues may bias recall of certain events in a given direction Kunda 1990 . Pursuing motivated reasoning can affect opinion formation Bolsen et al. 2014 , argument strength assessments Druckman et al. 2013 , and fact interpretation Kahan et al. 2013 . We expand research on motivated reasoning by exploring whether LLMs mimic motivated reasoning, particularly without persona prompts. 2.2 Bias in Persona-Based and Non-Persona-Based LLMs Central to the idea of motivated reasoning is how personal biases shape information processing. Within the context of LLMs, it is well-known that LLMs tend to exhibit a left-leaning bias Bernardelle et al. 2025 ; Bang et al. 2024 ; Faulborn et al. 2025 . Such bias implicit in LLM responses can present in a variety of ways, such as a bias towards certain media outlets Dai et al. 2025 ; Proma et al. 2025 . LLMs have also been known to take on other human-like preferences, such as message structure Li et al. 2025 and self-preference Panickssery et al. 2024 . The inclusion of personality traits (or personas) in LLM prompts can also affect how bias presents in LLM responses Borah and Mihalcea 2024 , though LLMs may fail to reconcile conflicts within a provided persona Liu et al. 2024 . While LLMs may struggle in identifying human-like motivated reasoning in text Yong et al. 2025 , the inclusion of personalities in prompts has been shown to bias LLM behavior in ways similar to motivated reasoning Cho et al. 2025 ; Dash et al. 2025 . Given that providing LLMs with personas can lead to certain biases in LLM behavior, it becomes important to study how LLMs respond under motivational