Personalized Large Language Models Can Increase the Belief Accuracy of Social Networks

Adiba Proma, Neeley Pate, Sean Kelty, Gourab Ghoshal, James N. Druckman, Ehsan Hoque · 2025

Read Personalized Large Language Models Can Increase the Belief Accuracy of Social Networks by Adiba Proma, Neeley Pate, Sean Kelty, Gourab Ghoshal, James N. Druckman, Ehsan Hoque (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06153.txt.

Introduction

Adiba Proma and co-authors show that personalized language models can raise belief accuracy across social networks in a 2025 paper. Chatbots increasingly shape public views on contested topics, but their effect on whole networks was untested.

A pre-registered experiment with over twelve hundred people ran around the 2024 United States presidential election. The personalized models tailored messages to each profile and carried guardrails for factual retrieval. People exposed to one updated their beliefs toward the truth. They also chose to keep following the model and rebuilt their networks to include others with similar, more accurate beliefs. The authors cast such models as corrective agents for online spaces, with lessons for responsible AI communication.

Catalog record

CREATORAdiba Proma, Neeley Pate, Sean Kelty, Gourab Ghoshal, James N. Druckman, Ehsan Hoque
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE86 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2506-06153

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Personalized Large Language Models Can Increase the Belief Accuracy of Social Networks Adiba Proma, Neeley Pate, Sean Kelty, Gourab Ghoshal, James N. Druckman, Ehsan Hoque Abstract Large language models (LLMs) are increasingly involved in shaping public understanding on contested issues. This has led to substantial discussion about the potential of LLMs to reinforce or correct misperceptions. While existing literature documents the impact of LLMs on individuals’ beliefs, limited work explores how LLMs affect social networks. We address this gap with a pre-registered experiment (N = 1265) around the 2024 U.S. presidential election, where we empirically explore the impact of personalized LLMs on belief accuracy in the context of social networks. The LLMs are constructed to be personalized, offering messages tailored to individuals’ profiles, and to have guardrails for accurate information retrieval. We find that the presence of a personalized LLM leads individuals to update their beliefs towards the truth. More importantly, individuals with a personalized LLM in their social network not only choose to follow it (as a way of indicating they would like to obtain information from it in subsequent interactions), but also construct subsequent social networks to include other individuals with beliefs similar to the LLM – in this case, more accurate beliefs. Therefore, our results show that LLMs have the capacity to influence individual beliefs and the social networks in which people exist, and highlight the potential of LLMs to act as corrective agents in online environments. Our findings can inform future strategies for responsible AI-mediated communication. Misinformed beliefs pose major challenges across a range of domains, from public health and civic trust to the understanding of basic facts [ 1 , 2 , 3 , 4 ] . These beliefs can erode confidence in voting systems [ 5 , 6 ] , vaccination programs [ 7 , 8 ] , and mainstream news organizations [ 9 ] . They also are often tied to misperceptions about policies and empirical reality [ 1 ] . The rise of large language models (LLMs) has amplified both the risks and opportunities in this space. LLMs can generate misinformation at scale [ 10 ] , sometimes producing false claims that humans find difficult to distinguish from truth [ 11 ] . At the same time, LLMs are being explored for their potential to support factual accuracy through automated fact verification [ 12 , 13 ] , disinformation detection [ 14 ] , and conversational engagement [ 15 ] . Central to both their promise and their risk is the persuasive nature of LLMs. Empirical work has shown that LLMs can be highly persuasive on their own [ 16 ] , sometimes rivaling trusted human authorities such as public health agencies [ 17 ] . LLMs can also be personalized — that is, LLMs can structure communications to match a person’s reasoning styles [ 15 ] or psychological profiles [ 18 ] . Such personalized LLM messages are often more effective than general non-personalized ones [ 18 , 19 ] , especially if they match their audience’s personality [ 20 ] , and can exert more influence than human-generated messages [ 18 ] . A crucial limitation of existing work on LLMs and belief formation is its exclusive focus on one-on-one interactions, where a user directly engages with the model in isolation [ 15 ] . In reality, beliefs are rarely formed or revised in isolation. They are shaped by social context with exposure to multiple information streams [ 21 , 22 , 23 ] . Prior work has shown that social networks and peer influence play critical roles in belief formation and resistance to correction [ 2 , 4 , 24 , 25 ] . Despite this, little is known about how personalized LLM-generated content might function in networked environments where users are exposed to diverse and competing signals. Moreover, users actively construct their social networks, and how exposure to a personalized LLM might influence that process has gone unstudied. Here, we study the impact of personalized LLMs in the context of a social network. The LLMs are constructed to provide accurate information, in line with work on misperception corrections [ 26 ] . Specifically, we expect that, relative to individuals in social networks without a personalized LLM, those in networks with access to a personalized LLM response will be more likely to move toward belief accuracy (hypothesis 1). This follows since the LLM will provide persuasive messaging, tailored to individual preferences. Additionally, we predict that those in social networks with an LLM are likely to include the LLM in their subsequent social network at rates greater than chance (hypothesis 2a). Prior research suggests that individuals tend to seek factual truth [ 27 , 28 ] , and LLMs, equipped with credible information, might be better able to facilitate that shift, making them desirable agents for subsequent engagement . Finally, this same logic (i.e., the truth-seeking nature of individuals) leads us to hypothesize that, relative to individuals in social networks without an LLM (i.e., the control), those with access to an LLM will form subsequent social networks that include significantly more members whose beliefs tend towards accuracy (hypothesis 2b). The belief updating a la hypothesis 1 combined with the desire to associate with others similar to themselves contributes to this prediction. This prediction suggests that adding an LLM to a network not only stimulates belief accuracy but it also shapes subsequent social networks such that they include more accurate beliefs overall. Here, we evaluate our hypotheses in a dynamic, socially embedded, empirical setting (Figure 1 ). We combine three elements rarely studied together: real-time belief updated, network-mediated exposure, and personalized LLM messaging. Specifically, we were interested in whether the presence of the LLM caused participants to have more accurate beliefs, and if it impacted how participants constructed their social networks. In a pre-registered experiment (N = 1265) conducted from October to December 2024 (see Supplemental Materials (SM), Section 3.1 and Tables S9 , S10 , and S11 ), individuals rated on a five-point scale the extent to which they believed a randomly assigned political statement that could be objectively true or false (see SM Subsection 2.1.2 , and Tables S1 , S2 , S3 ). They also provided a rationale for their belief (see SM Figure S1 for examples). Next, participants were exposed to other users’ responses and had an opportunity to update their beliefs. In the control condition, participants only saw three responses from peers who had simultaneously rated the same statement. In the treatment condition, participants saw two such peer responses, and an additional personalized response from the LLM bot. Finally, in the third stage, participants were shown six overall responses. These include the responses of those they previously followed, the bot in case of treatment, and the rest randomly ‘recommended’ by the system. Participants were instructed to follow overall three out of these six entries, and could subsequently choose to follow or unfollow others (including the bot), allowing us to observe changes in the network structure (see Materials and Methods , and Supplemental Figures S1 , S2 , S3 ). Participants were told they may or may not interact with an LLM bot in the experiment, and

Read the complete text