Operational Validation of Large-Language-Model Agent Social Simulation: Evidence from Voat v/technology

Aleksandar Tomašević, Darja Cvetković, Sara Major, Slobodan Maletić, Miroslav Anđelković, Ana Vranić, Boris Stupovski, Dušan Vudragović, Aleksandar Bogojević, Marija Mitrović Dankulov · 2025

Read Operational Validation of Large-Language-Model Agent Social Simulation: Evidence from Voat v/technology by Aleksandar Tomašević, Darja Cvetković, Sara Major, Slobodan Maletić, Miroslav Anđelković, Ana Vranić, Boris Stupovski, Dušan Vudragović, Aleksandar Bogojević, Marija Mitrović Dankulov (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-21740.txt.

Introduction

Computer agents powered by language models are now used to fake whole online communities, but few studies check how faithful the fakes are. Aleksandar Tomašević and co-authors run thirty month-long simulations of a technology forum modeled on Voat's v/technology board. They then compare the simulations against thirty real month-long windows from the actual board.

The check covers five dimensions, from posting activity and network shape to toxicity and topics. User counts and daily activity matched the real board closely. Simulated threads ran longer and carried more toxicity overall, though the toxicity sat in the wrong layer: fake posts were nastier than real ones, while fake comments were tamer. Both networks formed a core with a loose fringe. The authors point to the agents' lack of memory as one fixable cause.

Catalog record

CREATORAleksandar Tomašević, Darja Cvetković, Sara Major, Slobodan Maletić, Miroslav Anđelković, Ana Vranić, Boris Stupovski, Dušan Vudragović, Aleksandar Bogojević, Marija Mitrović Dankulov
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE65 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-21740

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Operational Validation of Large-Language-Model Agent Social Simulation: Evidence from Voat v/technology Aleksandar Tomašević, Darja Cvetković, Sara Major, Slobodan Maletić, Miroslav Anđelković, Ana Vranić, Boris Stupovski, Dušan Vudragović, Aleksandar Bogojević, Marija Mitrović Dankulov Abstract Large Language Models (LLMs) enable generative social simulations that can capture culturally informed, norm - guided interaction on online social platforms. We build a technology community simulation modeled on Voat, a Reddit - like alt-right news aggregator and discussion platform active from 2014 to 2020. Using the YSocial framework, we seed the simulation with a fixed catalog of technology links sampled from Voat’s shared URLs (covering 30+ domains) and calibrate parameters to Voat’s v/technology using samples from the MADOC dataset. Agents use a base, uncensored model (Dolphin 3.0, based on Llama 3.1 8B) and concise personas (demographics, political leaning, interests, education, toxicity propensity) to generate posts, replies, and reactions under platform rules for link and text submissions, threaded replies, voting, and daily activity cycles. We run a 30-day simulation and evaluate operational validity by comparing distributions and structures—specifically, activity patterns, interaction networks, toxicity, and topic coverage—with matched Voat data. Limitations of the current study include the stateless agent design and evaluation based on a single 30-day run, which constrains external validity and variance estimates. The simulation generates realistic discussions, often featuring toxic language, primarily centered on technology topics such as Big Tech and AI. This approach offers a valuable method for examining toxicity dynamics and testing moderation strategies within a controlled environment. 1 Introduction Online social media is a primary arena for public discourse and collective sense - making, yet it often generates polarization, echo chambers, harassment, and norm violations that co - evolve with moderation and platform design [ 10 , 5 , 23 , 22 ] . Studying such dynamics in vivo is increasingly difficult in the post - API era due to ethical, access, and reproducibility constraints [ 20 ] . Complementary to traditional agent - based models, recent LLM - based simulations embed language - model agents in platform - faithful environments to test whether recognizable online regularities emerge from culturally informed, norm - conditioned interactions [ 3 , 31 , 17 , 1 ] . As online platforms begin to mix people and artificial users in the future, human–machine interactions will become an increasingly important subject of social - science research [ 29 ] . We take a first step by studying machine–machine interaction in a thread - and - feed community. We treat LLM agents as carriers of cultural knowledge and conversational norms [ 8 ] , and ask whether they can share, remix, and react to content in ways that yield familiar social - media patterns—without explicit rules shaping the results. These machine–machine baselines show what AI agents alone can produce and set the stage for later human–machine studies, where even suspected machine presence may change how people act [ 29 ] . We report a representative 30 - day run. We evaluate operational validity via activity rhythms and thread shape, network core–periphery structure, toxicity stratified by layer, topic overlap and embedding similarity, named - entity structure, and stylistic convergence. By operational validity we mean that simulated aggregate distributions and meso - level structures resemble those observed in the reference community at comparable scales, without attempting one - to - one reproduction of specific users or texts. We view LLM agents as cultural technologies rather than payoff - maximizing decision makers. Agents do not optimize utilities for posting, replies, or visibility. Instead, models trained on large conversational corpora encode roles, norms, and routines. In practice, agents respond by what seems appropriate in context—given a topic, a thread position, and a persona—rather than by trying to achieve a specific goal through reward maximization. This lets us test whether compressed cultural knowledge alone can generate familiar online patterns when placed inside a realistic platform setting [ 8 , 12 , 31 ] . Concretely, we simulate a technology forum modeled on Voat using the YSocial digital - twin platform with LLM - driven agents [ 28 ] . Voat was a Reddit - like news aggregator and discussion platform launched in April 2014 and shut down in December 2020. It was organized into user - created “subverses” (the subreddit equivalent) with threaded comments and voting, positioned as a free - speech site with minimal moderation; it later became known for hosting deplatformed and alt - right communities after Reddit bans. In our simulations, we seed discussions around technology topics with a fixed catalog of links drawn from the Voat dataset’s shared URLs (spanning more than 30 technology domains; e.g., TechCrunch, Wired, CNN Tech, Fox News Tech), served through a reverse - chronological feed with a 180 - round visibility window, so agents can share stories that have been part of real - world online discussions without live RSS ingestion. Agents have concise persona profiles (e.g., interests, political lean, toxicity propensity) and use a base, uncensored language model (Dolphin 3.0, based on Llama 3.1 8B) 1 1 1 https://ollama.com/library/dolphin3 allowing disagreement and, at times, toxic speech. We select Dolphin as a recent, unrestricted open - source option to avoid third - party alignment constraints that suppress toxic language and to ensure reproducibility without external dependencies. See Table 5 for model references. We focus on Voat because it shared Reddit’s UI and architecture while hosting a smaller community, enabling faster simulation and tractable analysis, and because we have a complete historical dataset for calibration [ 19 , 21 ] . We calibrate population size and growth, posting and reply likelihoods (and per - round action budgets), to Voat’s v/technology and run 30 - day simulations. We do not aim to perfectly match a single community or assert that model outputs equal human behavior. Instead, we adopt a complex - systems view [ 17 , 3 ] : if agents carry compressed cultural knowledge and operate under realistic platform rules and content flows, do familiar platform - level patterns emerge? In sum, this paper makes the following contributions: 1. A platform - faithful machine–machine baseline for a Voat - like technology forum using YSocial, seeded with a fixed catalog from 30+ technology domains and calibrated to v/technology. 2. An empirical calibration and validation against Voat MADOC samples covering activity rhythms and thread shape, network core–periphery structure, toxicity by layer, topic alignment and embedding similarity, named - entity structure, and stylistic convergence. 3. An operational - validation framework and reproducible recipe (personas, seeds, parameters, link catalog, evaluation scripts). The paper proceeds as follows: we outline the simulation approach and design, detail evaluation methods, present results on activity, network structure, toxicity, topics/embeddings, named - entity structure, and stylistic convergence, and discuss implications and limitations. 2 LLMs as social agents Recent work argues that large language models are best understood as cultural and social technologies: systems that reorganize and render usable the information produced by people, just as print, markets, and bureaucracies have historically done [ 12 ] . From this perspective, the key question is not whether models can be autonomous minds, but how they moderate access to, and coordination of, cultural repertoires and human knowledge. Much like earlier technologies, LLMs reduce complexity and function as mediators

Read the complete text