PoSSUM: A P r o tocol for S urveying S ocial-media U sers with M ultimodal LLMs

Roberto Cerina · 2025

Read PoSSUM: A P r o tocol for S urveying S ocial-media U sers with M ultimodal LLMs by Roberto Cerina (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-05529.txt.

Introduction

Roberto Cerina presents PoSSUM, a 2025 protocol for polling social media users through multimodal language models. Real posts, images, and traces feed silicon samples that stand in for real users. Those samples carry facts the model never learned in training. Platform bias skews who posts, so raw tallies would mislead.

A multilevel regression and post-stratification step corrects that skew. Priors keep the adjustment stable. The protocol was tested on the 2024 US presidential election. Five polls ran live during the race. The final poll called nearly every state right with lower bias than most human pollsters. The open tools may widen who can run credible surveys.

Catalog record

CREATORRoberto Cerina
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE119 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-05529

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PoSSUM: A P r o tocol for S urveying S ocial-media U sers with M ultimodal LLMs Roberto Cerina Abstract This paper introduces PoSSUM , an open-source protocol for unobtrusive polling of social-media users via multimodal Large Language Models (LLMs). PoSSUM leverages users’ real-time posts, images, and other digital traces to create silicon samples that capture information not present in the LLM’s training data. To obtain representative estimates, PoSSUM employs Multilevel Regression and Post-Stratification (MrP) with structured priors to counteract the observable selection biases of social-media platforms. The protocol is validated during the 2024 U.S. Presidential Election, for which five PoSSUM polls were conducted and published on GitHub and 𝕏 \mathbb{X} . In the final poll, fielded October 17–26 with a synthetic sample of 1,054 𝕏 \mathbb{X} users, PoSSUM accurately predicted the outcomes in 50 of 51 states and assigned the Republican candidate a win probability of 0.65. Notably, it also exhibited lower state-level bias than most established pollsters. These results demonstrate PoSSUM ’s potential as a fully automated, unobtrusive alternative to traditional survey methods. 1 Introduction This article describes PoSSUM , an open source 1 1 1 https://github.com/robertocerinaprojects/PoSSUM protocol to poll social-media users unobtrusively using multimodal Large Language Models (LLMs). The protocol seeks to address the skepticism [ 42 ] surrounding Artificially Intelligent (AI) polling by establishing a methodology comparable to that used by traditional pollsters who use online panels [ 63 ] . Concerns around AI polling are summarised aptly by this anonymous review to a related paper [ 14 ] : ‘… The goal of polling is quite simple: TO. LEARN. FROM. PEOPLE. NOW. I believe this model here does not learn, not from people, and not now’. Three necessary conditions emerge from this animated critique – to be a credible alternative to random digit dial (rdd) or self-selected online panels, Silicon samples [ 4 ] must enable novel learning – i.e. must contain more information than the mould 2 2 2 Silicon samples as per Argyle et al. [ 4 ] can be elicited from LLMs by using high-quality, real-life survey responses of humans as a mould . See Figure 1 for a graphical illustration of silicon sampling. on which they are based; must be human-aligned – i.e. conditional on the same generating process, they must produce a distribution of responses which matches that of humans; must be time-sensitive – i.e. we must be able to learn about changes in preferences and attitudes over time by studying these samples, and these changes should be reflective of true societal dynamics, rather than artifacts of data engineering. Mould Prompt LLM Synthetic Response Figure 1: A conceptual description of silicon sampling. PoSSUM proposes to poll the public by inferring attitudes and preferences of real-life social-media users with multimodal LLMs. A key innovation of PoSSUM entails the use of real-time unstructured digital-trace data to inform a mould . The unstructured nature of the data, in its free-flowing and unobtrusivly observable nature, provides a much richer compendium of measurable and non-measurable information than the sterile tabular socio-demographic data that has come to define the literature [ 4 , 56 , 9 ] . LLMs can pick up on textual or visual cues that are not easily observable for humans [ 70 ] . Importantly, the underlying unstructured data forms a unique digital footprint for a human existing in the world, hence allowing PoSSUM to allocate a unique mould to each individual whose preferences and attitudes we seek to simulate. Others have recently picked up on the potential for informing agents with large unstructured data generated by real human beings (e.g. interview transcripts [ 50 ] ), but PoSSUM remains alone, at the time of writing, in having these unique moulds update dynamically over time as subjects offer up new text, images, video and other media to the web. This dynamic, time-sensitive mould contributes to solving issues around degradation of simulated agents as we move further away in time from the point at which the mould was created. A second innovation pertains the generation of representative samples from unrepresentative pools of social media users [ 2 ] . Here too PoSSUM leverages the power of LLMs to digest unstructured user profiles and produce socio-demographic labels. This opens the door to traditional non-probability survey sampling techniques for social media users. I introduce a quota sampling module, where quotas are satisfied by matching the users’ inferred characteristics to a stratification frame. Whilst quota sampling can itself produce unrepresentative samples under self-selection [ 5 ] , it is a valuable tool to partially curb platform selection effects and demographic imbalances [ 27 ] . To further address selection and representation issues the protocol relies on Multilevel Regression and Post-Stratification (MrP) [ 21 , 49 ] with structured priors [ 19 ] to analyse the resulting synthetic responses. PoSSUM ’s formulation therefore is set-up to satisfy the three necessary conditions outlined above. It enables novel learning , in that the LLMs help us infer previously unknowable tabular survey responses from the unstructured flow of social media data for each user. The temporally updating mould addresses the time-sensitivity of learning, clearly enabling the study of preferences changing over time – PoSSUM can learn from synthetic panel data. Finally, efforts to tackle selection and representation via matching individuals to a stratification frame address human alignment , under the condition that the LLMs can faithfully infer the distribution of social media users’ preferences from their unstructured data 3 3 3 LLMs have been shown to be at least as capable as humans on this specific task in previous work [ 14 ] . . The central argument of this paper is that artificially intelligent polls generated using the PoSSUM protocol are valid instruments for measuring public opinion, on par with traditional polling methods. To test this proposition, PoSSUM is deployed during the 2024 US Presidential election – a setting that allows assessing LLMs’ ability to generalise understandings of political preferences beyond their training data. In this study I employ gpt-4o-2024-05-13 , whose training concluded in October 2023 . Hence the model faces a novel candidate choice-set and unforeseen demographic realignments, characteristic of the 2024 election. Data leakage type criticisms do not apply to this paper as a result. The paper is organized as follows. Section 2 presents an overview of the PoSSUM protocol. Section 3 describes the get_pool routine, which acquires an initial subject pool of social media users. Section 4 introduces the modular prompting architecture employed throughout the protocol. Section 5 reviews the LLM-enabled filtering procedure that selects statistically informative users from the subject pool. Section 6 details the implementation of MrP with structured priors tailored to this framework. Section 7 delineates the evaluation criteria for assessing PoSSUM ’s performance, including a multi-dimensional appraisal of predictive accuracy, capacity for novel learning, degree of human alignment, and sensitivity to temporal shifts. Finally, Section 8 provides a comprehensive discussion of key insights arising from the PoSSUM experiment, the limitations of the protocol, and directions for future research. 2 Protocol The goal of PoSSUM is to make granular inference about preferences and attitudes, for a given digital fieldwork period, which is representative of the true underlying population. What I describe is

Read the complete text