Revisiting the Predictability of Performative, Social Events
Juan Carlos Perdomo · 2025
Read Revisiting the Predictability of Performative, Social Events by Juan Carlos Perdomo (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11713.txt.
Introduction
Juan Carlos Perdomo revisits an old puzzle, in a 2025 preprint, about whether social events can be predicted at all. Predictions about society do not just describe the future. They shape it, by changing what people expect and do. Twentieth-century thinkers such as Merton and Simon treated this loop as a core problem for social science. Perdomo gives the question a modern form using tools from machine learning theory.
He draws on performative prediction, where the forecast changes the data, and on outcome indistinguishability. His main result says accurate prediction of social events is always efficiently possible. That holds no matter how strongly the forecast bends the outcome. Yet he also shows such forecasts are often not what we actually want. The paper closes by weighing what forecasters should aim for instead. It speaks to both social scientists and builders of prediction systems.
Catalog record
| CREATOR | Juan Carlos Perdomo |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 64 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-11713
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Revisiting the Predictability of Performative, Social Events Juan Carlos Perdomo Abstract Social predictions do not passively describe the future; they actively shape it. They inform actions and change individual expectations in ways that influence the likelihood of the predicted outcome. Given these dynamics, to what extent can social events be predicted? This question was discussed throughout the 20th century by authors like Merton, Morgenstern, Simon, and others who considered it a central issue in social science methodology. In this work, we provide a modern answer to this old problem. Using recent ideas from performative prediction and outcome indistinguishability, we establish that one can always efficiently predict social events accurately, regardless of how predictions influence data. While achievable, we also show that these predictions are often undesirable, highlighting the limitations of previous desiderata. We end with a discussion of various avenues forward. 1 Introduction Social predictions do not passively describe the future; they actively shape it. They inform actions and change expectations in ways that influence the likelihood of predicted events. For instance, economic forecasts influence market prices, election predictions influence voter turnout, and climate forecasts shape policies that impact future weather patterns. This dynamic, where predictions are performative and shape data distributions, is pervasive throughout social prediction. It is also a feature, not a bug. The overarching goal of building systems that make predictions about people is to inform actions that drive positive changes in the world. For example, we predict health outcomes to cure disease and poverty to alleviate it. While desirable, this dynamic also introduces methodological challenges and casts doubt on the predictability of social events. Motivated by these issues, we revisit the following question: Is it possible to efficiently find a predictor that actively influences the likelihood of observed events, yet still produces valid predictions? During the 20th century, multiple researchers recognized that social predictions shape social patterns and posed this question in their work. To give a few examples, in his 1928 thesis, Oskar Morgenstern argued that accurate economic forecasts are generally impossible since public predictions can be self-negating [ Morgenstern, 1928 ] . Later, Simon [1954] , as well as Grunberg and Modigliani [1954] , used recently popularized fixed point theorems from topology to make progress on this problem and proved the existence of predictions that influence outcomes and yet remain calibrated. However, they did not address the algorithmic question of how one might actually find these predictors efficiently (both computationally and statistically speaking). In this work, we revisit this question using modern mathematical tools. Framing our analysis in the language of performative prediction – a recent learning-theoretic framework introduced by Perdomo et al. [2020] that formalizes the causal aspects of social prediction – we establish the following result. One can always efficiently find predictions that actively shape the data distribution over outcomes and simultaneously satisfy rigorous validity guarantees like multi-calibration [ Hébert-Johnson et al., 2018 ] or outcome indistinguishability [ Dwork et al., 2021 ] . Moreover, the statistical and computational complexity of finding these predictors is, in many cases, just as easy as in supervised learning, where the distribution is static. However, while these rigorously calibrated predictions are always achievable, they are not always desirable. The fact that predictions are performative and influence the likelihood of future events invalidates naive extensions of traditional solution concepts inherited from supervised learning like calibration. Through simple examples, we show that these prediction rules can lead to poor equilibria where predictions exactly match the conditional distribution over outcomes – and hence are perfectly multicalibrated – while simultaneously explaining none of the variance in outcomes. We conclude by discussing avenues forward. 1.1 Overview of Technical Results In this work, we study the problem of finding prediction rules f that actively shape the distribution of observed outcomes y , yet still produce a valid forecast. We cast this problem in the language of performative prediction [ Perdomo et al., 2020 ] , a recent learning theoretic framework that formalizes how data in social contexts is not static, but rather a function of the published predictor f . The key conceptual device of the framework is the notion of the distribution map {\cal D}(\cdot) , a function which maps predictors f to distributions over feature, outcome pairs (x,y) . It captures how different predictors induces different distributions. In this work, we focus on the outcome performative case, in which predictions shape the distribution over outcomes y but not features x . In particular, given a randomized predictor f , we write (x,p,y)\sim{\cal D}(f) as shorthand for x\sim{\cal D}_{x},p\sim f(x), and y\sim{\cal D}_{y}(x,p) . The distribution {\cal D}_{x} over x is static, a prediction p is sampled from f(x) , and the conditional distribution over outcomes y is any function of x and the forecaster’s revealed prediction p\in\mathbb{R} . This setup exactly captures the prediction dynamics present in domains like health or education (see e.g. Perdomo et al. [2025] for a real world example). The learner makes a prediction p regarding someone’s individual outcome (future heart disease) using historical features (age, smoking history). The prediction influences treatments and individual behavior in ways that shape the likelihood of the predicted outcome (disease), but not the historical realization of features.. We assert that a forecast is valid if it is performatively multicalibrated , or synonymously, indistinguishable . Intuitively, a predictor is multicalibrated if f(x) equals y (in expectation), not just overall, but also when we condition information about the individual x and the prediction p\sim f(x) . A predictor is outcome indistinguishable if its correctness cannot be efficiently falsified, on the basis of the observed data and a pre-specified collection of computational tests. We use the terms multicalibration and (outcome) indistinguishability interchangeably since, interestingly, both conditions are formally equivalent [ Dwork et al., 2021 ] . We formally define these concepts for our outcome performative setting below: Definition 1.1 . A randomized predictor f mapping features x to predictions p is \varepsilon performatively multicalibrated (indistinguishable) with respect to a collection {\cal C}\subseteq\{\mathcal{X}\times\mathbb{R}\rightarrow\mathbb{R}\} if \displaystyle\big|\E_{\begin{subarray}{c}x\sim{\cal D}_{x},p\sim f(x)\\ y\sim{\cal D}_{y}(x,p)\end{subarray}}[c(x,p)(y-p)]\big|\leqslant\varepsilon\quad\text{ for all }c\in{\cal C}. Here, {\cal C} parametrizes the degree to which predictions match outcomes. If {\cal C} consists of just the constant 1 function, we recover the guarantee considered in Simon [1954] and Grunberg and Modigliani [1954] that \E_{{\cal D}(f)}[y]=\E_{{\cal D}(f)}[f(x)] overall. If {\cal C} consists of all bounded, measurable functions, then f(x) must be equal to the conditional distribution over outcomes, f(x)=\E_{{\cal D}(f)}[y|x] for all x . We can interpolate between these two extremes by varying {\cal C} . While the guarantee in Definition 1.1 is known to be achievable in supervised learning where {\cal D}(f)={\cal D}(f^{\prime}) for all f and f^{\prime} , the situation is substantially more complicated in this performative setting. Predictions can, in general, be self-negating and outcomes can move away from the published prediction p\sim f(x) . Given this possibility, it is not immediately obvious that performatively calibrated f even exist (that is a predictor f such that \E_{{\cal D}(f)}[f(x)]=\E_{{\cal D}(f)}[y] ) without making strong regularity assumptions on {\cal D}(\cdot) . The situation is further complicated by the fact that the distribution map {\cal D}(\cdot) is unknown to the learner. They can only deploy a predictor f and observe