Nothing Deceives Like Success: Social Learning and the Illusion of Understanding in Science
Avery W. Louis, Marina Dubova, [0.6ex] Symbolic Systems, Stanford University, Stanford, CA 94305, USA, [0.4ex] Corresponding author. Email: [email protected] · 2026
Read Nothing Deceives Like Success: Social Learning and the Illusion of Understanding in Science by Avery W. Louis, Marina Dubova, [0.6ex] Symbolic Systems, Stanford University, Stanford, CA 94305, USA, [0.4ex] Corresponding author. Email: [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27188.txt.
Introduction
Louis and Dubova model how groups build theories in science, to test whether copying ideas that look successful helps or harms. In agent-based simulations, scientists overrate their own theories. That creates a lasting gap between perceived and actual quality, an illusion of understanding. Favoring successful theories narrows exploration. Poor theories are filtered fast, but promising rivals are also crowded out before they can prove themselves.
Catalog record
| CREATOR | Avery W. Louis, Marina Dubova, [0.6ex] Symbolic Systems, Stanford University, Stanford, CA 94305, USA, [0.4ex] Corresponding author. Email: [email protected] |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 66946 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.27188
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Nothing Deceives Like Success: Social Learning and the Illusion of Understanding in Science Avery W. Louis, Marina Dubova, [0.6ex] Symbolic Systems, Stanford University, Stanford, CA 94305, USA, [0.4ex] Corresponding author. Email: [email protected] Abstract Success-driven social learning, in which individuals preferentially adopt the ideas and methods that appear most successful, is a foundational principle of collective behavior across systems ranging from ant colonies to scientific communities. But science is a particular kind of collective search—one in which the quality of an explanation is itself difficult to assess. Is success bias adaptive in this setting? In agent-based simulations of collective theory building, we find that it is not. Scientists in our model systematically overestimate the quality of their own theories, creating an illusion of understanding : a persistent gap between perceived and actual performance. Success bias amplifies this illusion; communities that favor apparently successful theories explore a narrower range of possibilities, efficiently filtering out poor explanations but failing to discover better ones. This effect intensifies with problem complexity, as scientists in more complex environments become increasingly unable to assess how well their theories actually perform. Most strikingly, when agents optimize their social behavior to maximize the perceived success of their theories, they paradoxically undermine their actual performance, and produce levels of inequality that mirror those found in real scientific communities. 1 1 1 All code, data, and simulation outputs are available at https://osf.io/u3xc9/?view_only=309cb49465554a869932147b1ab74b50 . Science is a fundamentally social enterprise. If we had no means of building on the work of others, we couldn’t possibly hope to build thinking machines, create life-saving vaccines, or go to the moon and beyond. But the cumulative nature of science introduces a critical question: how should scientists learn from one another? Every scientist must decide whose work to build on, which ideas to adopt or abandon, and where to focus their limited attention—decisions that collectively shape the trajectory of research. Of course, there are many ways to navigate these decisions, and a substantial body of work across social psychology, collective behavior, and cultural evolution has explored how individuals do so [ 6 , 31 , 20 , 3 ] . Among the possible approaches, success-biased learning, wherein scientists preferentially adopt and build on the ideas that appear “best", seems to be a neatly rational approach [ 47 , 1 , 58 ] . If, for example, you were a cosmologist hoping to explain dark matter, a very natural first step would be to ask which theories currently seem most promising and work on one of those. It is easy to see why: promising theories are promising for reasons (they might fit more data, have fewer free parameters, or connect to other successful frameworks) so building on existing success seems like a rational use of limited time and attention. By doing so, communities can, in principle, concentrate their collective efforts on the most promising solutions, weeding out dead ends. Success bias also aligns with one of the most powerful ideas in both nature and engineering—that systems improve by selecting for what works. It is the logic of natural selection, of ant colonies reinforcing productive foraging trails, and of gradient descent in optimization. Much contemporary science policy pushes in precisely this direction; hiring and promotion increasingly rely on quantitative metrics of success such as citation counts and h-indices [ 24 ] , funding agencies favor proposals with demonstrated track records of results, and there is a broader institutional push to make science more “accountable" through measurable outputs [ 48 , 23 ] . The implicit assumption is that science would improve if it were more systematically success-biased. But this assumption rests on two premises: that we can reliably identify which scientific contributions are successful, and that following the most successful leads to genuine discovery. Recent computational work challenges the second premise, showing that success-biased strategies can impede innovation [ 60 ] . Here, we challenge both. What makes a scientific theory “good"? The straightforward answer, that good theories accurately describe reality, obscures a fundamental epistemic constraint: scientists never have direct access to how well their theories map onto the world. Instead, the quality of theories scientists produce can only be estimated indirectly, through proxies such as predictive accuracy, empirical fit over collected data, parsimony, coherence, and others. The history of science is marked by episodes in which this process of evaluation has gone awry. Scientists routinely work with theories that appear adequate, or even compelling, only to later discover they systematically fail to account for observations at hand [ 27 , 32 , 46 ] . This happened in the decades-long focus on amyloid plaques as the primary cause of Alzheimer’s disease, which resulted in a string of failed clinical trials [ 29 ] . Occasionally, entire research programs revolve around phenomena that are later decided not to exist at all, as demonstrated by the case of N-rays, where dozens of physicists reported detecting a form of radiation that was ultimately shown to be illusory [ 42 , 7 ] . These episodes reveal an important limitation of the scientific enterprise: because all theories are provisional attempts to capture aspects of an incompletely observed world, it may be genuinely difficult, at any given moment, to assess how good they actually are [ 37 , 54 ] . This results in a potential divergence between the scientific community’s in-the-moment judgments of theoretical quality and the longer-term performance of those theories, called an illusion of understanding [ 39 ] . If such illusions are routine rather than exceptional, they carry significant implications for social learning in science, as the very signal that learners rely on may be systematically misleading. Here, we investigate the consequences of success-biased social learning for collective theory development using 2,301 agent-based simulations. Our model captures key features of scientific practice: scientists actively sample data from their environment, construct compressed representations (theories) that explain their observations, and engage in social learning by sharing data and selectively adopting theoretical components from colleagues. We measure both perceived theory success (how well each scientist’s theory fits the data the scientific community has access to) and actual success (how well theories account for the held-out ground truth), allowing us to directly measure illusions of understanding and track their effects. We find that although illusions of understanding exist regardless of the social learning strategies employed, success bias amplifies them. When scientists in the model favor seemingly successful theories by their peers, they explore a narrower range of possibilities, efficiently discarding poor explanations but also missing opportunities to discover better ones. Moreover, social learners confronting more complex problems are especially prone to overestimating the success of their theories compared to those facing simpler tasks. Finally, when learners adjust their social strategies to maximize perceived success, they paradoxically worsen actual performance. 1 Materials and Methods We extend an agent-based model of scientific inquiry to investigate how the structure of a learning problem and scientists’ social learning strategies—in particular, the tendency to preferentially adopt theories that appear more successful—jointly shape the quality of developed theories and the social organization of scientific communities [ 16 ] . Each agent in the model represents an individual scientist who conducts experiments and builds a theory of their environment