Recommender systems, stigmergy, and the tyranny of popularity
Zackary Okun Dunivin, Paul Smaldino · 2025
Read Recommender systems, stigmergy, and the tyranny of popularity by Zackary Okun Dunivin, Paul Smaldino (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-06162.txt.
Introduction
Zackary Okun Dunivin and Paul Smaldino argue that science search tools lean too hard on fame. Tools like Google Scholar rank papers in part by past clicks and cites. That loop, which they link to stigmergy, lets a few well known papers soak up most of the view.
Over time, they say, this narrows what scholars read and deepens gaps in who gets seen. Their 2025 essay calls for search sites that let each user tune the weight of fame, age, and fit. They also urge care in how text models are built into search, so the tools widen choice rather than shrink it. The point applies to search tools well past science.
Catalog record
| CREATOR | Zackary Okun Dunivin, Paul Smaldino |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-06162
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Recommender systems, stigmergy, and the tyranny of popularity Zackary Okun Dunivin, Paul Smaldino Abstract Scientific recommender systems, such as Google Scholar and Web of Science, are essential tools for discovery. Search algorithms that power work through stigmergy, a collective intelligence mechanism that surfaces useful paths through repeated engagement. While generally effective, this “rich-get-richer” dynamic results in a small number of high-profile papers that dominate visibility. This essay argues argue that these algorithm over-reliance on popularity fosters intellectual homogeneity and exacerbates structural inequities, stifling innovative and diverse perspectives critical for scientific progress. We propose an overhaul of search platforms to incorporate user-specific calibration, allowing researchers to manually adjust the weights of factors like popularity, recency, and relevance. We also advise platform developers on how word embeddings and LLMs could be implemented in ways that increase user autonomy. While our suggestions are particularly pertinent to aligning recommender systems with scientific values, these ideas are broadly applicable to information access systems in general. Designing platforms that increase user autonomy is an important step toward more robust and dynamic information ecosystems. Keywords: search algorithms; information retrieval; human-computer interaction; artificial intelligence; complex systems Introduction Success in both cultural and intellectual markets is not driven by quality alone. Instead, visibility and accessibility increase salience and lower search costs, boosting the popularity of high-profile items and creating a cycle in which popularity begets more popularity. Take the example of the Music Lab experiment ( Salganik et al., 2006 ) , which constructed a simplified recommender system for songs, and found that songs that were initially advertised on the landing page were far more likely to be played, largely independent of quality. This further increased the visibility of those songs, and eventually led to the sort of massive inequality in exposure to which we’ve become increasingly accustomed (Frank 2011). This experiment highlights how recommender systems can shape patterns of consumption, consequently influencing tastes, affordances, and the surrounding culture. Recommender systems have become a ubiquitous element of daily life, governing access to media, consumer goods, and information more generally. Over time, the content of our personal and professional lives has become increasingly shaped by the algorithms at their core. These systems are also key drivers of scientific research, shaping how we discover existing ideas, methods, and findings. Scientific recommender systems such as Google Scholar and Web of Science don’t only facilitate scientific research; they shape its content. It is therefore worth reflecting on the role of these systems in scientific knowledge production, assessing the implicit values they reflect, and considering how we might alter them to better support good science. Stigmergy Drives Recommender Systems Recommender systems work because of stigmergy, a process of indirect coordination in human and non-human social systems ( Theraulaz and Bonabeau, 1999 ) . Ant pheromone trails, for instance, exemplify how local interactions produce emergent patterns of collective behavior without centralized control. As each ant follows the strongest pheromone gradient it can sense while laying down more pheromones of its own, the colony’s movement patterns become increasingly aligned until all the ants eventually travel along the same narrow path. In recommender systems like Google Search and Google Scholar, stigmergy manifests as preferential attachment ( van Dyke Parunak, 2005 ) . Highly cited or frequently accessed items are disproportionately surfaced, creating heavy-tailed distributions where a small number of items dominate attention—the rich get richer ( Frieze et al., 2006 ) . Such distributions can have advantages. They may reflect genuine differences in quality or utility, helping shared vocabularies and paradigms coalesce across communities. These collective frameworks enable large-scale scientific coordination by ensuring researchers encounter a common set of references. However, the same dynamics can also exacerbate inequality and homogeneity. The extreme “heaviness” of the tail in the distribution of influence—the degree of concentration of resources at the top—raises concerns. Overreliance on recommender systems narrows the field of visibility, crowding out less mainstream ideas. In science, this reduces access to diverse perspectives, methods, and paradigms, which in turn stifles innovation ( Foster et al., 2015 ; Smaldino and O’Connor, 2022 ) . The Costs of Inequality and Homogeneity in Science Heavy-tailed distributions of influence in science contribute to entrenched hierarchies. Articles with high citation counts reinforce the reputations of their authors, enabling them to attract more attention and funding ( Bol et al., 2018 ) . This cycle also favors dominant paradigms, making it difficult for alternative perspectives to gain traction. The implications are broad. Equity and diversity are essential for social and scientific progress. Diversity enables the exploration of a larger solution space, improving collective problem-solving and creating opportunities to synthesize ideas across disciplines ( Smaldino et al., 2024 ) . It also mitigates structural inequities that disproportionately affect underrepresented groups. Scholars from marginalized backgrounds face greater barriers to drawing on professional networks that catalyze careers ( Forret, 2006 ; Pedulla and Pager, 2019 ) ; a study by Hofstra et al., 2020 found that minority scholars’ contributions were less likely to find academic positions, despite, and in part due producing more innovative research. While recommender systems may seem like an unbiased and preferable alternative to “old boy networks,” in truth networks and recommender systems work in concert to reproduce inequity. Without early visibility, structurally marginalized scholars’ contributions may be less impactful and unrewarded in faculty placement, advancement, and grant-funding, perpetuating systemic injustice. Diversity is not without trade-offs. Shared language and aligned goals facilitate collaboration and productivity. And a subordinate or peripheral network of innovators may be critical for maintaining the diversity that disproportionately benefits the mainstream ( Milzman and Moser, 2023 ; Turner et al., 2023 ) —diversity does not necessarily imply equity. Yet, overemphasis on similarity narrows the landscape of ideas. Excessive inequality and lack of diversity harm both individuals and society, as innovative contributions from less prominent voices may be stifled. A healthier ecosystem therefore requires tempering, not flattening, inequality across levels of scientific organization, from single articles to entire disciplines. Rethinking Recommender Systems What can we do to counteract the tyranny of popularity? Researchers could deliberately engage with less cited papers that articulate similar ideas with comparable cogency, and include less prominent researchers in collective efforts. However, individual actions alone are unlikely to shift systemic patterns without coordination. System-level changes are needed, and they are feasible through an overhaul of search algorithms and their user-interfaces. Recommender systems, being proprietary, generally obscure their inner workings. Nevertheless, certain mechanisms and biases are evident, and these can guide our interventions. At the same time, any proposal must consider the financial and structural incentives that constrain the corporations operating these platforms. In commercial contexts, recommender systems are calibrated to maximize engagement. For instance, music streaming platforms promote certain content—sometimes paid for by producers—while aiming to keep users listening. This dynamic has roots in early commercial radio’s “payola” practices. Similarly, general search engines like Google balance relevance with monetization, incorporating click-through rates and “sponsored content”—paid advertisements—into their ranking systems. By contrast, research platforms such as Google Scholar or Web of Science, are less likely to employ direct monetization, but biases remain. These systems may, for instance, over-prioritize highly cited papers, as doing so may reduce user complaints by surfacing “safe,” broadly accepted results despite the hidden cost