Adapting to LLMs: How Insiders and Outsiders Reshape Scientific Knowledge Production DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Human-centered computing Empiri
Huimin Xu, Houjiang Liu, Yan Leng, Ying Ding · 2025
Read Adapting to LLMs: How Insiders and Outsiders Reshape Scientific Knowledge Production DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Human-centered computing Empiri by Huimin Xu, Houjiang Liu, Yan Leng, Ying Ding (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-12666.txt.
Introduction
Huimin Xu, Houjiang Liu, Yan Leng, and Ying Ding measure how large language models are changing who makes scientific knowledge. Their paper appeared in two thousand twenty-five. The study views researchers as insiders or outsiders to their fields. An evaluation workflow blends that lens with a knowledge production framework. Generative AI is now woven into daily research practice. Its effects ripple through collaboration and credit.
Findings show the models sparking both innovation and reorganization. Communities shift as outsiders gain new entry points. Insiders change how they work and publish. The authors draw lessons for computer-supported cooperative work. New research chances appear in supporting AI-driven science. The knowledge economy of academia is being quietly rewired.
Catalog record
| CREATOR | Huimin Xu, Houjiang Liu, Yan Leng, Ying Ding |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 57 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-12666
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Adapting to LLMs: How Insiders and Outsiders Reshape Scientific Knowledge Production DOI: XXXXXXX.XXXXXXX Conference: Make sure to enter the correct conference title from your rights confirmation emai; June 03–05, 2018; Woodstock, NY ISBN: 978-1-4503-XXXX-X/18/06 CCS: Human-centered computing Empirical studies in HCI Huimin Xu, Houjiang Liu, Yan Leng, Ying Ding Abstract . CSCW has long examined how emerging technologies reshape the ways researchers collaborate and produce knowledge, with scientific knowledge production as a central area of focus. As AI becomes increasingly integrated into scientific research, understanding how researchers adapt to it reveals timely opportunities for CSCW research — particularly in supporting new forms of collaboration, knowledge practices, and infrastructure in AI-driven science. This study quantifies LLM impacts on scientific knowledge production based on an evaluation workflow that combines an insider-outsider perspective with a knowledge production framework. Drawing on the OpenAlex dataset, we identify 7,106 researchers as insiders or outsiders based on their involvement in LLM development. We then analyze their publications using few-shot prompting to reveal shifts in knowledge production, demonstrating how LLMs reshape their research directions. To ensure robustness, we further validate these findings with bibliometric measurements. Our findings reveal how LLMs catalyze both innovation and reorganization in scientific communities, offering insights into the broader transformation of knowledge production in the age of generative AI and sheds light on new research opportunities in CSCW. 1. Introduction “It is not the strongest of the species that survives, nor the most intelligent that survives. It is the one that is most adaptable to change.” - Charles Darwin Darwin’s quote, rooted in the natural world, resonates strongly with the challenges facing both companies and universities today. In rapidly changing environments, adaptability, the ability to evolve and innovate, serves as a key driver of technological and scientific advancement. One of the most pressing challenge organizations face is to balance the imperative to produce with the need to innovate in response to change ( Uhl-Bien and Arena 2018 ) . This tension is often framed as a trade-off between exploitation (maximizing existing strengths) and exploration (pursuing novel opportunities). As Uhl-Bien and Arena 2018 argue, the most resilient organizations are those that can dynamically integrate both modes of operation. The rise of generative AI and large language models (LLMs) has accelerated the pace for scientists to adapt their research portfolios in response to emerging opportunities and challenges. Scientists who previously relied on traditional or disciplinary theories and methods have begun incorporating LLMs and large-scale data analytics into their established work. This shift is not merely technical; it reflects a deeper cognitive adaptability, a willingness to learn new tools, critically reflect on possible issues, rethink long-standing research paradigms, and envision the future scientific knowledge production. LLMs are rapidly reshaping what is possible across disciplines, from accelerating discovery in the natural sciences ( Luo et al. 2025 ) to transforming methodologies in the social sciences and humanities ( Bail 2024 ) . In natural science, LLMs have outperformed experts in predicting neuroscience experimental results ( Luo et al. 2025 ) . In social science, Bail 2024 claimed that “generative AI has the potential to improve survey research, online experiments, automated content analyses, agent-based models, and other techniques commonly used to study human behavior.” LLMs not only bring tremendous potential and applications for scientists, they have also raised new reflections about ethics, fairness, hallucination, reproducibility, and plagiarism ( Bail 2024 ; Stokel-Walker and Van Noorden 2023 ; Otis et al. 2024 ) . Aligning the adaptive phenomenon of scientists, we propose the central research question: How are scientists adapting their research practices in response to LLMs, and what does this reveal about the broader transformation of scientific knowledge production in the age of generative AI? Although many researchers now use LLMs to support or automate parts of their scientific workflows, or investigate LLMs as research subjects themselves, large-scale empirical evaluations of AI’s broader impact on science remain limited ( Ackerman et al. 2013 ; Correia et al. 2019 ; Jirotka et al. 2013 ; Neang et al. 2021 ) . Most current assessments rely on individual experiences or narrow, domain-specific experiments, rather than systematic, cross-disciplinary studies. This urges a need for a comprehensive framework to understand how AI is transforming scientific practice. Toward this end, we develop an evaluation workflow to explain how LLMs are reshaping scientific knowledge production. Our workflow builds on the insider–outsider classification of radical innovation ( Hill and Rothaermel 2003 ; Van De Poel 2000 ) , integrated with a comparative framework of knowledge production modes ( Scott et al. 2012 ; Hessels and van Lente 2008 ) . In the context of LLM-driven scientific knowledge production, we first distinguish between “insiders”, such as computer scientists and AI researchers, who are directly involved in building, fine-tuning, and advancing LLM technologies, and “outsiders”, who apply LLMs within specific domain contexts, such as medicine, biology, or psychology, often without direct involvement in the technical development. We then examine research publications based on the knowledge production framework, which identifies key differences between two modes of knowledge production across five dimensions : application focus, disciplinary orientation, collaboration patterns, social accountability, and evaluation criteria. To quantify the impact of LLMs based on the evaluation workflow above, we conduct a computational analysis that combines few-shot classification powered by LLM with bibliometric validation. Specifically, using the OpenAlex dataset ( Priem et al. 2022 ) , we identify 7,106 researchers, classified as insiders or outsiders, who have published LLM-related work, along with their prior publications. We then classify research abstracts from these scholars along five knowledge dimensions using LLM few-shot prompting, enabling us to assess researchers’ knowledge production shifts before and after LLM adoption. To ensure robustness, we validate the LLM-generated classifications through bibliometric techniques, providing quantitative support for the observed patterns and trends. Our findings reveal that the adoption of LLMs is reshaping scientific knowledge production in distinct ways. Outsiders, researchers from non-AI domains, are increasingly leveraging LLMs to pursue application-focused research, engage in transdisciplinary work, address social accountability, and experiment with new evaluation practices. In response, insiders are restructuring their collaboration networks, forming partnerships across a broader range of institutional contexts. These shifts highlight timely opportunities for CSCW research: to support outsider-led integration of LLMs through the design of domain-specific tools, and to investigate emerging mediator roles that facilitate transdisciplinary collaboration, data ownership, and shared infrastructure in AI-driven science. 2. Related Work 2.1. The Role of AI in Scientific Knowledge Production CSCW has long examined how technologies shape the ways researchers collaborate and produce knowledge, with scientific knowledge production being a key area of focus ( Ackerman et al. 2013 ; Correia et al. 2019 ; Jirotka et al. 2013 ; Neang et al. 2021 ) . With the increasing integration of AI technologies into research workflows, new opportunities and challenges emerge. In particular, we identify two prominent lines of inquiry concerning the role of AI in scientific knowledge production and collaboration. 2.1.1. AI as a core infrastructure that mediates and transforms scientific collaborative work Recent advances in LLMs have accelerated the development of new tools that assist researchers in tasks such as literature synthesis, idea generation, data analysis, and academic writing. These tools include open-source ones like ScholarQA ( Asai