Artificial Intelligence Tools Expand Scientists’ Impact but Contract Science’s Focus (Just accepted by Nature , to be online soon)

Qianyue Hao, Fengli Xu, Yong Li, James Evans · 2024

Read Artificial Intelligence Tools Expand Scientists’ Impact but Contract Science’s Focus (Just accepted by Nature , to be online soon) by Qianyue Hao, Fengli Xu, Yong Li, James Evans (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-07727.txt.

Introduction

Qianyue Hao, Fengli Xu, Yong Li, and James Evans study how AI tools reshape science itself in a paper from 2024. The team trained a language model to flag research that used AI help. They then applied it to more than forty-one million papers across the natural sciences. Adoption of AI tools has grown fast, and the authors compare users with non-users.

Scientists using AI published far more papers and earned far more citations. They also became project leaders sooner. Yet the whole field studied a narrower set of topics, and scientists engaged less with each other's work. The authors call this a paradox of personal gain and collective narrowing. They warn that AI may deepen old fields rather than open new ones.

Catalog record

CREATORQianyue Hao, Fengli Xu, Yong Li, James Evans
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE142 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-07727

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Artificial Intelligence Tools Expand Scientists’ Impact but Contract Science’s Focus (Just accepted by Nature , to be online soon) Qianyue Hao, Fengli Xu, Yong Li, James Evans Abstract Development in Artificial Intelligence (AI) has accelerated scientific discovery [ 1 ] . Alongside recent AI-oriented Nobel prizes [ 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 ] , these trends establish the role of AI tools in science [ 10 ] . This advancement raises questions about the potential influences of AI tools on scientists and science as a whole, and highlights a potential conflict between individual and collective benefits [ 11 ] . To evaluate, we used a pretrained language model to identify AI-augmented research, with an F1-score of 0.875 in validation against expert-labeled data. Using a dataset of 41.3 million research papers across natural science and covering distinct eras of AI, here we show an accelerated adoption of AI tools among scientists and consistent professional advantages associated with AI usage, but a collective narrowing of scientific focus. Scientists who engage in AI-augmented research publish 3.02 times more papers, receive 4.84 times more citations, and become research project leaders 1.37 years earlier than those who do not. By contrast, AI adoption shrinks the collective volume of scientific topics studied by 4.63% and decreases scientist’s engagement with one another by 22.00%. Thereby, AI adoption in science presents a seeming paradox—an expansion of individual scientists’ impact but a contraction in collective science’s reach—as AI-augmented work moves collectively toward areas richest in data. With reduced follow-on engagement, AI tools appear to automate established fields rather than explore new ones, highlighting a tension between personal advancement and collective scientific progress. Introduction Artificial intelligence (AI) has made significant strides in recent decades, promising to impact myriad aspects of society, including education [ 12 , 13 ] , healthcare [ 14 , 15 ] , and industry [ 16 ] . Major investments in predictive and generative AI have catalyzed society-level debates over the future of AI at home and in the workplace. Perhaps more than any other domain, AI tools have become deeply entwined with the process of knowledge production, yielding findings that attract disproportionate attention in various scientific fields [ 1 ] . For example, AlphaFold learns known protein structures to accurately predict the unexplored ones, circumventing the capital and human cost of conventional structural inference and recently granted a 2024 Nobel Prize [ 9 , 17 ] . Models improved via deep reinforcement learning have become tuned to contain complex fusion reactions [ 18 ] and discovered new, hardware-optimized forms to matrix multiplication that recursively accelerate deep learning itself [ 19 ] . Autonomous laboratory systems driven by ChatGPT have helped some chemists and material scientists upscale the number of adaptive high-throughput experiments [ 20 , 21 , 22 ] . Moreover, recent developments in large language models are making them increasingly incorporated in assisting scientific writing [ 23 , 24 , 25 , 26 ] , facilitating the distillation of scientific findings, but also raising concerns about weakened confidence in AI-generated content [ 21 , 22 , 27 ] . AI’s increasing capabilities to influence scientific research suggest that it manifests potential to both increase the productivity of individual scientists and raise the visibility of science it supports. Despite the increasing adoption of AI in science, large-scale empirical measurements of AI’s scientific impact are limited, and a detailed, dynamic understanding of AI’s impact on the entire character of science remains largely unknown. Recent work suggests AI has brought widespread benefits to individual scientists but may lead to demographic disparity resulting from gaps in AI education [ 10 ] . Researchers have also identified evolving citation patterns that signal a changing scientific landscape in AI research [ 28 ] . Here we seek to explore the impact of AI in scientific research at different scales by posing the question: How does the adoption of AI influence individual scientists’ careers and the collective exploration of science as a whole? We conduct a large-scale quantitative analysis of the impact of AI on scientists and science, covering 41,298,433 research papers spanning from 1980 to 2025 in the OpenAlex dataset [ 29 ] , with patterns corroborated using the Web of Science [ 30 , 31 ] . Notably, we do not focus on computer science or mathematics, fields that develop AI methodologies directly, but rather on papers that augment research in natural science fields by adopting AI, primarily covering decades involving development and deployment of conventional machine learning algorithms and also extending to a necessarily more preliminary analysis of the latest generative AI techniques. Specifically, we select six representative disciplines that cover the vast majority of natural science contributions—biology, medicine, chemistry, physics, materials science, and geology. We then leverage the BERT language model [ 32 , 33 ] to accurately identify such AI-augmented research papers based on their titles and abstracts. We separate the periods in which AI was predominantly conventional machine learning, deep learning, and most recently generative designs including large language models. With abundant data-based evidence across decades of conventional machine learning and deep learning, we validate these AI-based measurements and use them to reveal that the adoption of AI leads to an amplifying effect on the career of individual scientists, bringing acceleration in the production and visibility of science produced by those scientists who incorporate AI. Nevertheless, this effect corresponds with a contracted focus within collective science. Measured with “knowledge extent”, the “diameter” covered by a sampled batch of papers in vector space, AI-driven science spans less topical ground and is associated with a decrease in follow-on scientific engagement, suggesting that AI is currently more likely to focus on existing popular research problems rather than explore new ones. Meanwhile, analyses using currently available data within the latest era of generative AI including large language models reveal a preliminary consistency with prior periods, providing a starting point for further study as generative AI develops over a longer period. Results Figure 1: Increasing prevalence of AI adoption in science. (a) Increasing performance of AI paper identification during the two-stage fine-tuning of the BERT pre-trained models, where we use rough training data in Stage 1 to evolve precise assessments in Stage 2. We independently train two models based on titles and abstracts, respectively, and then integrate them into an ensemble that selects the optimal models during both stages (red stars) to identify all selected papers. (b) Accuracy evaluation of our identification results by human experts. For samples spanning three eras of AI, experts reached consensus with Fleiss’ Kappa ( \kappa ) \geq 0.93 . Our model identification results have strong accuracy in validation against expert-labeled data with an F1-score \geq 0.85 . (c) Relative adoption frequency of the top 15 AI methods across all disciplines for all selected AI development eras. (d)-(e) The growth of AI-augmented papers (d, n=41,298,433 ) and AI-adopted researchers (e, n=5,377,346 ) across the eras of ML, DL, and GAI between 1980 and 2025 in selected scientific disciplines. The y-axes are set to log-scale. (f) The average monthly growth rates for AI papers and researchers across the eras of ML, DL, and GAI across all

Read the complete text