The Unreasonable Effectiveness of Open Science in AI: A Replication Study
Odd Erik Gundersen, Odd Cappelen, Martin Mølnå, Nicklas Grimstad Nilsen · 2024
Read The Unreasonable Effectiveness of Open Science in AI: A Replication Study by Odd Erik Gundersen, Odd Cappelen, Martin Mølnå, Nicklas Grimstad Nilsen (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-17859.txt.
Introduction
Odd Erik Gundersen, Odd Cappelen, Martin Mølnå, and Nicklas Grimstad Nilsen test whether famous AI studies can be repeated in a 2024 paper. Science faces a broad repeatability crisis, but how deep it runs in AI research was unclear. The team picked thirty highly cited AI studies and tried to redo them, using the original materials when shared.
Eight studies had to be dropped because data or hardware could not be obtained. Six studies were fully repeated and five in part, so half were repeated to some degree. Sharing code and data made a large gap: most such studies repeated, while few data-only studies did. Good data notes helped, but code notes mattered surprisingly little. The study backs open science habits.
Catalog record
| CREATOR | Odd Erik Gundersen, Odd Cappelen, Martin Mølnå, Nicklas Grimstad Nilsen |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-17859
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Unreasonable Effectiveness of Open Science in AI: A Replication Study Odd Erik Gundersen, Odd Cappelen, Martin Mølnå, Nicklas Grimstad Nilsen Abstract A reproducibility crisis has been reported in science, but the extent to which it affects AI research is not yet fully understood. Therefore, we performed a systematic replication study including 30 highly cited AI studies relying on original materials when available. In the end, eight articles were rejected because they required access to data or hardware that was practically impossible to acquire as part of the project. Six articles were successfully reproduced, while five were partially reproduced. In total, 50% of the articles included was reproduced to some extent. The availability of code and data correlate strongly with reproducibility, as 86% of articles that shared code and data were fully or partly reproduced, while this was true for 33% of articles that shared only data. The quality of the data documentation correlates with successful replication. Poorly documented or miss-specified data will probably result in unsuccessful replication. Surprisingly, the quality of the code documentation does not correlate with successful replication. Whether the code is poorly documented, partially missing, or not versioned is not important for successful replication, as long as the code is shared. This study emphasizes the effectiveness of open science and the importance of properly documenting data work. Introduction There is evidence that science is undergoing a reproducibility crisis ( Baker 2016 ) . ( Ioannidis 2005 ) claims that most published research findings are false. Research in artificial intelligence (AI) is not spared ( Hutson 2018 ; Haibe-Kains et al. 2020 ; Gundersen 2020 ) , and machine learning-based science, which is science that uses machine learning algorithms, is severely affected ( Kapoor and Narayanan 2023 ) . Some even argue that AI might be one of the causes of irreproducibility ( Ball 2023 ) . However, it is not clear how much science is reproducible. Notable examples of studies that have tried to estimate the reproducibility exist for psychology ( Collaboration 2015 ) , social science ( Camerer et al. 2018 ) , economics ( Camerer et al. 2016 ) , medicine ( Prinz, Schlange, and Asadullah 2011 ) , and social psychology ( Klein et al. 2014 ) . ( Raff 2019 ) estimated the reproducibility of machine learning to be 63.5% by attempting to independently reproduce 255 articles by reimplementing the algorithms, while ( Gundersen and Kjensmo 2018 ) estimated the reproducibility of research published in AAAI and IJCAI to be 26% indirectly by analyzing 400 research articles. Figure 1: An R3 type reproducibility study is conducted where only the research report and data are shared publicly, so the code implementing both the AI method and experiment as well as hardware an ancillary software differ from the original experiment, as illustrated by differing icons. Many assumptions must be made when conducting a reproducibility study as the research report cannot possibly cover all design decisions. Difference in implementation as well as differences in hardware and ancillary software can introduce differences between the output produced by the original and the reproducibility experiments. Sometimes the differences lead to different conclusions even after doing the exact same analysis. This is indicated for one of the reproducibility experiments by the black light bulb icon. When the original article describes several experiments, the overall result of the reproducibility study depends on whether all the conclusions of all these experiments agree or not. If only a subset of the conclusions is the same, the reproducibility study is considered Partial Success . Many studies in machine learning have investigated reproducibility in various ways. ( Pham et al. 2020 ) and ( Zhuang et al. 2022 ) investigate how different sources of variability affect the results of neural networks, and ( Gundersen et al. 2023 ) investigate how variability affects conclusions. Natural language processing is the focus of ( Melis, Dyer, and Blunsom 2018 ) , who find that standard LSTM architectures, when properly regularized, outperform more recent models, and ( Belz et al. 2021 ) found that replication under the same conditions yields results that are more often worse than results that are better than those reported. ( Lucic et al. 2018 ) did not find any evidence that the algorithms they tested consistently outperformed ( Goodfellow et al. 2014 ) , although this was claimed in the articles introducing these algorithms. ( Bouthillier, Laurent, and Vincent 2019 ) focus on image recognition. They note the lack of clarity on whether exploratory or confirmatory research is reported and argue the need to increase the rigor of empirical research in deep learning. ( Varoquaux and Cheplygina 2022 ) argue that as long as there is no incentive for clarity in publications, researchers optimize for publication. Therefore, publication norms should be improved. The empirical results presented in ( Makridakis, Spiliotis, and Assimakopoulos 2018 ) stress the need for objective and unbiased ways to test the performance of forecasting methods, while ( Dacrema, Cremonesi, and Jannach 2019 ) investigates recommender systems and finds that less progress is made than what is reported. ( Henderson et al. 2018 ) investigate reinforcement learning and illustrates several practices that lead to poor reproducibility. ( Werner et al. 2024 ) studies how to extend machine learning research by reexecuting, reimplementing, and evaluating an algorithm on another dataset. ( Arvan, Pina, and Parde 2022 ) reproduce eight NLP papers based on open code and data at a success rate of 25% and propose that the complete experiment is shared as self-contained and executable artifacts. ( Gundersen et al. 2022 ) provides an overview of 41 design decisions that could be sources of irreproducibility. The purpose of this study is to shed light on what is important in ensuring the reproducibility of artificial intelligence research. Evidence is found by attempting to reproduce 30 AI studies, documenting the problems encountered during reproduction of them, and analyzing which of the problems lead to irreproducible studies. Our main contributions are: • A systematic replication study of 30 AI studies that identifies 20 different problem types and their correlation with irreproducibility. • Empirically establishing the importance of sharing both code and data to ensure the reproducibility of AI research. We also found that the quality of the data documentation has a higher correlation with reproducibility than the quality of the code documentation. • We estimate the reproducibility of AI research to be 33% when data from the original experiment are available and 86% when both code and data are available. Table 1: Overview of the results from the reproducibility studies where Compl. is Completed, Ident. is Identical, Cons. is Consistent and Fail. is Failures. Results of the reproducibility studies can be Success (S), Partial Success (PS), Failed (F), No Result (NR), and Not Started (NS). Paper with ID 23 through ID 30 were R1 studies that were not started (NS). Reference Type Time Total Compl. Ident. Cons. Fail. Cause Result ( Alexe, Deselaers, and Ferrari 2012 ) R4 40 18 22% 0% 22% 0% Time PS ( Chen et al. 2016 ) R3 40 4 50% 25% 0 25% Time PS ( Li, Niu, and Xiao 2012 ) R3 40 46 100% 37% 37%