Designing reproducible large-language-model-assisted scientific analyses
Casey W. Dunn, Darrin T. Schultz, Jacob M. Musser · 2026
Read Designing reproducible large-language-model-assisted scientific analyses by Casey W. Dunn, Darrin T. Schultz, Jacob M. Musser (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/pmc-13576654.txt.
Description
Language models now write the code and run the analyses. Casey Dunn, Darrin Schultz, and Jacob Musser ask what reproducibility means when the analyst is a moving target, in a 2026 paper. Their key distinction: whether the model sits on the data path, an active step the published result depends on, or off it, producing durable artifacts like code.
An on-path model becomes part of the runtime, and runtimes get deprecated, changed, or withdrawn. From that observation the authors derive six recommendations, headed by keeping the model off the data path where possible, preserving its artifacts, and recording its exact version. They also advise verifying results by methods matched to the model's role and considering open-weight models that cannot vanish. The paper is a rulebook for a new reproducibility hazard.
Catalog record
| CREATOR | Casey W. Dunn, Darrin T. Schultz, Jacob M. Musser |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46 KB |
| VERSION | 1.0 |
| LICENSE | CC BY 4.0 |
| RIGHTS | CC BY |
| SOURCE | Europe PMC |