Scientific judgment drifts over time in AI ideation
Lingyu Zhang, Mitchell Wang , Boyuan Chen · 2025
Read Scientific judgment drifts over time in AI ideation by Lingyu Zhang, Mitchell Wang , Boyuan Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-04964.txt.
Introduction
Lingyu Zhang, Mitchell Wang, and Boyuan Chen show that expert judgment of science ideas shifts over time. Most AI idea tools treat human ratings as a fixed gold standard, as if a rater would score the same idea the same way twice.
In a two-wave study, 63 active scholars gave 7,938 ratings, with one fixed control idea mixed in each time. Scores for that same idea moved a fair amount, though the weight placed on traits like novelty and clarity stayed steady. A model tuned to first-wave ratings then picked ideas that fit the drifted second-wave taste less well.
Catalog record
| CREATOR | Lingyu Zhang, Mitchell Wang , Boyuan Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 61 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-04964
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Scientific judgment drifts over time in AI ideation Lingyu Zhang, Mitchell Wang , Boyuan Chen Abstract Scientific discovery begins with ideas, yet evaluating early-stage research concepts is a subtle and subjective human judgment. As large language models (LLMs) are increasingly tasked with generating scientific hypotheses, most systems implicitly assume that scientists’ evaluations form a fixed gold standard, assuming that scientists’ judgments do not change. Here we challenge this assumption. In a two-wave study with 7,182 ratings from 57 active researchers across six scientific departments, each participant repeatedly evaluated a constant “control” research idea alongside AI-generated ideas. We show that scientists’ ratings of the very same idea systematically drift over time: overall quality scores increased by 0.61 points on a 0-10 scale (P = 0.005), and test–retest reliability was only moderate across core dimensions of scientific value, revealing systematic temporal drift in perceived idea quality. Yet the internal structure of judgment remained stable, such as the relative importance placed on originality, feasibility, clarity, and other criteria. We then aligned an LLM-based ideation system to first-wave human ratings and used it to select new ideas. Although alignment improved agreement with Wave-1 evaluations, its apparent gains disappeared once drift in human standards was accounted for. Thus, tuning to a fixed human snapshot produced improvements that were transient rather than persistent. These findings reveal that human evaluation of scientific ideas is not static but a dynamic process with stable priorities and requires shifting calibration. Treating one-time human ratings as immutable ground truth risks overstating progress in AI-assisted ideation and obscuring the challenge of co-evolving with changing expert standards. Drift-aware evaluation protocols and longitudinal benchmarks may therefore be essential for building AI systems that reliably augment, rather than overfit to, human scientific judgment. Scientific ideation is the engine of technical progress [ 21 , 8 ] . Yet the ever-expanding body of literature [ 7 , 47 ] and the rise of narrow specialization [ 24 ] have together created what Jones called the “burden of knowledge” [ 30 ] : as mastery deepens, exploring new directions becomes harder. Automating the generation of research ideas offers a way to aggregate dispersed expertise and to probe a broader space of possibilities. Because of its intellectual complexity, automated ideation has long been regarded as a grand challenge for Artificial Intelligence [ 38 , 6 , 40 ] . Early scientific-discovery systems relied on prespecified hypothesis templates [ 10 , 50 , 37 , 36 ] or curated knowledge bases [ 60 , 59 , 23 ] , which limited scale and adaptability. Recent advances in Large Language Models (LLMs) [ 9 , 3 , 53 , 13 , 54 , 55 , 15 ] have opened new possibilities. Trained on vast corpora, these models internalize extensive scientific and cultural knowledge and can be prompted in natural language to reason or hypothesize without domain-specific retraining. As a result, LLM-driven scientific ideation has rapidly emerged as a new frontier [ 40 , 51 , 58 , 64 , 28 , 65 , 25 , 46 ] . Evaluating the quality of generated ideas, however, remains a central obstacle. Implementing each idea and assessing them by outcomes is infeasible due to the resource- and time-intensive nature of scientific work. A practical alternative is to have scientists rate their potential value. However, designing evaluations to reflect genuine scientific value is difficult. Some studies use LLMs as proxies for human evaluators, either calibrated on human ratings [ 5 ] or justified by their correlation with human scores [ 40 , 22 ] , while others rely entirely on expert judgment to guide iteration [ 65 , 58 , 51 ] . In all cases, human ratings are treated as a fixed “gold standard” for evaluation. Ideation systems are optimized to match that snapshot, assuming that scientists’ criteria are stable and the resulting improvements will persist over time. That assumption is rarely questioned. In practice, the perception of what counts as a good idea changes with shifts in culture, resources, and the direction of scientific progress. At the community level, priorities evolve as new methods or crises redirect attention. At the individual level, judgments shift as researchers gain experience, encounter new information, or simply revisit earlier impressions. The variability of human judgment over time is well documented [ 31 ] . Preferences drift in recommender systems as users adapt to the very content they consume [ 34 , 18 , 32 , 12 ] . Even the same annotator can assign different scores to identical text samples when asked twice [ 33 ] . Exposure to AI-generated material further alters human standards and beliefs [ 61 , 62 , 19 ] . Such variability grows with task difficulty [ 2 , 1 ] . Early-stage research ideas are often abstract and open-ended. Evaluating them is precisely this type of difficult task. Ratings must consider originality, feasibility, and impact, but the perceived value of these dimensions can change as contexts and norms evolve. This presents a critical concern. If an ideation system is tuned to ratings from one moment in time, the resulting “improvements” may vanish once evaluative standards move. Cross-study comparisons can also be misleading to reflect changes in human baselines rather than genuine methodological gains. The risk echoes Goodhart’s law: a measure ceases to be good once it becomes a target [ 20 ] . History offers cautionary examples of what happens when human variability is ignored. In the 1990s, U.S. health systems adopted patients’ self-reported pain level as a core quality metric. As patients’ expectations of pain relief and reporting behavior changed, opioid prescriptions escalated and caused widespread harm [ 48 , 16 ] . Similar dynamics arise in LLM training. Models must balance factual accuracy and user agreement. When optimization focuses narrowly on feedback collected at a single time point, it can overshoot and amplify the tendency rewarded most strongly at that time. Recent examples include “sycophancy” in GPT-4o [ 45 , 44 ] , where models learned to agree excessively with users after alignment updates based on static preference data, followed by subsequent over-corrections once the issue was recognized [ 17 ] . In both medicine and AI, optimizing against an unmodeled, time-dependent human metric has produced fragile and unintended outcomes. The temporal dynamics of scientists’ own evaluation criteria have not been systematically examined in ideation research. Here we ask: do scientists’ evaluations of research ideas change over time? And if so, in what ways? We examine two axes: (i) leniency or strictness, captured by shifts of absolute ratings across overall quality and specific value dimensions (e.g., originality, implementability); and (ii) relative importance, reflected by how scientists weight those dimensions when forming overall judgments. We study these questions in a two-wave study with a repeated control idea rated by the same participants. Finally, we ask whether tuning an ideation system to a fixed snapshot of human criteria yields improvements that persist once evaluation standards drift. Figure 1 : Experimental design: two-wave evaluation with repeated controls and drift-aware analysis. (A) Each scientist completed two evaluation waves. In each wave, they rated six AI-generated research ideas and one repeated human-written idea (control). The repeated control provides a direct measure of test–retest