Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation
Srujan P Mule, Aniketh Garikaparthi, Manasi Patwardhan · 2026
Read Teaching Language Models to Forecast Research Success Through Comparative Idea Evaluation by Srujan P Mule, Aniketh Garikaparthi, Manasi Patwardhan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-21491.txt.
Introduction
Srujan P Mule and co-authors teach language models to forecast research success through comparative idea evaluation. Evaluating hundreds of AI-generated ideas without exhaustive experiments is a bottleneck. They ask whether models can forecast empirical success before experiments run. Given a goal and two candidate ideas, predict which performs better on benchmarks. On eleven thousand four hundred eighty-eight idea pairs, fine-tuning lifts accuracy to seventy-seven point one percent, beating GPT-5 at sixty-one point one percent.
Catalog record
| CREATOR | Srujan P Mule, Aniketh Garikaparthi, Manasi Patwardhan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 138797 |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.21491