Evaluating Retrieval-Augmented Generation Agents for Autonomous Scientific Discovery in Astrophysics

Xueqing Xu, Boris Bolliet, Adrian Dimitrov, Andrew Laverick, Francisco Villaescusa-Navarro, Licong Xu, Íñigo Zubeldia · 2025

Read Evaluating Retrieval-Augmented Generation Agents for Autonomous Scientific Discovery in Astrophysics by Xueqing Xu, Boris Bolliet, Adrian Dimitrov, Andrew Laverick, Francisco Villaescusa-Navarro, Licong Xu, Íñigo Zubeldia (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-07155.txt.

Description

Xueqing Xu and co-authors test AI research helpers in a 2025 paper on astrophysics. Their helpers pair a language model with a search over stored texts, so answers can draw on real sources. The team built one hundred five cosmology questions with known answers, then ran nine helper setups against them.

A human expert graded all nine hundred forty-five answers by hand. The best setup, using one firm's search and writing models, reached just over ninety-one percent accuracy. The team then tuned an automated judge against the human grades, so future tests can scale to thousands of questions. All data and tools were released for the field. The work guides a larger multi-helper system for fresh discovery.

Catalog record

CREATORXueqing Xu, Boris Bolliet, Adrian Dimitrov, Andrew Laverick, Francisco Villaescusa-Navarro, Licong Xu, Íñigo Zubeldia
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-07155

Browse the full library · About the Tower of Babel