In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts

Matthias Busch, Marius Tacke, Sviatlana V. Lamaka, Mikhail L. Zheludkevich, Christian J. Cyron, Christian Feiler, Roland C. Aydin · 2026

Read In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts by Matthias Busch, Marius Tacke, Sviatlana V. Lamaka, Mikhail L. Zheludkevich, Christian J. Cyron, Christian Feiler, Roland C. Aydin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25857.txt.

Description

Busch and co-authors test whether language models truly learn molecular properties from context. Models might instead recall memorized answers from training data. The team runs blinding experiments that strip away information step by step. Nine models from the GPT and Gemini families face three MoleculeNet datasets. Shot counts of 0, 60, and 1000 act as a further control. Controls and bootstrap intervals back the analysis. The authors find no sign of verbatim recall on the old benchmarks. Blinding does expose clashes between stored knowledge and context.

Catalog record

CREATORMatthias Busch, Marius Tacke, Sviatlana V. Lamaka, Mikhail L. Zheludkevich, Christian J. Cyron, Christian Feiler, Roland C. Aydin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE51085
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.25857

Browse the full library · About the Tower of Babel