In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts
Matthias Busch, Marius Tacke, Sviatlana V. Lamaka, Mikhail L. Zheludkevich, Christian J. Cyron, Christian Feiler, Roland C. Aydin · 2026
Read In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts by Matthias Busch, Marius Tacke, Sviatlana V. Lamaka, Mikhail L. Zheludkevich, Christian J. Cyron, Christian Feiler, Roland C. Aydin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-25857.txt.
Description
Busch and co-authors test whether language models truly learn molecular properties from context. Models might instead recall memorized answers from training data. The team runs blinding experiments that strip away information step by step. Nine models from the GPT and Gemini families face three MoleculeNet datasets. Shot counts of 0, 60, and 1000 act as a further control. Controls and bootstrap intervals back the analysis. The authors find no sign of verbatim recall on the old benchmarks. Blinding does expose clashes between stored knowledge and context.
Catalog record
| CREATOR | Matthias Busch, Marius Tacke, Sviatlana V. Lamaka, Mikhail L. Zheludkevich, Christian J. Cyron, Christian Feiler, Roland C. Aydin |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 51085 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.25857