Hallucinations Can Improve Large Language Models in Drug Discovery

Shuzhou Yuan, Michael Färber · 2025

Read Hallucinations Can Improve Large Language Models in Drug Discovery by Shuzhou Yuan, Michael Färber (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2501-13824.txt.

Introduction

Yuan and co-authors asked in 2025 whether hallucinations from language models can help drug discovery, under a head title that poses that question directly. Hallucinations are plausible but wrong text, usually seen as a fault. The team tested them as input for predicting molecule properties, an early step in drug work.

Models wrote natural language descriptions from molecule codes, and those often wrong descriptions fed sorting tasks. For some of seven tested models, accuracy rose clearly when the text was included. One model beat all baselines, and text from the strongest model gave the largest gains across models. Structural misdescriptions helped most, the authors found.

Catalog record

CREATORShuzhou Yuan, Michael Färber
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE50 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2501-13824

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hallucinations Can Improve Large Language Models in Drug Discovery Shuzhou Yuan, Michael Färber Abstract Concerns about hallucinations in Large Language Models (LLMs) have been raised by researchers, yet their potential in areas where creativity is vital, such as drug discovery, merits exploration. In this paper, we come up with the hypothesis that hallucinations can improve LLMs in drug discovery. To verify this hypothesis, we use LLMs to describe the SMILES string of molecules in natural language and then incorporate these descriptions as part of the prompt to address specific tasks in drug discovery. Evaluated on seven LLMs and five classification tasks, our findings confirm the hypothesis: LLMs can achieve better performance with text containing hallucinations as part of the input compared to prompts without hallucination or with reference description text. Notably, Llama-3.1-8B achieves an 18.35% gain in ROC-AUC compared to the baseline without hallucination. Furthermore, hallucinations generated by GPT-4o provide the most consistent improvements across models. Additionally, we conduct empirical analyses and a case study to investigate key factors affecting performance and the underlying reasons. Our research sheds light on the potential use of hallucinations for LLMs and offers new perspectives for future research leveraging LLMs in drug discovery. 1 Introduction Figure 1: Factual consistency scores evaluated using the HHEM-2.1-Open Model, which measures the degree of hallucination relative to the reference descriptions generated by MolT5 . The scores represent the average across five datasets used in this work (see § 5.1 ). With the exception of ChemLLM, all other LLMs achieve scores below 10%, indicating a high level of hallucination when generating natural language descriptions of molecules. Figure 2: Illustration of the evaluation method with an example molecule from the HIV dataset: 1. We use LLMs to generate a textual description of the molecule based on its SMILES string. 2. The generated text, which contains hallucinations, is added to the prompt, and the LLM is tasked with predicting the specific property of the molecule. The answer is constrained to “Yes” or “No” to evaluate the LLM’s performance. We highlight obvious hallucinations that are unrelated to the input using colors. Large Language Models (LLMs) have not only been extensively applied in daily life to address real-world tasks ( Ge et al. 2023 ; Yao et al. 2024 ) , but also increasingly utilized as tools or agents in scientific domains, including materials science ( Zhang et al. 2024b ) , biology ( Madani et al. 2023 ) and chemistry ( Boiko et al. 2023 ; Zhang et al. 2024a ) . However, LLMs often generate plausible yet incorrect factual information or unrelated content in reference to the source text ( Huang et al. 2023 ; Bang et al. 2023 ; Yuan and Faerber 2023 ) . The issue of hallucination has raised major concerns in the NLP community regarding their reliability and applicability ( Rawte et al. 2023 ) . While various methods have been proposed to mitigate hallucinations in LLMs ( Dhuliawala et al. 2024 ; Ji et al. 2023 ) , a novel perspective suggests that hallucinations may contribute to fostering creativity ( Lee 2023 ; Wang 2024 ) . As creativity extends beyond merely retrieving information to include recombining and expanding existing knowledge, it closely resembles hallucination ( Ye et al. 2023 ) . This creativity is vital in drug discovery, where exploring vast chemical spaces and devising innovative solutions to complex biological challenges are indispensable. The process of drug discovery involves evaluating a vast number of potential drugs and requires long-term experimental trials, making it highly time-consuming and expensive ( Gaudelet et al. 2021 ) . Prior work has leveraged machine learning to assist with this process ( Rifaioglu et al. 2018 ; Mak et al. 2024 ) , and generative models have also been utilized to facilitate drug discovery ( Chakraborty et al. 2023 ; Murakumo et al. 2023 ) . Additionally, researchers have discovered that textual descriptions in chemistry can improve models’ generalization ability ( Liu et al. 2023 ) . As discovering new drugs requires domain-specific knowledge and the creativity to identify new patterns, hallucination becomes a potential strength for discovering molecules with high-level functions and abstract properties ( Edwards et al. 2024a ) . Researchers have started exploring the potential of LLMs for tasks such as drug generation ( Pal et al. 2023 ) . Edwards et al. 2022 ; Edwards et al. 2024b used language models to generate descriptions of molecules as a first step toward leveraging AI for higher-level control over molecule design, studying the connection between molecules and natural language. While this approach helps researchers study and understand molecular structures through natural language, LLMs without domain-specific fine-tuning often generate text with hallucinations. As shown in Figure 1 , we use LLMs to generate descriptions of molecules and compare them with reference descriptions generated by the domain-specific model MolT5 ( Edwards et al. 2022 ) . The factual consistency is notably low, indicating that LLMs hallucinate extensively. However, these hallucinated descriptions may still provide valuable insights that help LLMs differentiate between molecules and support drug discovery. For instance, they might include high-level descriptions of the molecule, potential applications of the compounds, and other relevant information. We acknowledge that the exploration of leveraging hallucination, rather than mitigating it, remains unexplored. Thus, we propose the hypothesis that hallucinations can improve LLMs in drug discovery . As LLMs are sensitive to prompts, a subtle change can result in entirely different outputs ( Sclar et al. 2024 ) . Regarding the connections between language and molecules, we assume that the prompt text is of paramount importance for LLMs’ performance in drug discovery. To verify our hypothesis and investigate the effects of hallucinations on LLMs in drug discovery, we design comprehensive experiments that involve modifying the prompt formatting and input to the LLMs. As shown in Figure 2 , we first use LLMs to generate textual descriptions of molecules based on their SMILES strings ( Weininger 1988 ) . We then add the text descriptions, along with the SMILES strings of molecules, to the model’s input and provide task-specific instructions to predict whether a molecule possesses a specific function, such as antiviral properties or toxicity, depending on the dataset. For comparison, we use MolT5 to translate molecules into natural language ( Edwards et al. 2022 ) as the reference description. In our study, we evaluate seven instruction-tuned LLMs: Llama-3-8B , Llama-3.1-8B ( Dubey et al. 2024 ) , Ministral-8B ( MistralAI 2024 ) , Falcon3-Mamba-7B ( Zuo et al. 2024 ) , ChemLLM-7B ( Zhang et al. 2024a ) , GPT-3.5 , and GPT-4o ( Achiam et al. 2023 ) . The results confirm our hypothesis and show that LLM-generated hallucinations can improve models for drug discovery tasks. Notably, Llama-3.1-8B achieves an 18.35% gain in ROC-AUC compared to the SMILES baseline and a 13.79% gain compared to the MolT5 baseline. Furthermore, hallucinations generated by GPT-4o provide the greatest average improvement across models. We also investigate additional factors that potentially affect hallucination and model performance, namely model size, generation temperature, and the language of the LLM-generated hallucinations. While generation temperature has minimal impact on the observed improvements, model size plays a significant role, with larger models demonstrate

Read the complete text