Hallucinations in AlphaFold 3 for Intrinsically Disordered Proteins with disorder in Biological Process Residues

Shreya Gopalan; Sundaraparipurnan Narayanan · 2025

Read Hallucinations in AlphaFold 3 for Intrinsically Disordered Proteins with disorder in Biological Process Residues by Shreya Gopalan; Sundaraparipurnan Narayanan (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2510-15939.txt.

Introduction

Shreya Gopalan and Sundar Narayanan audit AlphaFold3 on proteins that have no fixed shape. Intrinsically disordered proteins make up a large share of the human proteome and drive transcription, signalling, and disease, yet their floppy regions defy the folded structures predictors handle best. The pair test seventy-two such proteins curated from the DisProt database.

Predictions run across several random seeds and ensemble outputs, and per-residue confidence scores are compared with experimental disorder labels. About a third of residues disagree with the record, counting cases where the model invents order in disordered stretches or the reverse. A further share shows context-driven mismatch, and residues tied to biological processes hallucinate at a notable rate. The findings warn drug and disease researchers to treat predicted order in such regions with care.

Catalog record

CREATORShreya Gopalan; Sundaraparipurnan Narayanan
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE23 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2510-15939

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Hallucinations in AlphaFold 3 for Intrinsically Disordered Proteins with disorder in Biological Process Residues Shreya Gopalan; Sundaraparipurnan Narayanan Abstract Protein structure prediction has advanced significantly with the introduction of AlphaFold3, a diffusion-based model capable of predicting complex biomolecular interactions across proteins, nucleic acids, small molecules, and ions. While AlphaFold3 demonstrates high accuracy in folded proteins, its performance on intrinsically disordered proteins (IDPs)—which comprise 30–40 percent of the human proteome and play critical roles in transcription, signaling, and disease—remains less explored. This study evaluated AlphaFold3’s predictions of IDPs with a focus on intrinsically disordered regions (IDRs) using 72 proteins curated from the DisProt database. Predictions were generated across multiple random seeds and ensemble outputs, and residue-level pLDDT scores were compared with experimental disorder annotations. Our analysis reveals that 32 percent of residues are misaligned with DisProt, with 22 percent representing hallucinations where AlphaFold3 incorrectly predicts order in disordered regions or vice versa. Additionally, 10 percent of residues exhibited context-driven misalignment, suggesting that AlphaFold3 implicitly incorporates stable structural assumptions. Importantly, 18 percent of residues associated with biological processes showed hallucinations, raising concerns about downstream implications in drug discovery and disease research. These findings highlight the limitations of AlphaFold3 in modeling IDRs, the need for refined hallucination metrics beyond the pLDDT, and the importance of integrating experimental disorder data to improve the prediction reliability. 1 Introduction 1.1 Significance of Protein Structure Prediction and AlphaFold 3 advancements Protein structure prediction is of great importance because of its varied downstream applications in drug discovery and disease studies, along with the economic challenges of using experimental techniques to determine protein structure. AlphaFold3, developed by Google DeepMind in collaboration with Isomorphic Labs, is a transformative advancement in biomolecular (protein) prediction that allows progress in drug design and therapeutics Desai et al. (2024) . Advancements include a diffusion-based architecture capable of predicting complex biomolecular interactions, including proteins, nucleic acids, small molecules, ions, and modified residues Abramson and others (2024) . AlphaFold is consistently evolving with widespread adoption in the domains of structural biology, and the open-sourcing of AlphaFold3 in November 2024 is a significant contribution to research progress. In addition, its predictions have been consistently validated against experimental studies Kovalevskiy et al. (2024) . 1.2 Intrinsically Disordered Proteins (IDPs) and Regions (IDRs) The success of AlphaFold 3 and its earlier versions in folded proteins is discussed while expressing its accuracy; however, the accuracy of intrinsically disordered proteins is limited. Intrinsically disordered proteins (IDP) are proteins which lack a stable three-dimensional structure at physiological conditions but are fully functional. The flexibility in the intrinsically disordered regions (IDRs) of the IDP offers the scope to interact with multiple targets Sato (2022) . One of the fundamental aspects of understanding is that disorders are not necessarily binary in nature, and may be represented based on context-dependent diverse properties and functions. These properties, when observed in a specific context, exhibit variability in parameters such as pH, localization, binding, and post-translational modifications Aspromonte and others (2024) . The biological relevance of IDPs and Intrinsically Disordered Regions (IDR) is being studied further, considering their prevalence in the eukaryotic genome (60%). Furthermore, IDP’s play an indispensable role in biological processes such as transcription, translation, cell cycle, and signaling, and in turn, diseases. Intrinsically disordered proteins (IDPs), comprising 30–40% of the proteome, are critical in diseases including neuro-degenerative disorders and cancer. For instance, 80% of human cancer-associated proteins have long IDRs (e.g., p53 contains 50% IDR in its sequence) Wallin (2017) Lermyte (2020) Sato (2022) . Coskuner-Weber (2024) exhibited that neither AlphaFold2 nor AlphaFold3 did not fully capture structural characteristics of α-synuclein, an intrinsically disordered protein. Earlier studies have identified that Alphafold 2 demonstrates promising results in IDR, specifically with reference to conditionally folding binding regions/residues Piovesan et al. (2022) . Researchers have expressed some possible limitations in terms of chirality determination, conformation, modelling experimental parameters, and predicting structural flexibility through AF3 Krokidis and others (2025) Fang and others (2025) Abramson and others (2024) . However, there is no extensive research that specifically analyses AlphaFold3’s performance against that of IDR. AlphaFold3 has introduced a diffusion-based probabilistic model, which, while improving the prediction efficiency of many biomolecules and their interactions, has also increased the rate of hallucinations Abramson and others (2024) Fang and others (2025) . 1.3 Research Objectives and Scope This research attempts to conduct such an analysis of IDP’s with a focus on IDR’s that have an impact on biological processes, with a focus on hallucinations that arise in predicting disordered regions. Further, given the learnings from AlphaFold2 research studies and AlphaFold3 diffusion-based ensemble architecture, this research attempts to cover implications of reproducibility and accuracy with reference to context-dependent IDRs. 2 Methodology 2.1 DisProt Database DisProt is a database that links the structural and functional information of intrinsically disordered proteins Aspromonte and others (2024) . DisProt is the primary database manually curated and annotated with experimental results for IDPs and IDRs and has evolved with over 3000 IDPs. DisProt contains annotations relating to functional and structural aspects of disorders like disorder state, structural transition, biological processes, and molecular function based on experimental validation 2.2 Protein Selection Criteria This research leverages the DisProt database to identify relevant IDPs for the research and associated information. The study selected 72 proteins for this study from the DisProt database by identifying proteins with Bio-Process and having IDRs greater than 75% overall. 2.3 AlphaFold3 Prediction Setup The AlphaFold server generated predictions for the identified IDPs. In order to understand the five ensemble variabilities of AlphaFold3, we experimented with seed variations to observe the extent of variability in predictions for IDRs. To that end, three types of seeds were provided as input while generating the predictions on the AlphaFold Server as follows: (1) No seed was provided manually. The AlphaFold server automatically attributes a random seed. (2) 5 as seeds and (3) 1234567890 as seeds. This approach resulted in three seed variations and five ensemble outputs for each, thereby having 15 model prediction outputs to evaluate reproducibility across prediction runs for each identified IDP. 2.4 Analysis of pLDDT Scores and Variability The CIF files from the predictions were used to extract the pLDDT scores from the B-factor field programmatically. The pLDDT is a confidence score that indicates the reliability of the structural predictions for each residue, with scores above 70 typically considered to indicate high confidence Williams et al. (2025) . Furthermore, pLDDT scores represent protein disorder with a strong correlation with IDRs Bruley et al. (2022) . The pLDDT scores were parsed to obtain the residue-level scores for each protein. These scores were compared for each seed to determine the seed-based variation for each protein. pLDDT scores greater than 70 were considered as ‘ordered’ and less than 70 were considered as ‘disordered’ in correlation with the DisProt database. An analysis of the variance in predictions based on seed-based input variations for the identified IDP in determining the variability influenced by different seed-based protein predictions. 2.5 Classification of Predictions A systemic comparison between the IDRs (at a residue level) for each protein from the DisProt database and the predictions (ordered/disordered using pLDDT scores) supported the determination of how well the AlphaFold3 protein predictions align with the ordered/disordered references from DisProt. Based

Read the complete text