PhageBench: Can LLMs Understand Raw Bacteriophage Genomes?

Yusen Hou, Weicai Long, Haitao Hu, Houcheng Su, Junning Feng, Yanlin Zhang · 2026

Read PhageBench: Can LLMs Understand Raw Bacteriophage Genomes? by Yusen Hou, Weicai Long, Haitao Hu, Houcheng Su, Junning Feng, Yanlin Zhang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05775.txt.

Introduction

Hou and five colleagues test language models on raw phage genomes with PhageBench, copying expert bioinformatics workflows. Across 5,600 samples in five tasks, thinking models beat chance baselines on contig spotting and host calls. Long-range thinking and fine-grained pinpointing stay weak, pointing to a need for sequence-savvy next-generation models.

Catalog record

CREATORYusen Hou, Weicai Long, Haitao Hu, Houcheng Su, Junning Feng, Yanlin Zhang
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE57960
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05775

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

PhageBench: Can LLMs Understand Raw Bacteriophage Genomes? Yusen Hou, Weicai Long, Haitao Hu, Houcheng Su, Junning Feng, Yanlin Zhang Abstract Bacteriophages, often referred to as the dark matter of the biosphere, play a critical role in regulating microbial ecosystems and in antibiotic alternatives. Thus, accurate interpretation of their genomes holds significant scientific and practical value. While general-purpose Large Language Models (LLMs) excel at understanding biological texts, their ability to directly interpret raw nucleotide sequences and perform biological reasoning remains underexplored. To address this, we introduce PhageBench, the first benchmark designed to evaluate phage genome understanding by mirroring the workflow of bioinformatics experts. The dataset contains 5,600 high-quality samples covering five core tasks across three stages: Screening, Quality Control, and Phenotype Annotation. Our evaluation of eight LLMs reveals that general-purpose reasoning models significantly outperform random baselines in phage contig identification and host prediction, demonstrating promising potential for genomic understanding. However, they exhibit significant limitations in complex reasoning tasks involving long-range dependencies and fine-grained functional localization. These findings highlight the necessity of developing next-generation models with enhanced reasoning capabilities for biological sequences 1 1 1 Our PhageBench dataset and evaluation code are available at https://anonymous.4open.science/r/PhageBench-D428 . $\dagger$ $\dagger$ footnotetext: Co-first authors Figure 1: PhageBench challenges general-purpose LLMs to perform five phage analysis tasks, processing raw nucleotide sequence inputs to generate reasoning-based phenotypic predictions. 1 Introduction The intersection of artificial intelligence and biology has shifted from analyzing biological texts to decoding, the language of life, DNA sequences. Recent Genomic Foundation Models (GFMs) such as, Evo ( 26 ; 7 ) and Caduceus ( 37 ) , have demonstrated that deep learning architectures can capture the complex syntax and semantics of DNA. These models can predict gene expression ( 19 ; 5 ) and even design whole genome of functional bacteriophages (phages) ( 21 ) . However, these specialized models require extensive pre-training on DNA sequence corpora, similar to how general LLMs are trained on large text corpora. In parallel, general-purpose Large Language Models (LLMs) have exhibited strong reasoning abilities and performance on various biology-related tasks ( 34 ; 42 ; 46 ) . This raises a critical question: Can general-purpose LLMs understand raw DNA sequences without specific pre-training? To address this question, we propose a new challenge for the natural language processing (NLP) community: phage genome annotation. We select phages, viruses that infect bacteria, as the testbed for three reasons. First, the length of a typical phage genome ranges from approximately 3 to 150 kilobase pairs (kb) 12 , which aligns well with the context windows of modern LLMs. This makes phages an ideal subject for evaluating non-natural language long-context reasoning. Second, unlike eukaryotic genomes which contain long non-coding introns, phage genomes exhibit a dense, linear organization where genes are arranged in a structured syntax similar to natural language 22 . Previous studies indicate that this structural conservation allows for functional prediction based on genomic position even when sequence homology is low ( 17 ) . This allows us to test whether LLMs can infer function in the absence of explicit sequence matches. Third, phages are the most abundant biological entities in the biosphere and play critical roles in regulating microbial ecosystems ( 35 ) , food engineering ( 33 ) , and phage therapy as alternatives to antibiotics ( 41 ; 40 ) . However, the vast majority of phage sequences identified in metagenomic data remain unannotated dark matter lacking basic taxonomic and phenotypic information. This annotation bottleneck limits our ability to harness phages for therapeutic and biotechnological applications. Therefore, achieving efficient and accurate phage genome annotation is of significant scientific and practical importance. To systematically investigate the ability of LLMs to understand phage genomes, we introduce PhageBench, a multi-task benchmark designed to mirror the complete workflow of expert phage analysis. PhageBench follows a standard structure across three stages: Screening (phage contigs identification from environmental noise), Quality Control (contamination detection and completeness estimation), and Phenotype Annotation (lifestyle prediction and host prediction) (Fig. 1 ). This design reflects how biologists progressively filter and characterize sequences, moving from simple identification to complex biological inference. The benchmark contains over 5,600 samples across five tasks, with careful controls for sequence length distribution and class balance to ensure models cannot rely on statistical biases. We evaluate eight advanced LLMs on PhageBench, including reasoning models. Our analysis reveals that general-purpose reasoning LLMs demonstrate promising potential in phage identification and host prediction tasks without domain-specific pre-training. However, they exhibit limitations in modeling long-range dependencies, leading to declines in performance in completeness estimation tasks involving global structures as sequence length increases. Furthermore, although these models are capable of applying accurate biological logic during reasoning, they struggle to precisely identify functional gene fragments within raw nucleotide sequences spanning tens of thousands of base pairs. To summarize, our contributions are: • We first introduce raw phage genome annotation as a challenging testbed for evaluating the reasoning capabilities of general-purpose LLMs. This task requires models to process long-context sequences and perform zero-shot logical reasoning, providing a proxy for measuring LLM capabilities in long-range and complex syntactic analysis in non-natural language domains. • We construct and release PhageBench, a comprehensive dataset containing 5,600 high-quality samples. PhageBench mirrors the real workflow of bioinformatics experts, covering three critical stages in phage genome annotation. • By applying general-purpose LLMs to raw phage genome annotation, PhageBench provides a novel analysis pathway for genomics that does not depend on sequence homology. Our evaluation demonstrates the potential of LLMs for understanding raw genomic sequences, offering preliminary evidence for the feasibility of using general reasoning engines to accelerate the discovery, annotation, and therapeutic development of unknown phages in metagenomic data. Figure 2: The overview of data construction of PhageBench. We filter raw sequences from public repositories, apply task-specific processing rules, and format the resulting samples into standardized instructions for evaluation. For detailed procedures for each task, please refer to Appendix B.1 . 2 Related Work 2.1 Phage Genome Analysis The computational analysis of phage genomes has evolved from alignment-based sequence matching to feature-learning approaches. Traditional methods, such as BLAST 3 , rely on homology search against reference databases like GenBank ( 6 ) . While accurate for known sequences, these methods fail to identify sequences with low homology. To address this, learning-based approaches have been developed 8 ; 32 . VirSorter2 18 integrates multiple classifiers to capture diverse viral signals, while CheckV 24 assesses genome completeness based on gene content and terminal repeats. However, these discriminative models typically output binary classifications without biological reasoning and often specialized for a single task. Our work investigates whether general-purpose LLMs can bridge this gap by leveraging their inherent reasoning capabilities to analyze raw genomic sequences directly. 2.2 Genomic Foundation Models and Benchmarks The intersection of AI and genomics has spurred the development of specialized GFMs 48 ; 27 ; 11 . Models such as DNABERT 20 apply Transformer architectures to DNA sequences. Recent generative models, such as Evo, have demonstrated the ability to design whole phage genomes 21 . However, these specialized models require resource-intensive pre-training on massive genomic corpora. While some efforts, such as L2G 9 , explore repurposing general LLMs for genomics, they still rely on extensive fine-tuning. Existing genomic benchmarks, such

Read the complete text