NbBench: Benchmarking Language Models for Comprehensive Nanobody Tasks

Yiming Zhang, Koji Tsuda · 2025

Read NbBench: Benchmarking Language Models for Comprehensive Nanobody Tasks by Yiming Zhang, Koji Tsuda (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-02022.txt.

Introduction

Yiming Zhang and Koji Tsuda present a 2025 arXiv paper that sets a shared test for computer models of nanobodies. Nanobodies are tiny antibody pieces taken from camels and their kin, prized for their small size, toughness, and strong grip on targets. Many language models now read protein chains, but no common yardstick existed for this special family.

Their benchmark, NbBench, gathers eight tasks across nine data sets, from reading structure to predicting binding and drug-like traits. The team ran eleven models through the tests with their learned knowledge frozen. Antibody-focused models did best on binding tasks, while all models struggled to predict heat stability and binding strength as numbers. No single model won across the board. The shared setup lets future work compare new models fairly.

Catalog record

CREATORYiming Zhang, Koji Tsuda
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-02022

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

NbBench: Benchmarking Language Models for Comprehensive Nanobody Tasks Yiming Zhang, Koji Tsuda Abstract Nanobodies—single-domain antibody fragments derived from camelid heavy-chain-only antibodies—exhibit unique advantages such as compact size, high stability, and strong binding affinity, making them valuable tools in therapeutics and diagnostics. While recent advances in pretrained protein and antibody language models (PPLMs and PALMs) have greatly enhanced biomolecular understanding, nanobody-specific modeling remains underexplored and lacks a unified benchmark. To address this gap, we introduce NbBench, the first comprehensive benchmark suite for nanobody representation learning. Spanning eight biologically meaningful tasks across nine curated datasets, NbBench encompasses structure annotation, binding prediction, and developability assessment. We systematically evaluate eleven representative models—including general-purpose protein LMs, antibody-specific LMs, and nanobody-specific LMs—in a frozen setting. Our analysis reveals that antibody language models excel in antigen-related tasks, while performance on regression tasks such as thermostability and affinity remains challenging across all models. Notably, no single model consistently outperforms others across all tasks. By standardizing datasets, task definitions, and evaluation protocols, NbBench offers a reproducible foundation for assessing and advancing nanobody modeling. All datasets and source code used in our benchmark are publicly available at https://github.com/ZHymLumine/NbBench . 1 Introduction Antibodies are essential components of the adaptive immune system, capable of recognizing and neutralizing diverse antigens such as viruses, bacteria, and toxins lu2020development . This high specificity has made antibodies indispensable in therapeutics, diagnostics, and molecular biology nelson2010development . Structurally, conventional antibodies are composed of two heavy and two light chains forming a Y-shaped molecule, with the antigen-binding regions located within the variable domains schroeder2010structure . Nanobodies (also known as VHHs or single-domain antibodies ) are the antigen-binding domains of heavy-chain-only antibodies found in camelids el2025nanobody . Unlike conventional antibodies, nanobodies consist of a single variable domain and lack light chains, yet retain high binding affinity and specificity muyldermans2013nanobodies ; jovvcevska2020therapeutic . Their simplified architecture confers several advantages: small size ( 15 kDa), high solubility, strong thermal and chemical stability, and the ability to access hidden epitopes el2025nanobody ; DeMeyer2014NanobodybasedPA ; van2016nanobodies . These features make nanobodies attractive for applications in molecular imaging, therapy, and synthetic biology alexander2024discovery ; bao2021nanobody . Despite these advantages, nanobodies pose distinct modeling challenges. Their lack of paired light chains, longer and more flexible CDR3 loops muyldermans2013nanobodies , and smaller structural databases make it difficult to apply conventional antibody modeling techniques steeland2016nanobodies . While large datasets like the Observed Antibody Space (OAS) kovaltsuk2018observed ; olsen2022observed exist for conventional antibodies, nanobody-specific resources such as INDI deszynski2022indi and VHHCorpus-2M tsuruta2024sars remain relatively limited. Recent advances in protein language models (PLMs), such as ESM lin2023evolutionary , ProtBert elnaggar2021prottrans , and MSA Transformer rao2021msa , have demonstrated strong performance on structure and function prediction tasks rao2019evaluating ; madani2020progen ; meier2021language . These models leverage large-scale self-supervised training on protein sequences to learn contextual representations that capture structural and evolutionary information. Building on this, antibody-specific language models (PALMs) like AntiBERTy ruffolo2021deciphering , AbLang olsen2022ablang , AntiBERTa2 barton2024enhancing , and IgBert kenlay2024large have been developed to model immunoglobulin sequences with higher accuracy. However, their applicability to nanobody modeling remains largely unexplored, as they are predominantly trained and evaluated on conventional antibodies. More recently, nanobody-specific models such as NanoBERT hadsund2024nanobert , NanoBERTa-ASP li2024nanoberta , DeepNano deng2024nanobody , and VHHBERT tsuruta2024sars have emerged. However, these models are typically developed for narrow tasks and use inconsistent datasets and evaluation metrics, making fair comparisons difficult. Furthermore, most prior work does not assess generalization to unseen nanobody types or functions. To address these limitations, we introduce NbBench , the first comprehensive benchmark suite for nanobody modeling. NbBench spans eight biologically relevant tasks across nine curated datasets, covering structural annotation, sequence generation, function prediction, and developability assessment. It provides standardized task definitions, data splits, and evaluation metrics to facilitate fair, reproducible comparisons between general-purpose, antibody-specific, and nanobody-specific language models. Using NbBench, we evaluate eleven pretrained models and analyze their generalization performance across task types, revealing key insights into the capabilities and limitations of current language models for nanobody modeling. Overall, our contributions can be summarized as follows: • We introduce NbBench , the first comprehensive benchmark for nanobody modeling, spanning eight biologically meaningful tasks across nine curated datasets. • We systematically evaluate eleven pretrained protein, antibody, and nanobody language models, revealing their generalization capabilities and failure modes across structure, function, and developability tasks. • We establish a standardized evaluation protocol for nanobody tasks, enabling fair model comparison and providing actionable insights for future model development and deployment. Figure 1: Overview of NbBench. (a) Eight tasks spanning token- and sequence-level classification and regression. (b) Data sources include INDI, SAbDab, PLAbDab-nano, and task-specific datasets. (c) 10+ baselines from pretrained protein, antibody, and nanobody language models. (d) Visual representations of tasks grouped into structure understanding, binding, and biophysical and functional property prediction. (e) Standard evaluation metrics and clustering-based data splits. 2 Related Works Pretrained Protein and Antibody Language Models (PPLMs and PALMs). Protein language models such as ESM-2 lin2023evolutionary , ProtBert elnaggar2021prottrans , and MSA Transformer rao2021msa have demonstrated strong transfer performance on structure and function prediction tasks rao2019evaluating ; madani2020progen ; meier2021language ; chen2023structure ; zhang2025hyena . These models leverage large-scale self-supervised learning and evolutionary context to build general-purpose protein representations. In parallel, antibody-specific language models (PALMs) have been introduced to model immunoglobulin sequences more effectively. AntiBERTy ruffolo2021deciphering was the first such model trained on 588 million heavy/light chain sequences from the OAS database, revealing that antibodies can be embedded along trajectories resembling affinity maturation. AntiBERTa leem2022deciphering was fine-tuned for paratope prediction, while AbLang olsen2022ablang trained separate models on heavy (AbLang-H) and light (AbLang-L) chains, outperforming ESM-1b in recovering masked residues. BERT-DS porebski2024rapid and AntiBERTa2 barton2024enhancing further expanded training datasets to tens or hundreds of millions of antibody sequences, with AntiBERTa2 incorporating multimodal contrastive learning between sequence and structure. IgBert kenlay2024large , initialized from ProtBert, was trained on over 2 million paired antibody sequences. EATLM wang2023pre introduced evolution-guided pretraining objectives, such as germline ancestor prediction and mutation site masking, and also proposed the ATUE benchmark. Despite this rapid progress, most PALMs have focused on conventional antibodies and are primarily evaluated on limited downstream tasks such as paratope prediction, sequence recovery, or general binding classification. Nanobody modeling and current limitations. While antibody-specific language models have advanced rapidly, their applicability to nanobody modeling remains largely unexplored. Nanobodies—single-domain fragments derived from camelid antibodies—have emerged as promising tools for diagnostics, therapeutics, and molecular imaging salvador2019nanobody ; alexander2024discovery ; bao2021nanobody . Their compact size and ability to bind hidden epitopes make them attractive alternatives to conventional antibodies salvador2019nanobody ; liu2025unveiling . Classical in silico approaches—such as docking, molecular dynamics, and epitope mapping—have long been used to analyze nanobody-antigen interactions el2025nanobody ; joubbi2024antibody , but are often resource-intensive and lack scalability. Structure prediction tools like AlphaFold2 jumper2021highly and OmegaFold wu2022high improve modeling accuracy, yet still rely on structural inputs or evolutionary features valdes2023structural . Although a few deep learning models such as NanoBERT hadsund2024nanobert , NanoBERTa-ASP li2024nanoberta , and VHHBERT tsuruta2024sars have been proposed for thermostability, paratope, or binding prediction, each is designed for a narrow task and evaluated in isolation. A unified framework for evaluating nanobody modeling remains missing. Benchmarking efforts and the need for NbBench. Large-scale benchmarks have played a critical role

Read the complete text