Large Language Models for Patent Classification: Strengths, Trade-offs, and the Long Tail Effect

Lorenzo Emer, Marco Lippi, Andrea Mina, Andrea Vandin · 2026

Read Large Language Models for Patent Classification: Strengths, Trade-offs, and the Long Tail Effect by Lorenzo Emer, Marco Lippi, Andrea Mina, Andrea Vandin (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23200.txt.

Introduction

Lorenzo Emer, Marco Lippi, Andrea Mina, and Andrea Vandin test language models on patent classification. Assigning CPC codes to patents underpins large studies of technological change. The task is hierarchical, multi-label, and heavily imbalanced, with many rare categories.

Encoder models from before generative AI remain the standard for this job. The authors weigh the strengths and trade-offs of newer language models, focusing on the long tail of rare classes. Better classification of rare technologies improves how we track emerging fields. Rare and weakly represented categories are where language models are tested hardest, since encoders trained on frequency dominate there.

Catalog record

CREATORLorenzo Emer, Marco Lippi, Andrea Mina, Andrea Vandin
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE94 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-23200

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Large Language Models for Patent Classification: Strengths, Trade-offs, and the Long Tail Effect Lorenzo Emer, Marco Lippi, Andrea Mina, Andrea Vandin Abstract Patent classification into CPC codes underpins large-scale analyses of technological change but remains challenging due to its hierarchical, multi-label, and highly imbalanced structure. While pre–Generative AI supervised encoder-based models became the de facto standard for large-scale patent classification, recent advances in large language models (LLMs) raise questions about whether they can provide complementary capabilities, particularly for rare or weakly represented technological categories. In this work, we perform a systematic comparison of encoder-based classifiers (BERT, SciBERT, and PatentSBERTa) and open-weight LLMs on a highly imbalanced benchmark dataset (USPTO-70k). We evaluate LLMs under zero-shot, few-shot, and retrieval-augmented prompting, and further assess parameter-efficient fine-tuning of the best-performing model. Our results show that encoder-based models achieve higher aggregate performance, driven by strong results on frequent CPC subclasses, but struggle on rare ones. In contrast, LLMs achieve relatively higher performance on infrequent subclasses—often associated with early-stage, cross-domain, or weakly institutionalised technologies—particularly at higher hierarchical levels. These findings indicate that encoder-based and LLM-based approaches play complementary roles in patent classification. We additionally quantify inference time and energy consumption, showing that encoder-based models are up to three orders of magnitude more efficient than LLMs. Overall, our results inform responsible patentometrics and technology mapping, and motivate hybrid classification approaches that combine encoder efficiency with the long-tail coverage of LLMs under computational and environmental constraints. keywords CPC, Hierarchical Multi-label Patent Classification, LLM, Encoders Introduction Patent classification is a foundational task in the organisation and analysis of technological information. In scientometrics and patentometrics, automated classification methods are widely used to map technological domains, trace innovation trajectories, and study technological change and emergence ( 15 ; 22 ; 29 ; 35 ) . Accurate assignment of classification schemes such as the IPC or CPC is essential for patent offices and downstream users—including legal practitioners, technology analysts, and policymakers—to support prior art search, landscape analysis, and monitoring of emerging technologies ( 23 ; 28 ) . The rapid growth of global patent filings, together with the increasing granularity of classification systems, has intensified the need for accurate and scalable automated classification methods capable of handling millions of heterogeneous, multi-label, and hierarchically structured documents, while avoiding systematic biases across technological domains ( 23 ; 21 ) . Historically, automated patent classification has progressed through several methodological phases. Early approaches relied on sparse bag-of-words representations and classical machine learning algorithms, which capture surface-level lexical patterns but struggle with synonymy, polysemy, and domain-specific terminology ( 8 ; 54 ) . Subsequent work incorporated word embeddings and deep neural architectures such as convolutional and recurrent neural networks, offering more expressive representations of patent text and improved performance over traditional pipelines ( 27 ; 15 ) . More recently, transformer-based pre-trained language models such as BERT and domain-adapted variants like PatentSBERTa have become a strong baseline for patent classification, achieving robust performance on large-scale datasets across different levels of the IPC and CPC hierarchies ( 24 ; 2 ) . These encoder-based models remain comparatively efficient to fine-tune and deploy, making them attractive for large-scale patent analytics and indicator construction. In parallel, recent advances in large language models (LLMs) have reshaped natural language processing more broadly ( 5 ; 4 ) . LLMs are typically trained on massive text corpora and can be applied to downstream tasks through prompting , that is, by providing natural-language instructions and examples rather than task-specific supervised training. This enables zero-shot classification, where no labeled examples are provided, and few-shot classification, where a small number of illustrative examples is included in the prompt ( 5 ) . In addition, LLMs can be combined with external document retrieval mechanisms—commonly referred to as retrieval-augmented generation (RAG)—to ground predictions in relevant domain-specific text ( 26 ) . These capabilities call for a systematic assessment of the benefits that LLMs may offer for patent classification, especially in sparsely populated or weakly institutionalised regions of the classification space, as well as an evaluation of whether these benefits justify their substantially higher computational cost. Recent studies suggest that, despite their versatility, the effectiveness of LLMs in highly technical, domain-specific, and large-scale multi-label settings such as CPC classification remains uncertain ( 21 ; 38 ) . At the same time, recent work in scientometrics has begun to explore the use of large language models in research evaluation and knowledge mapping, often finding that LLMs can complement traditional indicator-based approaches while also introducing systematic biases and efficiency concerns ( 47 ; 49 ; 48 ; 40 ) . Moreover, inference with LLMs typically incurs substantially higher costs in terms of latency, memory usage, and energy consumption than encoder-based architectures, posing challenges for large-scale operational deployment ( 34 ) . While fine-tuning can improve LLM performance on domain-specific tasks, it requires access to open-weight models and introduces additional computational and environmental costs, making its practical value an open empirical question. This work addresses these issues by conducting a systematic comparison between strong encoder-based baselines (BERT, SciBERT, PatentSBERTa) and a diverse set of open-weight LLMs. We evaluate multiple inference strategies, including zero-shot prompting, few-shot prompting, retrieval-augmented generation (RAG), and hybrid RAG-plus–few-shot configurations, in order to assess how different forms of contextualisation affect CPC multi-label classification performance across frequent and infrequent subclasses. In addition, we apply parameter-efficient fine-tuning using Low-Rank Adaptation (LoRA) ( 18 ) to the best-performing LLM to examine whether targeted adaptation can narrow the gap with domain-specific encoder models. Finally, we quantify computational cost and energy consumption using the CodeCarbon framework ( 9 ) , enabling a principled analysis of the trade-off between predictive performance, scalability, and sustainability. To assess the robustness and generalisability of our findings across institutional contexts, we further conduct an external evaluation on an independent dataset of European Patent Office (EPO) patents, without retraining or adapting the models. Across these evaluations, we obtain four main results. First, supervised encoder-based models remain highly competitive on frequent CPC subclasses and achieve superior micro-averaged performance (which aggregates predictions over all labels and is driven by frequent subclasses), but they exhibit substantial performance degradation on rare subclasses, resulting in low macro-level (label-balanced) and hierarchical recall. Second, LLMs—particularly when augmented with few-shot prompting and lightweight retrieval—provide more balanced coverage across the CPC hierarchy and systematically improve recall on rare and infrequent subclasses. Third, these gains come at the cost of significantly higher computational and energy requirements, which limit the scalability of LLM-based approaches in large-scale classification settings. Fourth, no single modelling approach dominates across all evaluation criteria: encoder-based models offer unmatched efficiency and scalability, whereas LLMs provide complementary semantic flexibility that is most valuable in long-tail and exploratory classification scenarios. This is especially important for scientometric analyses concerned with emerging technologies, as these tend to first appear in the long tail of patent classification systems, where categories are sparse, unstable, or subject to revision ( 53 ) . By providing a controlled empirical comparison under realistic evaluation settings, this study clarifies the practical advantages and limitations of LLM-based approaches relative to established transformer architectures, and identifies when their use is justified in both research and operational patent classification contexts. Related Work This section reviews prior work on automated patent

Read the complete text