Efficient Patent Searching Using Graph Transformers

Krzysztof Daniell, Igor Buzhinsky, Sebastian Björkqvist · 2025

Read Efficient Patent Searching Using Graph Transformers by Krzysztof Daniell, Igor Buzhinsky, Sebastian Björkqvist (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-10496.txt.

Introduction

Krzysztof Daniell, Igor Buzhinsky, and Sebastian Björkqvist build a patent search tool meant to find prior art fast. Their paper from two thousand twenty-five notes the hard part: patent files are huge in number, and judging what is new takes fine comparisons. A good search tool can speed filing and review work a great deal.

Each invention is drawn as a graph of its features and how they link. A graph transformer reads those graphs, trained on citation choices made by patent examiners. That lets the tool learn domain likeness beyond plain text matching. In tests, it beat public text embedding models on both search quality and speed.

Catalog record

CREATORKrzysztof Daniell, Igor Buzhinsky, Sebastian Björkqvist
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE21 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2508-10496

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Efficient Patent Searching Using Graph Transformers Krzysztof Daniell, Igor Buzhinsky, Sebastian Björkqvist Abstract Finding relevant prior art is crucial when deciding whether to file a new patent application or invalidate an existing patent. However, searching for prior art is challenging due to the large number of patent documents and the need for nuanced comparisons to determine novelty. An accurate search engine is therefore invaluable for speeding up the process. We present a Graph Transformer-based dense retrieval method for patent searching where each invention is represented by a graph describing its features and their relationships. Our model processes these invention graphs and is trained using prior art citations from patent office examiners as relevance signals. Using graphs as input significantly improves the computational efficiency of processing long documents, while leveraging examiner citations allows the model to learn domain-specific similarities beyond simple text-based matching. The result is a search engine that emulates how professional patent examiners identify relevant documents. We compare our approach against publicly available text embedding models and show substantial improvements in both prior art retrieval quality and computational efficiency. [ [email protected], orcid=0009-0006-5959-1804 ] [ [email protected],orcid=0000-0003-3713-6051 ] [ [email protected], orcid=0009-0006-9039-8623 ] keywords patent search ,prior art search ,information retrieval ,dense retrieval ,graph neural networks † † copyrightyear: 2025 † † copyright: Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). † † venue: PatentSemTech’25: 6th Workshop on Patent Text Mining and Semantic Technologies † † address: IPRally Technologies Oy, Helsinki, Finland 1 Introduction Companies and individuals can protect their intellectual property by filing a patent application for their inventions. When granted, a patent provides its holder exclusive rights to the invention for a limited time. In exchange, the invention is publicly disclosed. The patent application process can be quite costly and time-consuming, so applicants typically seek a high certainty of it eventually being granted before initiating the application process. Since patents are granted only for novel inventions, conducting a prior art search is important to avoid unnecessary costs and delays. Finding relevant prior art is challenging due to the large number of existing patent documents and the detailed analysis required to distinguish novelty-destroying prior art from merely similar inventions. An effective patent search engine will both speed up the process and improve the results. In this work, we present an approach for patent searching using graph representations of inventions as input to a Graph Transformer network. The graph of a patent document describes the core of the invention disclosed in that document, namely the features and the relationships between them. This representation condenses the original document, significantly reducing the computational resources required to process the document. The graphs are used as input to a Graph Transformer model trained on patent office examiner citations created during the application process for a patent. This enables the model to capture relationships between similar inventions, even when described using different terms. Furthermore, examiner citations help the model learn domain-specific terminology across different technical domains. The result is a patent search engine that efficiently retrieves relevant novelty-destroying prior art. Figure 1: Example patent claim (US20170152638A1) converted into a graph. 2 Background 2.1 Patent Search Methods Traditionally, patent searching has relied on Boolean search, where text-based matching is done using Boolean operators like AND, OR, and NOT Clarke 2018 . Multiple Boolean search tools for patent searching exist, both paid Minesoft 2025 ; Patsnap 2025 ; Questel 2025 and free Google 2025 ; EPO 2025 ; USPTO 2025 . Performing high-quality Boolean searches often requires multiple iterations and substantial domain expertise to identify the most relevant terminology Ali et al. 2024 . Machine learning methods have been explored for patent searching to address the limitations of Boolean searching. Traditional approaches including TF-IDF Herbert et al. 2010 ; D’hondt and Verberne 2010 and word embeddings Helmers et al. 2019 have been applied. More recently, deep learning models such as BERT Ghosh et al. 2024 ; Acikalin and Kutlu 2022 ; Freunek and Bodmer 2021 and GPT-2 Lee and Hsiang 2021 have also been explored. In addition, several commercial machine learning-based patent search tools are available Ambercite 2025 ; Amplified AI 2025 ; IPScreener 2025 . In our previous work Björkqvist and Kallio 2023 , we describe a graph-based patent search engine where a Tree-LSTM model is trained for dense retrieval using patent office examiner citations as relevance signals. This paper extends that work by replacing the Tree-LSTM model with a Graph Transformer and improving the training procedure, as detailed in Section 3 . 2.2 Graph Transformers Graph Transformers adapt the Transformer architecture to graph-structured data Shehzad et al. 2024 . Early models, such as Graphormer Ying et al. 2021 and SAN Kreuzer et al. 2021 , used full attention over all node pairs, which becomes computationally prohibitive as graph size grows—a critical concern for large patent corpora. Sparse Graph Transformers Dwivedi and Bresson 2021 address this by restricting attention to actual edges or small neighborhoods, drastically reducing overhead while preserving power for long-range dependencies. This is useful for patent searching where long documents revolving around a limited set of distinct features or relationships are typical. Mapping these concepts to nodes, with edges only where genuine conceptual links exist, yields a much sparser graph than a fully connected alternative. 2.3 Context-Based Representations A recent approach in language modeling is using concept-level representations instead of token-based architectures. Large Concept Models team et al. 2024 can be used to embed entire sentences—or “concepts”—into a high-dimensional, language-agnostic space rather than embedding every token separately. This perspective parallels our approach, where each key textual segment (e.g., a feature of the invention) is treated as a “concept node.” We reduce computational overhead by applying sparse attention to these conceptual links yet still preserve the broad relationships essential for performing patent searches. 3 Approach Our patent search engine consists of two main parts: We first convert each patent document into a graph that describes the invention, as described in Section 3.1 . We then use a Graph Transformer model, described in Section 3.2 , to embed the graph into a vector space for dense retrieval. The Graph Transformer model is trained for patent searching using patent examiner citations as relevance signals, as described in Sections 3.3 and 3.4 . The main differences to our previous work Björkqvist and Kallio 2023 are replacing the Tree-LSTM model with a Graph Transformer and enhancements to the training procedure. 3.1 Graphs We convert each patent document into a graph to capture its core features and relationships while avoiding the overhead of full-text processing. An example of a graph is seen in Figure 1 . Each node corresponds to a key feature of the invention (e.g., snowthrower or motor ) or a text snippet indicating a relationship (e.g., frame for connecting handle device and auger housing ). These relationships include hierarchical ones—such as part-of (meronym) or example-of (hyponym)—and functional ones, linking specific features to describe how they interoperate. By focusing on the essential technical structure, the graph becomes much smaller than the raw text while still conveying the essential features of the invention. This method aligns with how professional examiners perform novelty evaluations: they isolate core features and examine

Read the complete text