Tracing the Flow of Knowledge From Science to Technology Using Deep Learning

Michael E. Rose, Mainak Ghosh, Sebastian Erhardt, Cheng Li, Erik Buunk, Dietmar Harhoff · 2025

Read Tracing the Flow of Knowledge From Science to Technology Using Deep Learning by Michael E. Rose, Mainak Ghosh, Sebastian Erhardt, Cheng Li, Erik Buunk, Dietmar Harhoff (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2512-24259.txt.

Introduction

Michael Rose and six colleagues trace how knowledge flows from papers into patents. They build a language model that handles both text types at once. Patents and papers use different styles. A shared similarity measure must bridge them. The team tests eight candidate models in a horse race. Each model predicts credible patent-paper citations.

Their own model, Pat-SPECTER, wins the race. It fine-tunes a scientific document model on patent text. Two real-world scenarios show its skill. It separates true patent-paper pairs from decoys. It also predicts new pairs directly. A final test checks a legal hypothesis. United States patents may cite less similar papers than other jurisdictions do. The duty of candor could explain that pattern. The model is open to academics and practitioners.

Catalog record

CREATORMichael E. Rose, Mainak Ghosh, Sebastian Erhardt, Cheng Li, Erik Buunk, Dietmar Harhoff
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2512-24259

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Tracing the Flow of Knowledge From Science to Technology Using Deep Learning Thanks: Our work has been enabled by grant 2023/8401 from the European Patent Office under their Academic Research Programme (EPO-ARP) 2021. The project has benefited from discussions with Fabian Gaessler, Karin Hoisl, Matt Marx and Felix Poege. We furthermore thank participants at the NBER I3 Technical Working Group Meeting in Fall 2024 in Cambridge, MA as well as EPIP 2025 in Antwerp for helpful feedback and suggestions. Finally, we also thank seminar participants at Polytechnic University of Milan, Manchester University (Alliance Manchester Business School) and University of Sussex for helpful comments. Michael E. Rose, Mainak Ghosh, Sebastian Erhardt, Cheng Li, Erik Buunk, Dietmar Harhoff Abstract We develop a language similarity model suitable for working with patents and scientific publications at the same time. In a horse race-style evaluation, we subject eight language (similarity) models to predict credible Patent-Paper Citations. We find that our Pat-SPECTER model performs best, which is the SPECTER2 model fine-tuned on patents. In two real-world scenarios (separating patent-paper-pairs and predicting patent-paper-pairs) we demonstrate the capabilities of the Pat-SPECTER. We finally test the hypothesis that US patents cite papers that are semantically less similar than in other large jurisdictions, which we posit is because of the duty of candor. The model is open for the academic community and practitioners alike. Keywords: language similarity model; semantic similarity; patents; publications; technology transfer; duty of candor 1 Introduction The translation of science to industrial applications is pivotal for modern innovation processes. Firms rely on knowledge generated in the public sector ( 1 ) , yet inventors face an enormous search and attention problem when engaging with academic literature ( 4 ) . At the same time, scholars and evaluators are interested in tracing the flow of knowledge from science to technology to understand innovation processes. The dominant approach for tracing knowledge flows is the analysis of explicit citations in scientific publications and patents. Such citations acknowledge intellectual priority and delineate the foundations upon which new inventions are built. This method, however, has significant limitations, such as the sparsity of citation graphs and the strategic or opportunistic selection of citations. To overcome both problems, we present a transformer-based similarity model that can be used for patent-publication comparisons. We identify four suitable transformer models and test them rigorously in a horse race. These models include SPECTER, developed by 5 for work with publications, and PaECTER, developed by 9 for work with patents, as well as derivations from these two models. We benchmark the models using standard test metrics on small datasets derived from Patent Paper citations by 15 ; 16 . The winning model, Pat-SPECTER, is the SPECTER model fine-tuned on patents. The open-source model generates embeddings that allow semantic comparisons of English publications and patents at the same time. To validate the Pat-SPECTER, we apply the model to two real-world scenarios, namely the separation of patent-paper pairs from random patent-to-paper pairings, and the prediction of patent paper pairs among millions of possible documents. We utilize an ElasticSearch database in the Logic Mill search system by 8 to identify (approximate) nearest neighbors among all of PATSTAT and OpenAlex. To show the model’s strength, we finally investigate a question at the core of innovation research: when does transfer from science to technology take place? More specifically, we investigate when citations to publications on patents are semantically similar to the patent. We find that in jurisdictions that require applicants to perform a ’duty of candor’, cited papers tend to be less related to the focal patent. The duty of candor includes an affirmative obligation to disclose all known material prior art, and violations can render a patent unenforceable. The duty of candor is a functional feature of the USPTO but also at the Israeli Patent Office (where it is called ’Affirmative Duty of Disclosure’). 17 have argued that the USPTO’s duty of candor induces applicants to cite more than is necessary – a hypothesis that 6 have tested and confirmed quantitatively by asking whether patent examiners actually use applicant-provided citations. Their answer is negative. Recently, 24 argue that the duty of candor leads applicants to cite a large amount of references, especially after a company sees its patents challenged in court. Though neither study distinguishes by type of cited document (patent or paper or else), the insights suggest that mandatory disclosure of prior art tends to produce irrelevant citations. By testing our model, we thus add a qualitative dimension to the existing quantitative analysis. The motivation for our research is a lack of tools for innovation economists. In the patent domain, citations to scientific literature are used to link patented inventions to their scientific underpinnings. As many patents do not have direct references to the non-patent literature, 2 propose the notion of “distance to the science frontier”. Patents citing scientific publications directly are taken to be at the science frontier. 20 show that highly cited scientific publications often serve as the foundation for particularly impactful patented inventions, highlighting the critical role of science in driving technological innovation. 20 linked around 950,000 patent families to over 2.2 million scientific articles, demonstrating that foundational scientific research significantly contributes to breakthrough inventions. Related studies such as 14 also use large-scale citation graphs to derive conclusions for innovation policy. However, the use of explicit citations has serious disadvantages, even if indirect links are taken into account. Citations are rare, and they may be selected strategically or opportunistically ( 10 ) . As an alternative to citations, scholars have investigated the possibility of using text as input data and comparing two documents to gauge their similarity. When vectors represent individual documents, vector operations (such as Cosine distance, Manhattan distance, Euclidean distance or others) yield similarity estimates. In his dissertation, 19 uses a term frequency–inverse document frequency (TF-IDF) model of textual similarity for technical texts. A more recent example is 11 , who identify breakthrough patents as patents with low textual similarity to the existing patent text corpus. They then establish that sectors with many breakthrough patents experience higher growth. This relationship could not have been demonstrated convincingly using patent citations, as the authors point out. Yet, turning massive amounts of text into data is challenging, and so far there exists no scalable solution that at the same time spans diverse text corpora. All applications intended to find similarities between patents or scientific publications suffer from one or several of the following shortcomings: they are specific to only one text corpus (i.e., only patents, such as patent maps); they do not account for semantic structure across different text corpora; they do not scale well. A common yet simple approach to transform text into a vector is the bag-of-word approach. Typically researchers used a weighted incarnation of this approach, the so-called term frequency–inverse document frequency (TF-IDF). TF-IDF vectorization, simple as it is, has two important limitations. First, it ignores the relative positioning of terms (to each other and within the document) and scales badly. For example, including new documents in the corpus may require the re-computation of the entire matrix. Thus, it becomes computationally expensive with the growing number of documents. Secondly, the TF-IDF matrix is sparse and high-dimensional, which

Read the complete text