Cross-lingual Comparison of Research Funding Projects with Multilingual Sentence-BERT : Evidence from KAKENHI, NIH, NSF, and UKRI

Miki KIMURA-IDA · 2026

Read Cross-lingual Comparison of Research Funding Projects with Multilingual Sentence-BERT : Evidence from KAKENHI, NIH, NSF, and UKRI by Miki KIMURA-IDA (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-27315.txt.

Introduction

Kimura-Ida asks whether multi-language sentence vectors support meaningful cross-language comparison of research funding projects. Such comparison matters more for science policy and planning. Language differences block it. KAKENHI descriptions are mainly Japanese, while NSF, NIH, and UKRI projects are documented in English. For each KAKENHI project she builds two versions. One is the original Japanese text. The other is its machine-made English version. Both are embedded in a shared meaning space with multilingual Sentence-BERT, and she tests how translation shifts meaning.

Catalog record

CREATORMiki KIMURA-IDA
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE29271
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.27315

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Cross-lingual Comparison of Research Funding Projects with Multilingual Sentence-BERT : Evidence from KAKENHI, NIH, NSF, and UKRI Miki KIMURA-IDA Abstract Cross-national comparison of research funding projects is increasingly important for science policy and strategic planning, but language differences remain a major obstacle. In particular, KAKENHI project descriptions are written primarily in Japanese, whereas projects from major overseas funding agencies, such as NSF, NIH, and UKRI, are documented in English. This study investigates whether multilingual sentence embeddings can support meaningful cross-lingual comparison of research funding projects, with particular attention to the semantic effects of translating Japanese texts into English. For each KAKENHI project, we construct two representations: the original Japanese text and its machine-translated English version, both embedded in a shared semantic space using a multilingual Sentence-BERT model. We then compare their distances and nearest-neighbor relationships with respect to projects from English-language funding agencies. The results show that the Japanese and translated English representations of the same KAKENHI project are, on average, located closer to one another than to native English projects, indicating substantial cross-lingual alignment. However, the overlap of nearest neighbors between the two representations is limited, averaging 2.9 2.9 out of 10 10 . This suggests that multilingual embeddings capture semantic similarity across languages to a meaningful extent, while language differences and translation still affect the local structure of the embedding space. These findings suggest that multilingual embeddings provide a useful basis for large-scale exploratory comparison of funding projects across countries and agencies. At the same time, they offer an empirical reference for assessing semantic drift when Japanese research project data are translated into English for international analysis. K eywords multilingual embeddings \cdot cross-lingual comparison \cdot research funding \cdot semantic drift 1 Introduction Comparative analysis of research funding systems plays a crucial role in understanding global research trends, identifying emerging scientific fields, and informing science and technology policy. In recent years, increasing attention has been paid to cross-national comparisons of funding projects across major research systems. However, a fundamental challenge arises from language differences in project descriptions. In Japan, KAKENHI (Grants-in-Aid for Scientific Research), administered mainly by the Japan Society for the Promotion of Science (JSPS), is a major national competitive funding program for academic research. Research projects funded under KAKENHI are primarily documented in Japanese, while projects funded by English-language funding agencies, such as the National Science Foundation (NSF), the National Institutes of Health (NIH) in the United States, and UK Research and Innovation (UKRI) in the United Kingdom, are described in English. This linguistic discrepancy complicates the application of natural language processing (NLP) techniques for direct comparison. A straightforward solution is to translate Japanese texts into English using machine translation. While effective in some contexts, this approach introduces several issues. Machine translation may alter the original meaning, particularly in specialized scientific domains, and may introduce systematic biases or artifacts that affect downstream analysis. Recent advances in multilingual representation learning offer an alternative. Models such as Sentence-BERT (sBERT) [ Reimers19 ] have been extended to multilingual settings, enabling texts in different languages to be embedded into a shared vector space. In principle, semantically similar texts—regardless of language—should be located near each other in this space. This study explores the feasibility and limitations of such an approach in the context of research funding data. Specifically, we address the following research questions: RQ1: To what extent can multilingual embeddings enable cross-lingual comparison of research projects without machine translation? RQ2: How consistent are local neighborhood structures across languages? 2 Related Work Multilingual sentence embeddings have been widely studied in recent years, particularly in the contexts of cross-lingual information retrieval, semantic similarity, and machine translation evaluation. Models based on multilingual transformers aim to align textual representations across languages through shared training objectives, making it possible to compare semantically related texts within a common vector space. Sentence-BERT further advances this line of work by optimizing sentence embeddings for semantic similarity tasks, and multilingual variants of Sentence-BERT have shown strong performance in cross-lingual retrieval and matching tasks. Research on cross-lingual retrieval has also demonstrated the practical importance of directly comparing Japanese and English texts. For example, [ Asai21 ] presents a multilingual retrieval setting that explicitly addresses cross-lingual information access between languages such as Japanese and English, highlighting the importance of robust representation learning for retrieval across language boundaries. This line of work suggests that multilingual embedding models can provide a useful basis for comparing semantically related texts even when they are written in different languages. Recent studies have further improved the quality of cross-lingual sentence representations and their applicability to multilingual comparison tasks. In this context, [ Hoshino24 ] provides an important recent contribution by demonstrating the effectiveness of cross-lingual semantic representation methods for multilingual text analysis. Such studies support the view that multilingual sentence embeddings offer a promising foundation for analyzing semantic relationships across languages in a consistent and scalable manner. In parallel, prior studies have explored the use of distributed representations for analyzing scientific texts, including research papers, abstracts, and project descriptions. Earlier approaches often relied on word embeddings such as FastText [ Joulin16 ] 1 1 1 https://fasttext.cc/ and constructed document-level representations by aggregating word vectors, whereas more recent methods directly encode sentence- or document-level semantics using transformer-based architectures. These developments have improved the ability to capture semantic similarity beyond surface-level lexical overlap. However, relatively few studies have systematically examined cross-lingual comparisons of research funding data at scale, particularly in the context of science and innovation policy analysis. While multilingual embedding methods have been actively studied in information retrieval and general semantic similarity tasks, their application to large-scale funding databases remains limited. This study addresses that gap by conducting an empirical analysis of research project data from KAKENHI, NIH, NSF, and UKRI using multilingual sentence embeddings and nearest-neighbor analysis. 3 Materials and Methods This study investigates the cross-lingual comparability of research project texts by representing each project description as a dense vector in a shared semantic space and then examining the neighborhood structure of those vectors. The target corpora consist of research project records collected from KAKENHI, NIH, NSF, and UKRI for the period from about FY2017 to FY2025. For each project, the textual information used for analysis consists of the project title and abstract. Across all funding agencies, multiple funding categories, programs, and schemes are included together in a unified dataset and are not analyzed separately by scheme. Thus, the analysis focuses on semantic relationships across the full set of projects rather than on differences among funding categories or program types. The final dataset contains 264,881 KAKENHI projects, 137,102 NIH projects, 107,250 NSF projects, and 83,336 UKRI projects. Table 1: Number of projects included in the dataset Funding Counts KAKENHI 264,881 UKRI 83,336 Funding Counts NIH 137,102 NSF 107,250 To evaluate the effect of language and translation, two coordinate types are prepared for KAKENHI: the original Japanese text and its machine-translated English version. For the main analysis, the MT sample was restricted to projects for which both the title and the abstract had been translated by the LLM. If the source text was already written in English, it was retained without modification in the author-written English baseline. Translation of Japanese texts was performed primarily using AWS Bedrock with Claude

Read the complete text