GloCTM: Cross-Lingual Topic Modeling via a Global Context Space

Nguyen Tien Phat, Ngo Vu Minh, Linh Van Ngo, Nguyen Thi Ngoc Diep, Thien Huu Nguyen · 2026

Read GloCTM: Cross-Lingual Topic Modeling via a Global Context Space by Nguyen Tien Phat, Ngo Vu Minh, Linh Van Ngo, Nguyen Thi Ngoc Diep, Thien Huu Nguyen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-11872.txt.

Introduction

Nguyen Tien Phat, Ngo Vu Minh, Linh Van Ngo, Nguyen Thi Ngoc Diep, and Thien Huu Nguyen build topic models that work across tongues. A topic model finds themes in large text sets. For many tongues at once, themes should line up by sense, not just by tongue.

Most past models learn themes in split spaces, one per tongue. They then align themes with word lists, which can miss deep sense links. Their model, named GloCTM in the work, learns in one shared context space. A shared space could give cleaner theme matches for news and research sets.

Catalog record

CREATORNguyen Tien Phat, Ngo Vu Minh, Linh Van Ngo, Nguyen Thi Ngoc Diep, Thien Huu Nguyen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-11872

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

GloCTM: Cross-Lingual Topic Modeling via a Global Context Space Nguyen Tien Phat, Ngo Vu Minh, Linh Van Ngo, Nguyen Thi Ngoc Diep, Thien Huu Nguyen Abstract Cross-lingual topic modeling seeks to uncover coherent and semantically aligned topics across languages—a task central to multilingual understanding. Yet most existing models learn topics in disjoint, language-specific spaces and rely on alignment mechanisms (e.g., bilingual dictionaries) that often fail to capture deep cross-lingual semantics, resulting in loosely connected topic spaces. Moreover, these approaches often overlook the rich semantic signals embedded in multilingual pretrained representations, further limiting their ability to capture fine-grained alignment. We introduce GloCTM ( Glo bal Context Space for C ross-Lingual T opic M odel), a novel framework that enforces cross-lingual topic alignment through a unified semantic space spanning the entire model pipeline. GloCTM constructs enriched input representations by expanding bag-of-words with cross-lingual lexical neighborhoods, and infers topic proportions using both local and global encoders, with their latent representations aligned through internal regularization. At the output level, the global topic-word distribution, defined over the combined vocabulary, structurally synchronizes topic meanings across languages. To further ground topics in deep semantic space, GloCTM incorporates a Centered Kernel Alignment (CKA) loss that aligns the latent topic space with multilingual contextual embeddings. Experiments across multiple benchmarks demonstrate that GloCTM significantly improves topic coherence and cross-lingual alignment, outperforming strong baselines. Code — https://github.com/tienphat140205/GloCTM Introduction Uncovering latent thematic structures from large-scale text corpora is a foundational task in natural language processing, with topic modeling (TM) serving as a powerful technique for distilling meaning from data ( 2 ) . The challenge intensifies in a globalized world, where understanding shared narratives across different languages is paramount. Cross-Lingual Topic Modeling (CLTM) addresses this challenge, as illustrated in Figure 1 . The objective is to discover coherent topics that maintain their semantic identity across linguistic boundaries ( 18 ; 4 ; 22 ) . For instance, given documents sharing the same semantic content across different languages, an ideal CLTM infers similar document-topic proportions ( \theta ). This requires the underlying topics themselves, defined by their word distributions ( \beta ), to be robustly aligned, such that a single topic index corresponds to the same concept in each language. This capability thereby enables large-scale cross-cultural analysis and global knowledge discovery. Figure 1: Conceptual overview of Cross-Lingual Topic Modeling (CLTM) Table 1: Examples of misaligned topics generated by InfoCTM ( 29 ) across English (En) and Japanese (JP). The words grouped under each topic index show a clear semantic drift between the two languages. Early efforts in cross-lingual topic modeling (CLTM) often relied on parallel corpora to learn aligned topics across languages ( 18 ) . However, such data is scarce and expensive to obtain, particularly for low-resource language pairs. As a result, most subsequent approaches turn to bilingual dictionaries as a more practical alternative ( 14 ; 13 ; 26 ; 33 ; 30 ; 29 ) . While effective to some extent, this strategy introduces two key challenges. First, dictionary coverage is often limited ( 6 ) , leading to poor topic alignment ( 14 ; 12 ) . Second, according to the analysis in ( 29 ) , the direct alignment of topic-word distributions in models such as NMTM ( 30 ) and MCTA ( 26 ) can lead to a trivial solution. In this scenario, topic representations degenerate, causing the resulting topics to collapse and lose their interpretability. While recent models such as InfoCTM ( 29 ) have partially alleviated topic collapse and low dictionary coverage, their underlying architectural philosophy still inherits fundamental limitations from the broader CLTM landscape. A key concern is that these models continue to learn topic proportions ( \theta ) and topic-word distributions ( \beta ) in separate, disjoint spaces for each language, relying on an auxiliary loss function to indirectly bridge the gap between these otherwise disconnected language spaces. This architectural separation is the root cause of why topic alignment remains insufficiently robust. As illustrated in Figure 1 , this frequently leads to a ”semantic drift,” where a single topic index can correspond to a coherent theme in one language (e.g., ”video games” in English) but a completely unrelated one in another (e.g., ”footwear” in Japanese). This highlights the need for a framework that enforces alignment structurally, rather than handling it as an external or secondary component. Moreover, many models still depend on static dictionaries with limited coverage and shallow lexical cues, while overlooking the rich semantic signals offered by multilingual pretrained language models. Recent works ( 1 ; 19 ) utilize multilingual BERT for topic modeling over multilingual corpora, but their goal is not to learn aligned cross-lingual topics, and thus they fall outside the scope of cross-lingual topic modeling. Another line of work, the clustering-based refinement method by ( 5 ) , falls short, as it discovers latent groupings from multilingual embeddings but fails to model topic distributions, incorporate a generative structure, or enforce cross-lingual alignment. These trends highlight a critical gap: the absence of a unified framework that integrates multilingual embeddings with interpretable, aligned, and generative topic modeling. To overcome these limitations, we introduce GloCTM , a dual-pathway neural topic model that brings topic alignment to the core of its design. Rather than relying on auxiliary losses to bridge disjoint language spaces, GloCTM constructs a shared semantic space from the outset. It begins by enriching each document with semantically related words from both languages (identified via multilingual pretrained embeddings) to form a dynamic global bag-of-words representation. This enriched input is passed through a global encoder, which infers a unified topic proportion vector ( \theta_{\text{global}} ), later decoded via a global topic-word matrix ( \beta_{\text{global}} ) that is constructed from the topic-word matrices of the individual language pathways to structurally enforce cross-lingual topic sharing. At the same time, language-specific local encoders capture monolingual subtleties, while a KL divergence loss encourages their outputs to stay aligned with the global topic view. This joint architecture allows GloCTM to balance language-specific expressiveness with semantic coherence across languages—offering a principled, end-to-end solution to cross-lingual topic modeling. In light of the rapid advances in multilingual pretrained language models, incorporating their semantic knowledge into cross-lingual topic modeling has become not only natural but essential. These embeddings provide a rich, language-agnostic semantic space that can inform topic inference across languages. To harness this, we introduce a final calibration mechanism based on Centered Kernel Alignment (CKA) ( 8 ; 16 ) . Specifically, we apply a CKA loss to align the model’s latent topic proportion vectors with the deep semantic representations produced by multilingual pretrained language models (PLMs). This regularization enhances the semantic structure of the topic proportions ( \theta ), enabling them to reflect cross-lingual meaning beyond lexical overlap. As a result, GloCTM produces not only structurally aligned topics, but also semantically coherent ones as a natural consequence of improved latent representations. Our main contributions are summarized as follows: • We introduce GloCTM , a novel dual-pathway framework for cross-lingual topic modeling that combines a global and local VAE architecture, while leveraging multilingual pretrained embeddings to guide topic learning across languages. • We propose a dynamic global bag-of-words construction that enriches each document with semantically related words from both

Read the complete text