Bidirectional Topic Matching: Quantifying Thematic Overlap Between Corpora Through Topic Modelling

Raven Adam, Marie Lisa Kogler · 2024

Read Bidirectional Topic Matching: Quantifying Thematic Overlap Between Corpora Through Topic Modelling by Raven Adam, Marie Lisa Kogler (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-18376.txt.

Introduction

Raven Adam and Marie Lisa Kogler introduce a way to compare the themes of two text collections, in this 2024 paper. Their method, Bidirectional Topic Matching, trains a topic model on each corpus and then runs each model on the other corpus. That two-way pass reveals both the themes the collections share and the ones each holds alone.

The framework accepts several topic toolkits, from classic allocation models to modern embedding-based ones. Checks against similarity-based methods showed strong agreement. A case study on climate news separated shared ground from split angles between climate change and climate action coverage.

Catalog record

CREATORRaven Adam, Marie Lisa Kogler
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-18376

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Bidirectional Topic Matching: Quantifying Thematic Overlap Between Corpora Through Topic Modelling Raven Adam, Marie Lisa Kogler Abstract This study introduces Bidirectional Topic Matching (BTM), a novel method for cross-corpus topic modeling that quantifies thematic overlap and divergence between corpora. BTM is a flexible framework that can incorporate various topic modeling approaches, including BERTopic, Top2Vec, and Latent Dirichlet Allocation (LDA). BTM employs a dual-model approach, training separate topic models for each corpus and applying them reciprocally to enable comprehensive cross-corpus comparisons. This methodology facilitates the identification of shared themes and unique topics, providing nuanced insights into thematic relationships. Validation against cosine similarity-based methods demonstrates the robustness of BTM, with strong agreement metrics and distinct advantages in handling outlier topics. A case study on climate news articles showcases BTM’s utility, revealing significant thematic overlaps and distinctions between corpora focused on climate change and climate action. BTM’s flexibility and precision make it a valuable tool for diverse applications, from political discourse analysis to interdisciplinary studies. By integrating shared and unique topic analyses, BTM offers a comprehensive framework for exploring thematic relationships, with potential extensions to multilingual and dynamic datasets. This work highlights BTM’s methodological contributions and its capacity to advance discourse analysis across various domains. Keywords Topic Detection \cdot Corpus Analysis \cdot Topic Matching 1 Introduction Topic modeling has emerged as a widely used method for analyzing and organizing large textual corpora ( Churchill and Singh 2022 ) . One of its most common applications is uncovering latent topics within a given corpus, which human experts can then evaluate to derive quantitative insights ( Grundmann 2021 ) . However, topic modeling is not limited to analyzing single corpora. It can also be applied to compare similarities and differences across multiple corpora ( Bystrov et al. 2022 ) . In the field of corpus linguistics, cross-corpus similarity methods are commonly employed, though primarily during corpus creation or selection. Nevertheless, research has demonstrated that these methods can also provide valuable insights for discourse analysis ( Taylor 2018 ) . Recent studies have illustrated such applications, including comparing medical records from different hospitals ( Shaikina and Funkner 2020 ) , examining the economic evolution of Poland and Germany ( Bystrov et al. 2024 ) , analyzing discussions around German federal elections on Twitter ( Hellwig et al. 2024 ) and exploring migration discourse in The Times and British parliamentary debates ( Taylor 2018 ) . To advance the use of topic modeling in cross-corpus comparison, this study introduces a novel method called Bidirectional Topic Matching (BTM). BTM allows to identify shared topics across multiple corpora and understand their relationships while also highlighting corpus-specific topics. This approach facilitates both quantitative cross-corpus comparisons and the identification of potential areas for in-depth qualitative analysis. Existing methods for cross-corpus topic modeling typically rely on topic embeddings generated by Latent Dirichlet Allocation (LDA) ( Blei et al. 2003 ) or language embedding models to compute cosine similarities between topic representations ( Carniel et al. 2022 ; Hellwig et al. 2024 ) . Alternatively, some studies generate a single topic model by combining corpora and then analyze differences in topic distributions across each corpus ( Wang et al. 2023 ) . In contrast, BTM employs a distinct topic model for each corpus and applies these models reciprocally across all corpora. This allows each topic model to assign topics to all documents, enabling the analysis of topic co-occurrences to identify shared and unique topics among the corpora. 2 Method 2.1 Topic modelling BTM is a flexible framework for cross-corpus analysis that can incorporate various topic modeling approaches. For assessing corpus similarity, any method capable of inferring topics for new data is suitable. However, analyzing unique or corpus-specific topics requires a method that can identify intraclass outliers—documents that do not align with any topics generated by the chosen topic modeling approach. Language embedding-based methods, such as BERTopic ( Grootendorst 2022 ) or Top2Vec ( Angelov 2020 ) , are particularly well-suited for this purpose as they inherently support outlier detection. Traditional approaches like Latent Dirichlet Allocation (LDA), which assign a topic to every document, can also be adapted through post-processing techniques such as HDBSCAN ( McInnes and Healy 2017 ) or Local Outlier Factor ( Breuniq et al. 2000 ) to identify outliers. Given BERTopic’s state-of-the-art performance and its built-in outlier detection capabilities, this study demonstrates the application and efficacy of BTM using BERTopic as the underlying topic modeling approach. BERTopic presents an innovative method for topic modeling, capitalizing on recent advancements in embedding models. Derived from Bidirectional Encoder Representations from Transformers (BERT) ( Devlin et al. 2019 ) , this approach involves the representation of documents as points within a high-dimensional vector space. In this space, each coordinate represents contextual information corresponding to the respective document. As a result, semantically analogous documents will be in proximity to each other. Subsequently, dimensionality reduction and clustering algorithms are employed to identify compact clusters of documents with shared thematic content. Each of these clusters can then be interpreted as individual topics that are found within the investigated collection of documents and are represented by a set of keywords that are most indicative of the underlying theme. An outlier refers to a document that cannot be assigned to any of the identified topics due to its lack of thematic similarity. This occurs when the document does not align well with any of the topics, often because it is too different or semantically distant from the other documents in the model. In BERTopic, both topics and outliers can be easily accessed and handled, where outliers are grouped together under an outlier topic, often with a special identifier like -1. As a final step, a class-based term frequency inverse document frequency measure (c-TFIDF) is applied to extract the most salient terms from each topic and create interpretable topic representations ( Grootendorst 2022 ) . 2.2 Cross-Corpus Topic Assignment For BTM, which is schematically depictured in Figure 1 , two independent topic models are trained on two thematically related corpora, corpus 1 and corpus 2. Each model is used to identify the main themes within the respective corpus, generating topics T1 for corpus 1 and topics T2 for corpus 2. Individually, these native topic models provide a comprehensive understanding of the thematic structures specific to each dataset. To explore thematic alignment between the corpora, each model was applied to the corpus, it was not trained on. For this, the semantic similarity between the document’s embedding and the topic embeddings of the model trained on the other corpus was calculated. Specifically, each document in corpus 2 gets matched to a topic from T1, and each document in corpus 1 gets matched to a topic from T2, based on the highest similarity score. This process produced cross-corpus topic assignments, resulting in T12 (topics from T1 assigned to Corpus 2) and T21 (topics from T2 assigned to Corpus 1). Subsequently, topic pairs are generated by assigning each document from one corpus to the most similar topic from the opposite corpus. Specifically, for each document, the topics assigned by the corpus 1 model (T11 and T12) and the topics assigned by the corpus 2 model (T22 and T21) are combined into cross-corpus topic pairs.

Read the complete text