A Shared IPTC Topic Space for Comparable Topic Modelling of Social and Mainstream Media
Din Iskakov, Sebastian Gonçalves, Marco Idiat, Mendeli Vainstein, Aline Villavicencio, Ronaldo Menezes ⋆, Rodrigo Wilkens · 2026
Read A Shared IPTC Topic Space for Comparable Topic Modelling of Social and Mainstream Media by Din Iskakov, Sebastian Gonçalves, Marco Idiat, Mendeli Vainstein, Aline Villavicencio, Ronaldo Menezes ⋆, Rodrigo Wilkens (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26845.txt.
Introduction
Din Iskakov and six colleagues build a shared topic space so news and social media can be compared fairly. Topic models fitted to separate collections invent separate topic systems that cannot be lined up. Their framework instead anchors everything to one taxonomy, the IPTC media topics used by news organisations. Discovered topics are scored against the taxonomy and rolled up to its parent categories.
Development used a controlled newspaper corpus from one year. A guided topic model was screened, refined, and calibrated step by step. Guided models beat a zero-shot rival on mapped coverage under strict thresholds. Enriching the taxonomy targets with parent information improved both coverage and consistency. Coverage faded gently rather than collapsing as thresholds tightened. The method offers a reproducible route to like-for-like topic comparison across outlets.
Catalog record
| CREATOR | Din Iskakov, Sebastian Gonçalves, Marco Idiat, Mendeli Vainstein, Aline Villavicencio, Ronaldo Menezes ⋆, Rodrigo Wilkens |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| LICENSE | http://creativecommons.org/licenses/by-sa/4.0/ |
| RIGHTS | CC BY SA |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.26845
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
A Shared IPTC Topic Space for Comparable Topic Modelling of Social and Mainstream Media Din Iskakov, Sebastian Gonçalves, Marco Idiat, Mendeli Vainstein, Aline Villavicencio, Ronaldo Menezes ⋆, Rodrigo Wilkens Abstract Comparing topic attention across different media is hindered by a fundamental modelling problem: topic models fitted separately to each corpus produce corpus-specific topic spaces that cannot be aligned directly. This paper presents a reproducible framework that places corpora in a single shared topic space defined by a taxonomy. Discovered topics are obtained with guided BERTopic, scored against the ninety-four IPTC Media Topics’ taxonomy topics (level-1) through weighted keyword and target centroids, and then collapsed upward to seventeen IPTC parent topics by a maximum-similarity rule. The framework was developed and selected on a controlled New York Times 2011 corpus through a narrowing sequence: a broad model screen, a focused mapping refinement, a strict finalist comparison, a target-construction ablation, and a threshold calibration. In this corpus, the guided family retained substantially stronger mapped coverage than a zero-shot benchmark under stricter assignment thresholds, a parent-enriched target construction improved both coverage and parent consistency, and coverage declined gradually rather than collapsing as the assignment threshold was tightened. The contribution is an externally anchored method for constructing a shared topic space that enables reproducible cross-source topic comparison. $\star$ $\star$ footnotetext: Corresponding authors. 1 Introduction Social media platforms such as Twitter and mainstream news organisations represent structurally different information flows. One is decentralised and bottom-up, the other editorially structured and top-down, and although they often address the same subjects, they emphasise them differently. The digital traces that people produce online can be read as a sensor of public attention [ 5 ] , and comparing that sensor against institutional reporting has become more pressing as agenda-setting and misinformation dynamics now play out across both kinds of platform at once [ 13 , 12 ] . A natural way to make such a comparison is to model the topics present in each corpus and then compare the resulting topic distributions. This turns out to be harder than it first appears, and the difficulty is methodological rather than conceptual. The obstacle is comparability. Topic models assign their own corpus-specific identifiers to the structure they discover, so a topic recovered from tweets and a topic recovered from news articles are not expressed in the same vocabulary of labels. Without a common reference layer, the two topic outputs cannot be placed side by side, and any comparison built on top of them inherits that incompatibility. The closest prior work, by Zhao et al. [ 14 ] , applied Latent Dirichlet Allocation independently to Twitter and The New York Times and compared the resulting distributions, finding partial overlap in broad themes but substantial differences in attention. That study established the value of cross-media topic comparison, but it rests on two properties that limit a controlled, repeatable comparison. It depends on a model family that tends to produce diffuse, overlapping topics on short, noisy social-media text, and on unsupervised topic discovery that yields a different topic space for each corpus, so the comparison space itself shifts with the data. The first of these properties reflects a wider issue in topic modelling for short text. Classical bag-of-words models such as Latent Dirichlet Allocation [ 2 ] and Non-Negative Matrix Factorisation [ 9 ] rely on repeated word co-occurrence and tend to produce diffuse, overlapping topics on tweets [ 8 ] . Embedding-based alternatives address this by representing documents in dense semantic spaces. Top2Vec [ 1 ] clusters joint document-word embeddings, whereas BERTopic [ 6 ] combines sentence-transformer embeddings with dimensionality reduction and density-based clustering, and uses a class-based TF-IDF weighting to produce interpretable topic representations. Empirical comparisons on Twitter data report that BERTopic yields more coherent topics and fewer noisy clusters than LDA, NMF, or Top2Vec [ 4 ] . BERTopic also supports a guided variant, in which seed words shape topic boundaries, and a supervised variant for assignment to predefined classes [ 7 ] ; together these make consistent, like-for-like topic assignment across two corpora feasible. This paper targets the second limitation, the corpus-specific topic space. Rather than letting each corpus define its own topics, it fixes a single shared topic space in advance, defined by the IPTC Media Topics taxonomy 1 1 1 https://iptc.org/std/NewsCodes/guidelines/ , the international standard used by news organisations to classify reporting, so the comparison space no longer shifts with the data. Both corpora are mapped into that taxonomy, anchoring the comparison in an externally validated reporting scheme rather than in labels introduced per corpus. Anchoring it in this way makes the framework reproducible and the topic labels directly interpretable. The topic model is guided BERTopic, which preserves unsupervised topic discovery while allowing topic boundaries to be shaped by the taxonomy, and the mapping is hierarchical: discovered topics are first scored against the fine-grained level-1 topics and then collapsed upward to the seventeen IPTC parent topics that serve as the reporting layer. The framework is developed and selected on a controlled New York Times 2011 corpus. This paper makes the following contributions. First, it defines a shared topic space that maps heterogeneous corpora into the IPTC Media Topics taxonomy, so that social-media and news topic outputs become directly comparable. Second, it introduces a parent-enriched target construction together with a maximum-similarity collapse rule that moves from fine-grained level-1 scores to the seventeen reporting-level parent topics under conservative score and gap thresholds. Third, it sets out a development-corpus selection protocol, comprising a broad model screen, a strict finalist comparison, a target-construction ablation, and a threshold calibration, in which the guided family is preferred over a zero-shot baseline and the parent-enriched targets over simpler ones. Two questions organise the work: (1) which model family and mapping design produce adequate and defensible coverage of the shared topic space; and (2) whether the selected configuration depends on a single brittle threshold or remains usable across a range of assignment rules. 2 Methods The New York Times 2011 corpus serves as the development corpus on which the shared topic-space pipeline was designed and selected. News articles are long text pieces, editorially produced, and comparatively coherent, whereas tweets are shorter, noisier, and far more numerous. To pass both through one pipeline, the modelling inputs were reduced to a common text field for embedding and topic assignment, while publication timestamps were retained for later aggregation. The unit of analysis changes across the pipeline. At the document level, each article or tweet receives a mapped IPTC parent label or remains an outlier. At the daily level, topic shares form time series. At the monthly level, mapped document counts become seventeen-topic share vectors. The method is therefore built around a hierarchy of representations rather than a single fixed one, with source harmonisation as the central design requirement. The comparison space is defined by the IPTC Media Topics hierarchy 2 2 2 https://iptc.org/std/NewsCodes/guidelines/ , and results are reported at the level of the seventeen IPTC parent topics. This level is broad enough to support stable cross-source comparison while remaining interpretable as a reporting taxonomy. Assignment is not made directly to those seventeen labels. Instead, discovered topics are first scored against the ninety-four IPTC level-1 topics and then collapsed upward