Generalizability of Media Frames: Corpus creation and analysis across countries
Agnese Daffara, Sourabh Dattawad, Sebastian Padó, Tanise Ceron · 2025
Read Generalizability of Media Frames: Corpus creation and analysis across countries by Agnese Daffara, Sourabh Dattawad, Sebastian Padó, Tanise Ceron (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-16337.txt.
Introduction
Agnese Daffara, Sourabh Dattawad, Sebastian Padó, and Tanise Ceron test whether a United States media frame scheme travels to Brazil. Frames are the angles news stories stress in a debate. The scheme in wide use was built on a few salient American issues.
The team built a corpus of Brazilian Portuguese news on politics and economics, tagged with the same scheme. After several tagging rounds, most of the fifteen frames still fit with minor guide tweaks. Some frames were rarely used, and new issues fell back on broad catch-all frames. Trained and zero-shot models were also tested on the new data. The authors urge care when frame schemes cross cultures.
Catalog record
| CREATOR | Agnese Daffara, Sourabh Dattawad, Sebastian Padó, Tanise Ceron |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-16337
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Generalizability of Media Frames: Corpus creation and analysis across countries Agnese Daffara, Sourabh Dattawad, Sebastian Padó, Tanise Ceron Abstract Frames capture aspects of an issue that are emphasized in a debate by interlocutors and can help us understand how political language conveys different perspectives and ultimately shapes people’s opinions. The Media Frame Corpus (MFC) is the most commonly used framework with categories and detailed guidelines for operationalizing frames. It is, however, focused on a few salient U.S. news issues, making it unclear how well these frames can capture news issues in other cultural contexts. To explore this, we introduce FrameNews-PT , a dataset of Brazilian Portuguese news articles covering political and economic news and annotate it within the MFC framework. Through several annotation rounds, we evaluate the extent to which MFC frames generalize to the Brazilian debate issues. We further evaluate how fine-tuned and zero-shot models perform on out-of-domain data. Results show that the 15 MFC frames remain broadly applicable with minor revisions of the guidelines. However, some MFC frames are rarely used, and novel news issues are analyzed using general ’fallback’ frames. We conclude that cross-cultural frame use requires careful consideration. 1 Introduction Frames are interpretation schemes used to organize reality by isolating and highlighting salient aspects of it ( Otmakhova et al. 2024 ) . They combine three levels: cognitive, (concerning the mental representations of the world), semantic (concerning the linguistic structures involved); and communicative (their usage and impact on the audience). Frames are finding increasing application in NLP as a framework to analyze social media and news coverage. In particular the communicative dimension of frames can be seen as a bridge between belief and action ( Snow and Benford 2005 ) . Following Entman 1993 , frames have four objectives: i) promote a particular definition of a problem; ii) promote an interpretation of it; iii) propose a moral evaluation; and iv) recommend a treatment or a solution. This makes them a valuable tool for identifying, measuring, and quantifying particular worldviews and communication strategies ( Card et al. 2015 ; Mendelsohn et al. 2021 ) . Frames are primarily a methodological framework, and not a fixed set of categories. However, a widely used set is based on the Media Frame Corpus (MFC), 15 frames (see Table 2 ) with annotation guidelines ( Card et al. 2015 ) , developed to cover a limited number of issues concerning the U.S. context, e.g., gun control and climate change. As in other NLP tasks, however, generalization is an issue: coarse-grained frames risk failing to capture important aspects of a debate, notably ideological bias, while fine-grained frames may not generalize across debates. For example, Mendelsohn et al. 2021 demonstrate that in news about migration, the generic frame cultural identity shows only a slight tendency towards liberal texts. Introducing two related sub-frames hero: cultural diversity and threat: national cohesion leads to a clear separation by political ideology – liberal in the former and conservative the latter case – but these subframes are presumably irrelevant for other debates. Similar questions arise in cross-lingual contexts: The MFC frames were also applied by SemEval 2023 Task 3 to 9 languages Piskorski et al. 2023b , but in an arguably pragmatic setting where the applicability of the frames was not assessed, nor was inter-annotator agreement reported. Indeed, full cross-linguistic generalization of frames is far from clear, given previous work on the cross-lingual generalization of semantic inventories ( Peters et al. 1998 ; Reddy et al. 2017 ; Gilardi and Baker 2018 ) . For full generalization, frames would show high agreement – their definitions and annotation guidelines would allow each markable in a new language to be assigned to a unique frame – and completeness – for each instance in the new language, there would be a suitable frame. However, we do not have empirical evidence so far. Thus, we set out to answer these questions: 1. How high are agreement and completeness for MFC frames when annotating a representative sample of news articles despite having been originally developed on a small number of issues? 2. How high are agreement and completeness of the MFC frames when applied to news reporting from other countries? 3. Even if the frames generalize well, is it possible to transfer computational models of frame prediction between languages? Our study proceeds by creating FrameNews-PT , a corpus of 300 news articles of ’hard news’ ( Reinemann et al. 2012 ) in Brazilian Portuguese. 1 1 1 The corpus will be made available for the community upon acceptance. FrameNews-PT is based on the News Portal Recommendation dataset ( Lucas et al. 2023 ) . It contains Brazilian news about, e.g., new regulations, taxes, and education. We manually annotate the articles with MFC frames using a perspectivist approach ( Cabitza et al. 2023 ) . We carry out several rounds of annotation to a) adapt the MFC guidelines while maintaining their level of detail, b) value the individual perspectives of annotators, and c) examine generalizability and overlap of frames (RQ1+2). After corpus creation, we conduct modeling experiments (RQ3). We use two transformer-based models fine-tuned on the MFC and 9 chat-instructed models in a zero-shot setting, and analyze how well models perform out of domain on FrameNews-PT in comparison with the MFC. Our results are generally promising: (Most) MFC frames are annotated with high agreement in Brazilian hard news, with little confusion among them. However, some frames appear rarely, being bound to U.S.-specific issues. Also, completeness is not perfect: Brazilian issues outside the scope of the original MFC are often annotated with relatively generic frames as a fallback (e.g., Economic ), leading to some loss of information. On the modeling side, we find that zero-shot models perform better at frame prediction on the Brazilian data than transferred English classifiers, indicating that the data distributions are sufficiently dissimilar. Overall, use of MFC frames in new languages and issues calls for a reflected process. 2 Related Work Previous work in NLP has investigated how framing political language influences society and public opinion ( Lecheler et al. 2015 ) , and how it shapes the spread of information ( Gilardi et al. 2021 ) across communication contexts, such as agenda-setting campaigns ( Tsur et al. 2015 ; Field et al. 2018 ) and news media ( Card et al. 2015 ) . Frames can help analyze argumentation, and frame detection can be seen as argument assessment ( Lauscher et al. 2022 ) . Given their connection to political ideologies ( Alashri et al. 2015 ) , frames are also useful for analyzing and diversifying perspectives, e.g. in news recommendation ( Mulder et al. 2021 ) . The task of frame detection has often been approached through topic modeling with unsupervised methods ( Nguyen et al. 2013 ; Roberts et al. 2014 ; Tsur et al. 2015 ; Ajjour et al. 2019 ) , which, however, tend to be highly corpus-specific. Other studies employ supervised classification techniques, including logistic regression ( Card et al. 2016 ) , neural networks ( Naderi and Hirst 2017 ; Liu et al. 2019 ) , and fine-tuning of pre-trained models ( Kwak et al. 2020 ) , all of which require robust gold