Multi-Modal Framing Analysis of News

Arnav Arora Srishti Yadav · 2025

Read Multi-Modal Framing Analysis of News by Arnav Arora Srishti Yadav (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-20960.txt.

Introduction

Arnav Arora, Srishti Yadav, and colleagues presented a method in 2025 for studying how news stories frame issues. Framing means selecting which parts of a topic to stress, which shapes how readers judge it. Past automated studies used fixed frame lists and read only the words, ignoring the photos editors choose to run beside the text.

Their approach uses large vision and language models to label frames in both text and images at scale, with many labels allowed per story. Comparing the two channels can reveal where a photo pushes a different angle than the article. Tests on partisan coverage show the method can flag strongly slanted framing and give a fuller picture of bias.

Catalog record

CREATORArnav Arora Srishti Yadav
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE88 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-20960

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Multi-Modal Framing Analysis of News Arnav Arora Srishti Yadav Abstract Automated frame analysis of political communication is a popular task in computational social science that is used to study how authors select aspects of a topic to frame its reception. So far, such studies have been narrow, in that they use a fixed set of pre-defined frames and focus only on the text, ignoring the visual contexts in which those texts appear. Especially for framing in the news, this leaves out valuable information about editorial choices, which include not just the written article but also accompanying photographs. To overcome such limitations, we present a method for conducting multi-modal, multi-label framing analysis at scale using large (vision-) language models. Grounding our work in framing theory, we extract latent meaning embedded in images used to convey a certain point and contrast that to the text by comparing the respective frames used. We also identify highly partisan framing of topics with issue-specific frame analysis found in prior qualitative work. We demonstrate a method for doing scalable integrative framing analysis of both text and image in news, providing a more complete picture for understanding media bias. 1 Introduction Frames are conceptual tools that both communicators and audiences use to interpret and categorize issues and events Gitlin 1980 ; Eko 1999 ; Pan and Kosicki 1993 ; Reese et al. 2001 . By highlighting specific elements of a topic and minimizing others, communicators frame messages in ways they believe will resonate with audiences Goffman 1974 and can shape the way the topic is perceived by readers or viewers ( Schudson 2003 ) . In the field of journalism, framing is a core narrative device by which news consumption is framed within an interpretive perspective ( Card et al. 2015 ) . Most prior work on computational frame analysis has focused on linguistic structure and content analyses via text elements ( Ali and Hassan 2022 ) . However, framing is not solely textual; visual elements also play a crucial role in conveying implicit and explicit messages ( Messaris and Abraham 2001 ) . Figure 1: News can be intentionally framed to affect reader perception. Editorial choices decide what is communicated through words and images. Our approach systematically detects this framing. News media often employ images alongside text to reinforce or contrast the intended frame, leveraging the affective and cognitive impact of visuals Cope et al. 2005 ; Keib et al. 2018 ; Grabe and Bucy 2009 ; Geise et al. 2025 . An example can be seen in Figure 1 . The image depicts protesters with signs with police presence around, using the public opinion and security framing. The article text, on the other hand, talks about the quality of life of migrants, economic implications, crime, and policy framing. Such differences in portrayal can affect readers’ perception of the crisis and their induced emotions. Further, while communication in text is more explicit through linguistic framing, images encode frames in more subtle and implicit ways, requiring sophisticated interpretation models to capture their meaning Aiello and Parry 2019 . As such, when conducting media analysis, the communicated framing across both the image and the text should be considered. While visual framing analysis has been explored in communication science and journalism studies ( Wessler et al. 2016 ; Powell et al. 2017 ) , existing studies doing computational framing analysis have largely ignored this crucial aspect ( Ali and Hassan 2022 ) . Further, they have primarily focused on a fixed set of labels for framing analysis, performing prediction in a multi-class setting. This substantially limits the information one can derive from predictions as an article can convey several frames ( Figure 1 ) and fine-grained analysis of framing within a topic necessitates frames specific to a particular issue. Large vision and language models are particularly well suited for conducting this task at scale, considering the semantic understanding embedded into them through large scale pre-training. This led us to our overall research questions: (RQ1) Can LLMs and VLMs reliably detect framing in news articles? (RQ2) Are there differences in framing conveyed through text vs the images? and (RQ3) How do these framings vary across topics and publishers? Our contributions include the following. • We present the first computational study of multimodal framing in the news, outlining a methodology using LLMs and VLMs. • We provide a large-scale dataset of 500k U.S.-based news articles for framing analysis, automatically labeled with generic and issue-specific frames, validated through human annotations. 1 1 1 https://huggingface.co/datasets/copenlu/mm-framing • We conduct a thorough analysis of frames communicated in the image vs the text of articles, both at a corpus-level and at a fine-grained level for articles about immigration. • We discover meaningful differences in how frames are used in images and texts, across political leanings and topics. 2 News Framing Framing of news articles has been studied widely in communication studies. There are several definitions of framing and scholars often disagree on the method for extraction and its role in public communication, leading to it being termed a “fractured paradigm” ( Entman 1993 ) . The definition from Entman for framing at large, however, is the most widely accepted one. He defined framing as “making some aspects of reality more salient in a text in order to promote a particular problem definition, causal interpretation, moral evaluation, and/or treatment recommendation for the item described” . Narrowing the framework to news, De Vreese 2005 lays out a typology of news framing. He states that there are generic news frames and issue-specific news frames. Generic news frames “transcend thematic limitations and can be identified in relation to different topics, some even over time and in different cultural contexts” . They are particularly useful for uncovering broad patterns within or across countries. Issue-specific frames allow for richer, more fine-grained analysis of various aspects highlighted within a particular issue. Framing Through Visuals Photographs are an important vehicle of framing as a reader may process textual and visual messages differently. Readers may focus on photographs without also reading an accompanying story Miller 1975 or might select which news story to read depending on the image thumbnail. Images are potent framing tools because they evoke immediate emotional responses, provide contextual cues, and sometimes contradict textual narratives Geise and Xu 2024 . For visual data, however, automation is more challenging because computational image analysis often struggles with connotative and symbolic elements that are readily discernible to human annotators Rodriguez and Dimitrova 2011 . While some recent advancements in machine learning and computer vision offer promising avenues for automated visual framing analysis, the complexity of symbolic and ideological meanings typically requires human interpretation Coleman 2010 . Automated Framing Analysis In computational studies, framing has been studied with the help of machine learning methods for content analysis at scale. Ali and Hassan 2022 ; Vallejo et al. 2023 ; Otmakhova et al. 2024 survey these efforts towards on computational framing analysis, focusing on different aspects like methods, varying conceptualisations and inter-disciplinary connections of framings. The most widely used resource is one by Card et al. 2015 , who present the Media Frames Corpus (MFC), a dataset of US based news headlines annotated for generic frames. Most

Read the complete text