Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models
Enfa Fane, Mihai Surdeanu, Eduardo Blanco, Steven R. Corman · 2025
Read Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models by Enfa Fane, Mihai Surdeanu, Eduardo Blanco, Steven R. Corman (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-20469.txt.
Introduction
Enfa Fane and co-authors test in 2025 how well large language models can label the framing roles of entities in news stories, with no task training. How news casts a person or group shapes how the public reads an event. Their entry to a shared task sorts each entity first into a broad role, then into a fine grained one.
That two step plan beat one step labeling in their tests. The best input context and prompt wording differed between the broad and fine steps, so each level needed its own tuning. Their system scored high on main role accuracy, with a lower exact match rate on the full fine labels. The paper's lesson is practical: for framing work, prompt design is not one size fits all, and splitting the task helps models judge narrative roles more like human coders.
Catalog record
| CREATOR | Enfa Fane, Mihai Surdeanu, Eduardo Blanco, Steven R. Corman |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-20469
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Fane at SemEval-2025 Task 10: Zero-Shot Entity Framing with Large Language Models Enfa Fane, Mihai Surdeanu, Eduardo Blanco, Steven R. Corman Abstract Understanding how news narratives frame entities is crucial for studying media’s impact on societal perceptions of events. In this paper, we evaluate the zero-shot capabilities of large language models (LLMs) in classifying framing roles. Through systematic experimentation, we assess the effects of input context, prompting strategies, and task decomposition. Our findings show that a hierarchical approach of first identifying broad roles and then fine-grained roles, outperforms single-step classification. We also demonstrate that optimal input contexts and prompts vary across task levels, highlighting the need for subtask-specific strategies. We achieve a Main Role Accuracy of 89.4% and an Exact Match Ratio of 34.5%, demonstrating the effectiveness of our approach. Our findings emphasize the importance of tailored prompt design and input context optimization for improving LLM performance in entity framing. 1 Introduction News is produced and consumed at an unprecedented scale, yet the expectation of neutrality and objectivity in journalism ( Schudson 2001 ) often contrasts with the reality that media selectively determines what is newsworthy ( Eliaz and Spiegler 2024 ) . Through inclusion, omission, or emphasis of specific details, journalists shape public interpretation of events ( Iyengar 1990 ) , promoting particular recommendations ( Entman 1993 ) , which in turn influences public opinion and policy decisions ( Ziems and Yang 2021 ) . When such selective framing is used to shape perceptions of an entity, it is known as entity framing ( van den Berg et al. 2020 ) . SemEval-2025 Task 10 introduces a benchmark for analyzing entity framing ( Piskorski et al. 2025 ; Stefanovitch et al. 2025 ) , examining how linguistic choices from word selection to narrative structure affect perception. Example in Figure 1 demonstrates how strategic language choices portray the entity Bill Gates as an Antagonist. Despite mentioning his substantial investments in climate change initiatives, the narrative emphasizes his high carbon footprint, highlighting hypocrisy. Phrases such as “carefully constructed interview” and references to his connections with Epstein further reinforce an image of deception and evading accountability. This selective emphasis shapes the reader’s perception of Gates as a Deceiver and Corrupt figure, demonstrating how framing subtly influences interpretation. Gates claimed that because he continues to spend billions of dollars on climate change activism, his carbon footprint isn’t an issue […] Elsewhere during the carefully constructed interview , Gates said he was surprised that he was targeted by ‘conspiracy theorists’ for pushing vaccines during the pandemic […] While the BBC interview was set up to look like Gates was being challenged or grilled, he wasn’t asked about his close friendship with the elite pedophile Jeffrey Epstein Figure 1: An example document excerpt from the dataset. The author strategically uses loaded language, contrastive framing, and selective emphasis to shape the reader’s perception of Bill Gates as corrupt and deceptive. By highlighting contradictions in his public advocacy, casting doubt on the authenticity of his media appearances, and referencing controversial associations, the text reinforces a negative framing. Recent advances in large language models (LLMs) have demonstrated superior performance in text classification, frequently outperforming traditional machine learning approaches on complex tasks ( Kostina et al. 2025 ) . In this paper, we explore if this extends to entity framing. Relying solely on prompting, we ask whether LLMs accurately identify the framing roles assigned to entities in news articles. Given the sensitivity of LLM predictions to prompt variability ( Zhuo et al. 2024 ) , we assess how prompt engineering strategies such as role-based prompting, incorporating task related information such as label definitions, and rationale generation shape classification performance. To this end, we develop a modular zero-shot approach that relies exclusively on prompting. Our method decomposes the task into two stages, first predicting broad narrative roles (Protagonist, Antagonist, Innocent), and then refining into fine-grained roles. We systematically vary the input context comparing full articles, entity-centered excerpts, and framing-preserving summaries and design prompt templates incorporating expert personas and task related information. The main contributions of this paper are: (1) Systematic evaluation of LLMs for zero-shot entity framing: We assess how well LLMs infer framing roles from implicit narrative cues without fine-tuning. (2) A multi-step prompting strategy: We show that decomposing the task into two stages, predicting broad roles first, then fine-grained roles, yields better results than a single-step approach. (3) Strategic prompt design: We demonstrate that effective prompt design, incorporating role-based guidance and task definitions, improves classification. A carefully engineered prompt enables a smaller model to perform comparably to a larger one. We report a Main Role Accuracy of 89.4% and an Exact Match Ratio of 34.5% on the SemEval 2025 Task 10 Subtask 1 for English, placing sixth on among the participating teams. We make our codebase and associated prompts publicly available. 1 1 1 https://github.com/beingenfa/semeval2025task10 2 Background: Task & Dataset The SemEval 2025 Task 10: Multilingual Characterization and Extraction of Narratives from Online News introduces three subtasks for analyzing narrative elements in news articles across five languages: Bulgarian, English, Hindi, European Portuguese, and Russian. This paper focuses on Subtask 1: Entity Framing for English . The goal of Entity Framing is to determine how a named entity is framed within a news article. Framing is represented at two levels: Main Role and Fine-Grained Roles . The main role broadly categorizes the entity as a Protagonist, Antagonist, or Innocent, while fine-grained roles provide more detailed labels. 2 2 2 The fine-grained labels are listed in Appendix A An entity may be framed with multiple fine-grained labels, making this a multi-class, multi-label classification problem. For further details on the dataset, including domain coverage, corpus statistics, and annotation methodology, see ( Piskorski et al. 2025 ; Stefanovitch et al. 2025 ) . We now describe our approach to leveraging large language models (LLMs) for entity framing classification. 3 Related Work Language shapes interpretation by selectively emphasizing aspects of reality ( Entman 1993 ) . Prior research identifies two key linguistic mechanisms underlying framing effects: naming conventions , where respectful or formal titles correlate strongly with positive sentiment ( van den Berg et al. 2019 ; van den Berg et al. 2020 ) ; and narrative framing , reflecting partisan differences in media portrayals of events ( Ziems and Yang 2021 ) . However, these studies primarily treat framing as an implicit phenomenon correlating with other tasks, rather than explicitly classifying or predicting specific framing roles or labels. Beyond text, in the multimodal domain, Sharma et al. 2022 introduce HVVMemes, a meme dataset covering COVID-19 and US politics, annotated with coarse-grained framing roles: hero , villain , victim , or none . In contrast, our work focuses on news text and classifies entities at a more fine-grained, hierarchical roles. Large language models (LLMs) have recently achieved strong performance in text classification tasks, often surpassing traditional machine learning methods on complex datasets ( Kostina et al. 2025 ) . However, significant challenges remain, particularly concerning sensitivity to prompt phrasing and input structure ( Zhuo et al. 2024 ) . Structured prompting strategies, such as role conditioning ( Kong et al. 2024 ) and instruction reframing ( Mishra et al. 2022 ) , have