Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News

Raven Adam, David Maier, Marie Kogler · 2026

Read Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News by Raven Adam, David Maier, Marie Kogler (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-26489.txt.

Introduction

Raven Adam, David Maier, and Marie Kogler compare two ways to detect framing in German climate news. Framing is whether a sentence stresses threats from climate change or solutions to it. Coding sentences by hand does not scale to large news collections. The authors test a fine-tuned German BERT model against few-shot prompting of a large language model. Few-shot prompting gives the model a few worked examples in its instructions.

Both approaches run two separate classifiers, one for threat and one for solution framing. The test set holds four hundred forty Austrian newspaper articles coded by hand with expert help. The fine-tuned BERT classifiers scored higher on the standard F1 measure. F1 balances missed cases against false alarms. BERT reached a score the authors report as clearly ahead of the language model. Giving BERT the previous sentence as context helped a great deal.

Catalog record

CREATORRaven Adam, David Maier, Marie Kogler
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE34 KB
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2606.26489

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Comparing BERT Sentence-Pair Classification and Few-Shot LLM Prompting for Detecting Threat and Solution Framing in German Climate News Raven Adam, David Maier, Marie Kogler Abstract News media play a central role in shaping public perceptions of climate change, and whether coverage emphasizes threats or solutions has measurable effects on audience engagement and policy support. Automated detection of these framing patterns at the sentence level would allow researchers to analyze large corpora that are infeasible to code manually. We present a systematic comparison of two approaches for classifying sentences from German-language climate news articles as threat-oriented, solution-oriented, both, or neither. The first approach uses few-shot prompting with an open-weights large language model (Llama 4 Maverick), employing chain-of-thought reasoning and structured output with confidence scoring. The second approach fine-tunes a German BERT model ( deepset/gbert-large ) for sentence-pair classification, where the preceding sentence provides contextual information for the target sentence. Both approaches implement two independent binary classifiers, one for threat framing and one for solution framing. We evaluate both methods on a corpus of 440 Austrian newspaper articles that were manually coded following a detailed coding scheme developed with domain experts. The fine-tuned BERT classifiers achieve an F1 score of 0.83 for both the threat and solution tasks, while the LLM-based classifiers reach an F1 of 0.78. An ablation study confirms that providing the preceding sentence as context improves BERT classification performance substantially compared to single-sentence input. These results contribute to the growing body of work comparing fine-tuned encoder models with prompted generative models for text classification in computational social science. Keywords: climate change framing, text classification, BERT, large language models, few-shot prompting, German NLP, computational content analysis 1 Introduction How the media frames climate change affects what audiences understand, feel, and are willing to do about it. Research in climate communication has repeatedly shown that threat-oriented framing, which highlights risks and dangers, and solution-oriented framing, which presents actions and remedies, produce different audience responses ( 27 ; 21 ) . Threat framing without accompanying efficacy information can induce fatalism, while solution-oriented coverage promotes constructive engagement and policy support ( 20 ; 42 ) . Tracking the balance between these two framing types across large media corpora is therefore of direct interest to communication researchers studying climate discourse. Content analysis remains the primary method for studying media framing, but manual coding is labor-intensive and difficult to scale. A single study may require weeks of trained human effort to code a few hundred articles ( 24 ; 6 ) . Automated approaches based on natural language processing (NLP) can process orders of magnitude more text, though their reliability must be validated against human judgments ( 24 ) . Among the available automated approaches, two have become dominant in recent years. The first involves fine-tuning pre-trained transformer-based encoder models such as BERT ( 15 ) on labeled training data. The second relies on prompting large language models (LLMs) to perform classification through in-context learning, without parameter updates ( 7 ; 23 ) . Both approaches have demonstrated strong performance on various text classification tasks, but they involve fundamentally different trade-offs. Fine-tuned models require labeled training data and computational resources for training, but produce fast and deterministic inference. LLM-based approaches require no training data beyond a few optional examples in the prompt, but depend on careful prompt design and incur higher per-sample inference costs. Recent comparative studies find that fine-tuned encoder models often outperform zero-shot and few-shot LLMs on classification tasks, while prompting-based approaches such as chain-of-thought and structured outputs can improve performance in some settings ( 8 ; 22 ; 17 ; 46 ; 16 ) . Despite growing interest in automated framing detection ( 2 ; 39 ) and LLM-based content analysis ( 23 ; 48 ) , no study has directly compared these two approaches for detecting climate-specific framing at the sentence level. Most computational framing work uses generic frame taxonomies ( 9 ) rather than domain-specific categories. Generic frames, however, do not capture the threat-versus-solution distinction that is central to climate communication research, and classifiers trained on one frame taxonomy do not transfer reliably to another. Similarly, work on German-language climate media remains limited ( 28 ; 1 ) , even though framing patterns, editorial conventions, and linguistic structures differ across languages in ways that affect classifier performance ( 39 ; 37 ) . This paper makes three contributions. First, we present a systematic comparison of fine-tuned BERT and few-shot LLM prompting for sentence-level climate framing classification, evaluated on manually coded German newspaper articles. Second, we demonstrate that providing the preceding sentence as context through BERT’s sentence-pair input format yields substantial improvements over single-sentence classification. Third, we describe a detailed prompt architecture for LLM-based framing classification that incorporates chain-of-thought reasoning, a speech-act taxonomy, and domain-specific coding rules in the source language. 2 Related Work 2.1 Threat and Solution Framing in Climate Communication Framing, as defined by 18 , involves selecting and making salient certain aspects of a perceived reality to promote particular problem definitions, causal interpretations, and treatment recommendations. In climate communication, framing research has identified a wide range of recurring patterns. 38 proposed a typology of climate-specific frames including scientific uncertainty, economic development, and morality. 4 mapped 274 studies and found scientific, economic, and environmental frames to be dominant, while 26 identified 18 pre-defined frames across 25 years of literature, with “climate action” and “harmful impacts” as the two most commonly studied. The distinction between threat and solution framing is particularly relevant for understanding audience responses. 27 found that U.S. television news frequently conveys climate threats but provides inconsistent efficacy information, meaning that impacts and potential actions rarely appear together. 20 extended this finding to print media and documented partisan differences in how newspapers balance threat and efficacy. More recently, 25 analyzed the shift from doom framing toward sustainable solutions in international news magazines, and 43 conducted the most detailed empirical study of solution-oriented climate framing to date, analyzing 244 articles and identifying three previously undescribed solution frames. Research on German-language and Austrian climate media remains limited but growing. 28 found that climate coverage in Germany accounted for only 0.24% of all articles in their ten-country sample. 1 analyzed long-term trends in Austrian newspaper coverage using NLP pipelines. More recently, 14 used GPT-4-assisted analysis to study German media coverage of climate activist groups, and 36 examined framing differences across 21 German outlets with varying political orientations. 2.2 Transformer-Based Text Classification The Transformer architecture ( 44 ) and its bidirectional variant BERT ( 15 ) established the current paradigm for text classification through pre-training and fine-tuning. BERT’s sentence-pair input format, which concatenates two text segments with special separator tokens, was originally designed for tasks such as natural language inference and paraphrase detection. This format also enables classification of a target sentence with additional contextual information from a second segment. For German-language tasks, 11 introduced GBERT and GELECTRA, monolingual German models trained with whole-word masking. These models outperform multilingual alternatives such as XLM-RoBERTa ( 13 ) on German NLP benchmarks, making monolingual models the preferred choice for single-language classification tasks. The use of surrounding context for sentence-level classification has been explored in several studies. 12 showed that incorporating document context through joint

Read the complete text