Historical Ink: Exploring Large Language Models for Irony Detection in 19th-Century Spanish

Kevin Cohen, Laura Manrique-Gómez, Rubén Manrique · 2025

Read Historical Ink: Exploring Large Language Models for Irony Detection in 19th-Century Spanish by Kevin Cohen, Laura Manrique-Gómez, Rubén Manrique (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-22585.txt.

Introduction

Kevin Cohen, Laura Manrique-Gómez, and Rubén Manrique tested whether AI language models can spot irony in old Spanish newspapers, in a 2025 paper. Irony is hard for machines at the best of times, and nineteenth-century Latin American Spanish adds dated words and lost cultural cues. The team built a new labeled dataset from period newspapers to work with.

They tried two strategies with two kinds of models. Adding emotional and background cues to the data helped little. A second plan worked better: the model drafted labels and human experts corrected them, which fixed lopsided class counts and raised label quality. The authors argue that human judgment, armed with period knowledge, stays central when machines read historical texts. Their dataset and method are offered for wider use in sentiment studies.

Catalog record

CREATORKevin Cohen, Laura Manrique-Gómez, Rubén Manrique
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE42 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2503-22585

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Historical Ink: Exploring Large Language Models for Irony Detection in 19th-Century Spanish Kevin Cohen, Laura Manrique-Gómez, Rubén Manrique Abstract This study explores the use of large language models (LLMs) to enhance datasets and improve irony detection in 19th-century Latin American newspapers. Two strategies were employed to evaluate the efficacy of BERT and GPT-4o models in capturing the subtle nuances nature of irony, through both multi-class and binary classification tasks. First, we implemented dataset enhancements focused on enriching emotional and contextual cues; however, these showed limited impact on historical language analysis. The second strategy, a semi-automated annotation process, effectively addressed class imbalance and augmented the dataset with high-quality annotations. Despite the challenges posed by the complexity of irony, this work contributes to the advancement of sentiment analysis through two key contributions: introducing a new historical Spanish dataset tagged for sentiment analysis and irony detection, and proposing a semi-automated annotation methodology where human expertise is crucial for refining LLMs results, enriched by incorporating historical and cultural contexts as core features. 1 Introduction Irony is a nuanced and often subtle form of communication, especially in historical texts, where cultural context is crucial in understanding the intended meaning. Detecting irony in written language has long been a challenge for natural language processing (NLP) González-Ibáñez et al. (2011) , as it is a trope whose actual meaning differs from what is literally enunciated Hee et al. (2018) . Hence, irony detection involves identifying contradictions between what is said and the underlying meaning or context. This challenge becomes even more complex when dealing with historical texts, where linguistic expressions, cultural references, and societal norms differ significantly from modern times. This research focuses on irony detection in 19th-century Latin American newspapers, utilizing large language models (LLMs) and feed-forward neural networks to experiment with various strategies. The dataset, annotated by experts, was processed for multi-class and binary classification. BERT-like models were used for text encoding and transfer learning, while GPT-4o models were applied for sentiment classification with customized prompts. A BERT encoder with a feed-forward neural network was used as a comparative baseline to examine the enhancement of text analysis specific to irony detection. A semi-automatic annotation approach was also developed to incorporate new, untagged data. By employing GPT-4o models with tailored prompts, initial classifications were machine-generated and verified by human experts, reducing annotation time and effort while maintaining high-quality standards. Integrating human expertise with machine-generated results enhanced the dataset and allowed for more effective model training. Consequently, the study contributes to research on sentiment analysis in historical texts and demonstrates LLMs’ capabilities in enhancing text classification in specialized contexts. 2 Related Work Philosophers and linguists have yet to reach a definitive agreement on defining certain figurative tropes including irony, sarcasm, satire, hyperbole, analogy, restatement, paradox, and parody. There have been arguments regarding subtle differences, such as the humorous intention in irony versus the explicitly offensive intention in verbal sarcasm. However, there is a broader consensus on considering irony as the overarching category Kreuz (2020) ; Colston (2017) . In NLP, irony, sarcasm, and satire have often been used interchangeably, focusing on linguistic and sentiment features. Early work suggested satire exploits human tendencies like gullibility and confirmation bias Rubin et al. (2016) . Other approaches, achieving moderate success, explored linguistic patterns, including slang and profanity, using techniques such as Bi-normal Separation (BNS) Burfoot and Baldwin (2009) . Traditional rule-based methods relied on linguistic indicators like interjections and hyperbole Riloff et al. (2013) . However, deep learning models such as ELMo and BERT have revolutionized the field by incorporating embeddings that capture contextual information across sentences. Rajadesingan et al. emphasized the importance of user behavior and historical tweet data in improving sarcasm classification, demonstrating how conversational context can enhance detection accuracy Rajadesingan et al. (2015) . Both satire and sarcasm detection have significantly benefited from integrating contextual features and deep learning, showing superior performance compared to traditional methods. Nonetheless, challenges remain due to the reliance on cultural and societal context, which complicates creating highly accurate models. Irony detection has similarly grown in importance as researchers address the challenges of figurative language. Early approaches to irony detection primarily utilized static linguistic features and manually annotated datasets but struggled to capture irony’s nuanced and dynamic nature, particularly when dealing with context-specific language, such as that found in social media and historical texts. The introduction of deep learning models using transformer-based architectures for LLMs like BERT and GPT has significantly improved the capacity for irony detection. Huang et al. explored the application of deep learning techniques such as Recurrent Neural Networks (RNN), revealing that models with attention mechanisms outperformed others on irony detection tasks using social media data Huang et al. (2017) . These models excel by capturing both syntactic and semantic relationships within sentences, enabling comprehensive irony analysis. Similarly, Ren et al. proposed a knowledge-enhanced neural network that incorporates contextual information from external knowledge sources like Wikipedia to enhance irony detection performance Ren et al. (2023) . Recent studies have focused on improving irony detection by integrating emotional and contextual cues. Lin et al. introduced a newly developed method combining LLMs with emotion-centric text enhancement to improve the detection of irony Lin et al. (2024) . Their approach highlighted the significance of subtle emotional cues often overlooked in traditional models. By using GPT-4 to expand original texts with additional content, the researchers significantly enhanced irony detection in benchmark datasets. Ozturk et al. introduced a de-biasing approach for irony, satire, and sarcasm detection, utilizing generative LLMs to reduce stylistic biases produced by single-source corpus training datasets Ozturk et al. (2024) . Their findings indicate that such stylistic bias impacts model robustness and that LLMs-based enhancement can mitigate these biases. However, its effect on causal language models like Llama-3.1 remains limited. This approach aligns with recent trends in dataset enhancement and bias reduction to improve the detection of figurative language. Recent methodologies introduced a human-LLMs collaborative annotation framework that addresses the limitations of LLMs-generated labels by combining automated annotation with human expertise Wang et al. (2024) . Their method incorporates three major steps: LLMs predict labels and generate explanations; a verifier assesses label quality; and human annotators review and re-annotate labels flagged as low quality. This research shows the necessity for hybrid approaches that merge LLMs scalability with human precision, particularly for complex tasks such as irony detection. Despite advancements, challenges persist in handling class imbalance, stylistic bias, and the nuanced nature of figurative language, especially in historical texts. Prior work on deep learning models and LLMs-driven enhancements often focuses on contemporary datasets, neglecting historical linguistic and cultural context variations. Moreover, research on semi-automated annotation strategies leveraging human-LLMs collaboration is limited. Our study addresses these gaps by integrating a structured semi-automated annotation process, improving dataset balance, and fine-tuning domain-specific models for 19th-century Spanish. By combining LLMs-powered augmentation with human verification, we propose a scalable method for enriching training data while maintaining historical linguistic authenticity. This work refines irony detection in historical texts and provides a broader framework for augmenting and improving figurative language classification in underrepresented domains. 3 Data The dataset used in this research initially comprised a tagged corpus of 2,734 entries. Each entry contains text samples randomly extracted from 19th-century Latin

Read the complete text