Unlocking Cross-Lingual Sentiment Analysis through Emoji Interpretation: A Multimodal Generative AI Approach
Rafid Ishrak Jahan, Heng Fan, Haihua Chen, Yunhe Feng · 2024
Read Unlocking Cross-Lingual Sentiment Analysis through Emoji Interpretation: A Multimodal Generative AI Approach by Rafid Ishrak Jahan, Heng Fan, Haihua Chen, Yunhe Feng (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-17255.txt.
Introduction
Rafid Ishrak Jahan, Heng Fan, Haihua Chen, and Yunhe Feng test whether emojis can serve as a shared sentiment signal across languages, in work from 2024. Emojis cross language lines by design and carry feeling in a compact form. Whether a model can read that feeling reliably had not been tested at scale.
The team used a multimodal chatbot to read emoji sentiment in a data set drawn from thirty-two countries. Emoji-based calls matched text sentiment about eighty-one percent of the time. Accuracy rose as texts held more emojis. The authors argue emojis hold real promise as a global sentiment marker. Such a marker could aid brands and platforms that work across many tongues at once.
Catalog record
| CREATOR | Rafid Ishrak Jahan, Heng Fan, Haihua Chen, Yunhe Feng |
|---|---|
| YEAR | 2024 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2412-17255
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Unlocking Cross-Lingual Sentiment Analysis through Emoji Interpretation: A Multimodal Generative AI Approach Rafid Ishrak Jahan, Heng Fan, Haihua Chen, Yunhe Feng Abstract . Emojis have become ubiquitous in online communication, serving as a universal medium to convey emotions and decorative elements. Their widespread use transcends language and cultural barriers, enhancing understanding and fostering more inclusive interactions. While existing work gained valuable insight into emojis understanding, exploring emojis’ capability to serve as a universal sentiment indicator leveraging large language models (LLMs) has not been thoroughly examined. Our study aims to investigate the capacity of emojis to serve as reliable sentiment markers through LLMs across languages and cultures. We leveraged the multimodal capabilities of ChatGPT to explore the sentiments of various representations of emojis and evaluated how well emoji-conveyed sentiment aligned with text sentiment on a multi-lingual dataset collected from 32 countries. Our analysis reveals that the accuracy of LLM-based emoji-conveyed sentiment is 81.43%, underscoring emojis’ significant potential to serve as a universal sentiment marker. We also found a consistent trend that the accuracy of sentiment conveyed by emojis increased as the number of emojis grew in text. The results reinforce the potential of emojis to serve as global sentiment indicators, offering insight into fields such as cross-lingual and cross-cultural sentiment analysis on social media platforms. Code: https://github.com/ResponsibleAILab/emoji-universal-sentiment . 1. Introduction and Background Emojis have evolved from simple novelties into powerful tools for expressing sentiments and emotions across diverse social media platforms. They serve as substitutes for non-verbal cues, such as facial expressions and gestures, which are often absent in text-based communication ( Bai et al. 2019 ) . Due to their unique ability to enhance communication and enrich user experiences ( Müller et al. 2019 ) , emojis have become integral to digital interactions. Their universal appeal lies in their capacity to transcend linguistic and cultural barriers, making them a compelling subject of study, particularly in the domain of sentiment analysis. Recent research has provided valuable insight into understanding emoji sentiment utilizing emojis and their representations. For example, Novak et al. ( Kralj Novak et al. 2015 ) engaged 83 human annotators to label the sentiment polarity (negative, neutral, or positive) of 751 emojis, employing the mean of the discrete probability distribution. Similarly, Gavilanes et al. ( Fernández-Gavilanes et al. 2021 ) assessed the sentiment quality of emoji descriptions obtained from various online resources, including Emojipedia, Emojis Wiki, CLDR emoji character annotations, and iEmoji. Their analysis explored the extent to which these descriptions accurately reflect the intended sentiment of the emojis. However, these studies are constrained by their focus on European languages and the limited scope of emoji representations within their descriptions. Consequently, they fail to account for the influence of global linguistic diversity and diverse emoji representations—such as titles, pixel-level designs, and others—on capturing the intended sentiment of emojis. Further analysis sheds light on predicting emoji usages based on text-based context. Zhao et al. ( Zhao et al. 2018 ) introduced a multitask multimodality gated recurrent unit to predict emoji usage in social media posts. Likewise, it was found by Qiu et al. ( Qiu et al. 2024 ) that ChatGPT-4 (GPT-4) performs better than six LLMs when recommending emojis align with the semantics of a text. Yet, neither work portrays the picture of GPT-4 semantic preservation in the context of different languages and cultures. It is plausible that GPT-4 may occasionally generate inaccurate suggestions for languages impacted by low-resource language challenges ( Yong et al. 2024 ) . Building on emoji-based research, other studies have highlighted the cosmopolitan nature of emojis and the potential of Generative AI understanding emojis. For example, Barbieri et al. ( Barbieri et al. 2016 ) observed that the 150 most popular emojis preserve their semantics in four languages across four languages: American English, British English, Peninsular Spanish, and Italian. Additionally, Zhou et al. ( Zhou et al. 2024 ) reported that ChatGPT’s emoji annotations align closely with human interpretations. However, neither study explicitly examines the accuracy and consistency of emoji interpretations across a broader range of languages and cultural contexts. While existing studies laid strong baselines for emoji research, gaps remain in addressing whether emojis have the potential to serve as a standalone medium of sentiment across languages. Moreover, traditional sentiment analysis often encounters challenges in cross-lingual contexts. Language-specific models are difficult to adapt to texts from various languages, and training multilingual models requires substantial computing resources ( Shan et al. 2025 ) . Additionally, being intricately tied to culture, sentiment does not always translate seamlessly across languages. It necessitates a universal medium that can serve as a bridge in multilingual sentiment analysis. This paper investigates the feasibility of using emojis as a universal medium for sentiment indicators. Leveraging generative AI technologies, we analyze the sentiment of various emoji representations—including symbols, titles, descriptions, and pixel-level designs—across 19 languages and 32 countries. The contributions of this paper are summarized as follows: • We investigated diverse multimodal representations of emojis for sentiment analysis using generative AI, demonstrating that emoji pixels, icons, and descriptions deliver the best performance. • We proposed scalable, emoji-based, language-agnostic sentiment analysis approaches to infer the sentiment of text effectively. • Our comprehensive cross-language evaluation highlights the potential of emojis as universal and standalone sentiment markers. 2. Methodology This section begins by detailing the process of collecting various representations of emojis. Next, it describes the evaluation and selection of emoji representation combinations that achieve optimal performance for sentiment analysis. Finally, we introduce emoji-based approaches for sentiment analysis in textual data. 2.1. Emoji Representation Collection Emojis can be represented in various formats, including icons, textual titles, descriptions, and pixel-based images, where icons refer to the visual representation of emojis, textual titles provide their concise names, descriptions offer detailed semantic explanations, and pixel-based images capture their graphical structure. To effectively evaluate the performance of generative AI models in sentiment analysis using different emoji representations, it is essential to explore individual and combined representations. The initial step towards this goal involves constructing a comprehensive emoji representation dataset encompassing all available emojis across these multiple formats. Given that no single data source offers a complete set of emoji representations covering all four formats (icons, titles, descriptions, and pixels), we developed an open-source tool 1 1 1 https://github.com/ResponsibleAILab/emoji-universal-sentiment to address this gap. Specifically, we aggregated and combined data from Emojipedia 2 2 2 https://emojipedia.org and the Unicode Consortium 3 3 3 https://www.unicode.org to construct our emoji representation dataset for 5030 emojis, ensuring a comprehensive and structured resource for further analysis. 2.2. LLM-based Emoji Sentiment Estimation With diverse emoji representations established, we employ generative AI to estimate the sentiment of emojis across various representation formats. This sentiment can subsequently be used to infer the sentiment of multilingual text containing standalone emojis. Given that LLMs, such as ChatGPT, are trained on extensive datasets that inherently capture emoji sentiment, we propose that LLMs can provide a more optimal approach for estimating emoji sentiment compared to traditional manual labeling methods. To determine the most suitable emoji representation for sentiment inference, we used Emoji Sentiment Ranking v1.0 (ESR v1.0) ( Kralj Novak et al. [n. d.] ) , which offers a sentiment overview of 751 emojis based on category (positive, neutral, negative), as