MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models
Zhengyi Zhao, Shubo Zhang, Yuxi Zhang, Yanxi Zhao, Yifan Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu · 2025
Read MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models by Zhengyi Zhao, Shubo Zhang, Yuxi Zhang, Yanxi Zhao, Yifan Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17433.txt.
Introduction
Zhengyi Zhao and co-authors build a benchmark for meme understanding in context in a paper from 2025. A meme's meaning swings with the chat around it. The same image can praise, mock, or mourn depending on where it lands. Past tests judged memes alone, for harm or sense, and missed that swing.
MemeReaCon collects memes with their original conversational setting. Large vision-language models must grasp the intent the context gives. Human readers do this at a glance; models struggle, the team finds. The benchmark exposes a real gap in machine social sense. Better context reading matters for moderation tools that judge posts in the wild, not in isolation.
Catalog record
| CREATOR | Zhengyi Zhao, Shubo Zhang, Yuxi Zhang, Yanxi Zhao, Yifan Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 56 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17433
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models Zhengyi Zhao, Shubo Zhang, Yuxi Zhang, Yanxi Zhao, Yifan Zhang, Zezhong Wang, Huimin Wang, Yutian Zhao, Bin Liang, Yefeng Zheng, Binyang Li, Kam-Fai Wong, Xian Wu Abstract Memes have emerged as a popular form of multimodal online communication, where their interpretation heavily depends on the specific context in which they appear. Current approaches predominantly focus on isolated meme analysis, either for harmful content detection or standalone interpretation, overlooking a fundamental challenge: the same meme can express different intents depending on its conversational context. This oversight creates an evaluation gap: although humans intuitively recognize how context shapes meme interpretation, Large Vision Language Models (LVLMs) can hardly understand context-dependent meme intent. To address this critical limitation, we introduce MemeReaCon, a novel benchmark specifically designed to evaluate how LVLMs understand memes in their original context. We collected memes from five different Reddit communities, keeping each meme’s image, the post text, and user comments together. We carefully labeled how the text and meme work together, what the poster intended, how the meme is structured, and how the community responded. Our tests with leading LVLMs show a clear weakness: models either fail to interpret critical information in the contexts, or overly focus on visual details while overlooking communicative purpose. MemeReaCon thus serves both as a diagnostic tool exposing current limitations and as a challenging benchmark to drive development toward more sophisticated LVLMs of the context-aware understanding. 1 Introduction Memes are “amateur media artifacts, extensively remixed and recirculated by different participants on social media networks” Milner (2012) that have become a key part of how people communicate online. These combinations of images and text derive meaning not just from their content, but from their contextual placement: where they appear, why they are shared, and how communities respond to them. A meme posted in a programmer joke forum carries a fundamentally different meaning than the same meme shared in a generic community, as illustrated in Figure 1 . While humans naturally process these contextual distinctions, developing computational models that can achieve similar understanding remains a significant challenge Wang et al. (2024) . Figure 1: Demo illustrating how a single meme’s interpretation changes across different contextual settings. Meme here literally indicates you know the difference between “Stack” and “Heap” . The “Stack” and “Heap” mean specific terms in programmer community, but can mean condition of an item in general talk. Current meme-focused research has largely pursued two distinct paths, neither fully capturing the contextual richness of memes in real online communication. The first approach centers on detecting harmful or toxic meme content Sharma et al. (2022) ; Hee et al. (2023) ; Huang et al. (2024) . While crucial for content moderation systems, this research typically leverages context primarily as a classifier for harmfulness rather than for comprehensive meaning interpretation. The second research direction tackles isolated meme understanding through tasks like caption generation Hwang and Shwartz (2023) , intent description Park et al. (2024) , and role explanation Sharma et al. (2023) . Despite their value, these efforts examine memes divorced from their original context, separating them from post text, creator intent, and community reactions that collectively shape their contextual meaning. This decontextualization creates a fundamental evaluation gap: we lack methods to assess whether LVLMs can understand why particular memes are selected for specific communicative situations. As Park et al. (2024) observed, people create memes “with an intent to perform some action”. The same meme template can convey radically different meanings depending on its accompanying post title, community norms, or ongoing conversation thread Lin et al. (2024) . Without incorporating these contextual elements, we cannot effectively measure LVLMs’ capacity to process memes as humans naturally do in online environments. To address these limitations, we developed MemeReaCon : Meme Reasoning in Context, a comprehensive benchmark specifically designed to evaluate LVLMs’ ability to understand memes within their original contexts. We constructed MemeReaCon using content from five diverse Reddit communities, encompassing varied topics, styles, and community norms. Each example preserves three critical contextual elements: the meme image itself, the complete post text, and the top-rated community comments that reveal collective interpretation. Beyond mere data collection, our benchmark includes detailed annotations that enable targeted analysis of specific contextual understanding dimensions. Through MemeReaCon, we investigate two fundamental questions about current LVLM limitations: (1) To what extent do models understand the meme? (2) To what extent does the post context affect models’ understanding of meme? Our extensive evaluation of leading LVLMs reveals a persistent weakness in contextual integration. Models frequently fail to establish meaningful connections between memes and their context, either fail to interpret critical information in the contexts, or overly focus on visual details while overlooking communicative purpose. Detailed error analysis reveals that models are sensitive to context type, such that models often fail in culturally dominant contexts rather than giving specific tags or communities. Our work makes following contributions: • To our knowledge, we firstly identify how the post context and meme work together: post context mainly explains the meme, or the meme illustrates points made in the context. This helps us to evaluate models whether understand different ways people use memes to communicate. • We propose a novel benchmark, MemeReaCon, for meme understanding that maintains the essential relationship between meme images, post, and community reception, enabling the first systematic evaluation of how well LVLMs interpret memes as they actually function in online environments. • We conduct comprehensive evaluation, revealing contextual-insensitive limitations in current LVLMs to connect multimodal elements for contextual interpretation. Dataset Task Type Post Context Comments Size MultiOFF Suryawanshi et al. (2020) classify: meme hatefulness ✘ ✘ 743 HatefulMemes Kiela et al. (2020) classify: meme hatefulness ✘ ✘ 10k Jewtocracy Chandra et al. (2021) classify: meme hatefulness ✘ ✘ 6,611 HarMeme Pramanick et al. (2021a) classify: meme hatefulness/target ✘ ✘ 3,544 MAMI Fersini et al. (2022) classify: meme hatefulness ✘ ✘ 15k FigMemes Liu et al. (2022) classify: meme political opinion ✘ ✘ 5,141 HVVMemes Sharma et al. (2022) classify: meme character role ✘ ✘ 7k GOAT Lin et al. (2024) classify: meme hatefulness ✘ ✘ 6,626 HatReD Hee et al. (2023) explain: meme ✘ ✘ 3,228 ExHVV Sharma et al. (2023) explain: meme ✘ ✘ 4,680 MemeCap Hwang and Shwartz (2023) explain: meme + metaphors ✔ ✘ 6,387 MemeIntent Park et al. (2024) explain: metaphors ✔ ✘ 950 MemeReaCon (ours) classify: meme + post + comment type/affection ✔ ✔ 1,565 explain: meme + metaphors + post + post intents Table 1: Comparisons with other related meme benchmarks. 2 Related Works Meme Classification. The detection of harmful memes has emerged as a significant research area, supported by extensive benchmark datasets Kiela et al. (2019) ; Pramanick et al. (2021a) ; Lin et al. (2024) and community initiatives such as Facebook’s Hateful Memes Challenge Kiela et al. (2020) . Research in this domain has evolved along several trajectories. Early approaches employed two-stream architectures that separately encode textual and visual features before applying attention mechanisms and multimodal fusion techniques for classification Kiela et al. (2019) ; Suryawanshi et al. (2020) ; Pramanick et al. (2021b) . A parallel line of work has focused on fine-tuning pre-trained multimodal models specifically for harmful content detection