GSE: Evaluating Sticker Visual Semantic Similarity via a General Sticker Encoder
Heng Er Metilda Chee, Jiayin Wang, Zhiqiang Guo, Weizhi Ma, Min Zhang · 2025
Read GSE: Evaluating Sticker Visual Semantic Similarity via a General Sticker Encoder by Heng Er Metilda Chee, Jiayin Wang, Zhiqiang Guo, Weizhi Ma, Min Zhang (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2511-04977.txt.
Introduction
Heng Er Metilda Chee and co-authors define a new task: judging how close two stickers are in meaning. Stickers are a common form of visual chat, but their signs are varied and sly, so machines find them hard to compare.
The team built Triple-S, the first test set for this task, with 905 human-marked pairs of like and unlike stickers. Old vision and mixed models did poorly on it. Their new General Sticker Encoder learns strong sticker vectors and leads on new stickers, plus on tasks like mood sorting and sticker search.
Catalog record
| CREATOR | Heng Er Metilda Chee, Jiayin Wang, Zhiqiang Guo, Weizhi Ma, Min Zhang |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 30 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2511-04977
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
GSE: Evaluating Sticker Visual Semantic Similarity via a General Sticker Encoder Heng Er Metilda Chee, Jiayin Wang, Zhiqiang Guo, Weizhi Ma, Min Zhang Abstract Stickers have become a popular form of visual communication, yet understanding their semantic relationships remains challenging due to their highly diverse and symbolic content. In this work, we formally define the Sticker Semantic Similarity task and introduce Triple-S , the first benchmark for this task, consisting of 905 human-annotated positive and negative sticker pairs. Through extensive evaluation, we show that existing pretrained vision and multimodal models struggle to capture nuanced sticker semantics. To address this, we propose the General Sticker Encoder (GSE) , a lightweight and versatile model that learns robust sticker embeddings using both Triple-S and additional datasets. GSE achieves superior performance on unseen stickers, and demonstrates strong results on downstream tasks such as emotion classification and sticker-to-sticker retrieval. By releasing both Triple-S and GSE, we provide standardized evaluation tools and robust embeddings, enabling future research in sticker understanding, retrieval, and multimodal content generation. The Triple-S benchmark and GSE have been publicly released and are available here 1 1 1 https://anonymous.4open.science/r/triple-s-6E65/ . 1 Introduction Stickers are ubiquitous in online communication, serving as a rich and expressive medium for conveying emotions and ideas Chee et al. 2025b . Beyond casual use, recent research has begun to explore personalized sticker generation Xu et al. 2024 ; Shen et al. 2024 with semantic preservation Xu et al. 2025 . However, progress in this area is limited by the absence of a standardized sticker semantic similarity evaluation benchmark. Current works resort to general-purpose vision encoders such as CLIP Radford et al. 2021 , DINOv2 Oquab et al. 2023 , or ViT Dosovitskiy et al. 2021 to approximate sticker semantic similarity. Yet, the applicability of these encoders to sticker semantics has never been systematically validated, leaving a critical gap in evaluation. Figure 1: Examples of semantic pairings. The top row shows positive pairs, where both stickers convey similar emotions or actions. The bottom row shows negative pairs, where stickers differ in emotion, expression, or context despite visual or thematic similarities. Developing a benchmark for sticker semantic similarity presents us with a main challenge. Benchmark dataset construction is inherently difficult. While coarse sticker classification datasets exist, a high-quality evaluation requires carefully curated positive and hard negative pairs. Human annotation of sticker semantics is both subjective and labor-intensive, making the creation of such a dataset tedious and costly. In addition to existing baselines, providing a feasible evaluation method pose challenges. Though, general purpose vision-language models are capable of natural language, possibly sticker semantics Achiam et al. 2024 , they are computationally intensive and impractical for large-scale or rapid evaluation. This motivates the need for a lightweight, efficient, and versatile encoder tailored to the unique properties of stickers for the sticker semantic similarity task. We formally define the sticker semantic similarity task, which evaluates whether two stickers convey similar meaning, Figure 1 . To standardize evaluation, we introduce Triple-S , the first benchmark for this task, consisting of 905 human-annotated sticker pairs labeled as positive or negative with agreement from annotators of different backgrounds. Then, we benchmark several general-purpose image encoders such as CLIP, DINOv2, and ViT on Triple-S, highlighting their limitations. Finally, we present GSE (General Sticker Encoder) , a lightweight and effective baseline that surpasses existing methods on unseen data and demonstrates transfer to downstream tasks such as emotion classification and top- k image-to-image retrieval. We summarize our contributions: First, we propose and formally define the Sticker Semantic Similarity task. Second, we introduce Triple-S , the first benchmark for the visual sticker semantic similarity task, consisting of a curated dataset of 905 annotated semantically positive and negative sticker pairs. Third, we benchmark common strong pretrained vision encoders on Triple-S benchmark, revealing their limitations in capturing sticker semantics. Finally, we present GSE (General Sticker Encoder) , a lightweight and versatile sticker encoder that provides a strong baseline for the 3S task and generalizes effectively to other related tasks. 2 Related Work 2.1 Sticker Semantic Similarity Evaluation Image Encoders Recent vision-language models, such as CLIP Radford et al. 2021 , ViT Dosovitskiy et al. 2021 , and DINOv2 Oquab et al. 2023 , provide powerful image embeddings for capturing semantic content. These models have been successfully applied to various tasks, including image retrieval and similarity evaluation. However, they are not ideally suited for stickers. Stickers often convey meaning through stylized, exaggerated, or culturally specific visual cues that general-purpose models fail to capture. While StickerCLIP Zhao et al. 2023 attempts to address this, it is closed-source and not publicly accessible, limiting its use for research and evaluation. Vision-Language Model Recent large vision-language models (VLMs), including OpenAI’s ChatGPT series Achiam et al. 2024 and ChatGLM Zeng et al. 2023 variants with visual input, have achieved remarkable progress in understanding complex image-text relationships. They enable zero-shot classification, retrieval, and semantic similarity evaluation by jointly modeling visual and textual information. Despite their power, these models are not ideally suited for stickers. They can hallucinate or misinterpret stylized, exaggerated, or culturally specific visual cues, which are common in sticker designs. Moreover, they are resource-intensive and computationally expensive, making them impractical for large-scale or real-time evaluation tasks where lightweight and efficient metrics are required. 2.2 Sticker Datasets and Benchmarks With the rise of instant messaging, sticker datasets have grown significantly, but given the diversity and complexity of sticker tasks, existing datasets still represent only the tip of the iceberg. We broadly categorize these datasets into five groups: dialogues, sentiment, captions, multimedia, and retrieval. Dialogue datasets typically contain truncated conversations with stickers Gao et al. 2021 ; Gao et al. 2020 ; Fei et al. 2021 ; Liang et al. 2024 ; Shi & Kong 2024 ; Zhang 2024 ; Wang et al. 2025 . Notably, U-Sticker Chee et al. 2025a provides continuous user information, temporal context, and domain-specific sticker conversations. While some datasets are publicly available, others remain private or inaccessible. Sentiment datasets focus on predicting the sentiment conveyed by stickers, either through binary labels Ge et al. 2022 or fine-grained classifications Liu et al. 2022 . Certain datasets, such as Sticker820K Zhao et al. 2023 , also include OCR text or descriptive metadata. Among these, SER30K Liu et al. 2022 is publicly available. Caption datasets contain stickers with textual elements, e.g., TGIF Li et al. 2016 provides GIF captions. Unfortunately, StickerTag Wang et al. 2024b is unavailable for analysis. Multimedia datasets provide raw sticker images for general use or generation tasks. ChineseB2B Zhaoolee 2024 offers a large collection of images, while VSD2M Yuan et al. 2025 includes an extensive repository of sticker animations. Retrieval datasets pair user search queries with corresponding sticker images, capturing underlying sticker usage intent. StickerQueries provides bilingual (English and Chinese) queries curated by native speakers. PerSRV Chee et al. 2024 , another sticker–query pair dataset, is not publicly available. However, as can be seen in Table 1 , none of the existing datasets are suitable for evaluating sticker semantic similarity. Sentiment datasets are too coarse to capture fine-grained semantics, while retrieval-based datasets like StickerQueries are unreliable for distinguishing subtle nuances such as irony, sarcasm, or implied meaning. Consequently, a human-annotated, pairwise dataset remains essential