Multilingual Multi-Label Emotion Classification at Scale with Synthetic Data
Vadim Borisov · 2026
Read Multilingual Multi-Label Emotion Classification at Scale with Synthetic Data by Vadim Borisov (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-12633.txt.
Introduction
Vadim Borisov tackles multilingual emotion classification with synthetic data. Existing corpora are mostly English, single-label, and narrow in language coverage. His corpus holds over 1 million multi-label samples across 23 languages. Eleven emotion categories are covered, with culturally adapted generation and quality filtering. Six transformer encoders are trained and compared under identical conditions. The largest, XLM-R-Large, reaches 0.868 F1-micro on in-domain tests. Zero-shot tests on human-annotated English, Arabic, and Spanish data follow. The large model matches or beats English-only specialists while covering all 23 languages.
Catalog record
| CREATOR | Vadim Borisov |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 22234 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.12633
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Multilingual Multi-Label Emotion Classification at Scale with Synthetic Data Vadim Borisov Abstract Emotion classification in multilingual settings remains constrained by the scarcity of annotated data: existing corpora are predominantly English, single-label, and cover few languages. We address this gap by constructing a large-scale synthetic training corpus of over 1M multi-label samples (50k per language) across 23 languages: Arabic, Bengali, Dutch, English, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Mandarin, Polish, Portuguese, Punjabi, Russian, Spanish, Swahili, Tamil, Turkish, Ukrainian, Urdu, and Vietnamese, covering 11 emotion categories using culturally-adapted generation and programmatic quality filtering. We train and compare six multilingual transformer encoders, from DistilBERT (135M parameters) to XLM-R-Large (560M parameters), under identical conditions. On our in-domain test set, XLM-R-Large achieves 0.868 F1-micro and 0.987 AUC-micro. To validate against human-annotated data, we evaluate all models zero-shot on GoEmotions (English) and SemEval-2018 Task 1 E-c (English, Arabic, Spanish). On threshold-free ranking metrics, XLM-R-Large matches or exceeds English-only specialist models tying on AP-micro (0.636) and LRAP (0.804) while surpassing on AUC-micro (0.810 vs. 0.787) while natively supporting all 23 languages. The best base-sized model is publicly available. 1 1 1 https://huggingface.co/tabularisai/multilingual-emotion-classification 1 Introduction Emotion classification is among the most studied tasks in understanding natural languages, with direct applications in mental health monitoring, customer feedback analysis, and conversational artificial intelligence (AI) [ 15 , 18 ] . However, the vast majority of research targets English [ 6 , 17 ] , and existing multilingual resources when they exist at all, typically cover only a handful of languages with single-label annotation. This leaves a significant gap for real-world systems that must operate across languages and handle the inherent co-occurrence of emotions in natural text. Building multilingual, multi-label emotion classifiers faces two core challenges: Annotation scarcity. Labelling emotion data requires native-speaker fluency and cultural competence. Scaling to dozens of typologically diverse languages is prohibitively expensive, which is why most corpora remain English-only [ 6 ] or cover at most three languages [ 17 ] . Cultural variation. Emotions are not expressed uniformly across languages. Idiomatic phrases, social display norms, and pragmatic conventions differ substantially, a naïve translation-based approach to corpus construction fails to capture these differences [ 14 ] . We address both challenges by constructing a large-scale synthetic corpus using culturally-adapted generation. For each of 23 languages spanning 9 language families and 9 scripts, we generate 50k multi-label examples with language-specific prompting and programmatic quality filtering, yielding a training set of over 1M samples. We train six multilingual transformer encoders under identical conditions and evaluate them zero-shot on GoEmotions [ 6 ] and SemEval-2018 E-c [ 17 ] . Our contributions are: • Large-scale corpus. A culturally-adapted synthetic dataset of over 1M multi-label samples across 23 languages and 11 emotion classes. • Encoder comparison. A controlled comparison of six multilingual transformers establishing compute–quality trade-offs. • Cross-benchmark transfer. Models trained entirely on synthetic data match English-only specialists on threshold-free ranking metrics—while natively supporting 23 languages. • Public release. The best base-sized model is publicly available. 2 Dataset Construction This section describes the construction of our synthetic multilingual emotion corpus. The pipeline consists of four stages: (1) definition of the emotion taxonomy, (2) language selection, (3) culturally-adapted generation with quality filtering, and (4) corpus assembly and splitting. We describe each component below. 2.1 Emotion Taxonomy We define 11 emotion categories: anger , contempt , disgust , fear , frustration , gratitude , joy , love , neutral , sadness , and surprise . This taxonomy extends Ekman’s six basic emotions [ 8 ] with finer-grained social emotions ( contempt , frustration , gratitude , love ) and a neutral class. Compared to GoEmotions [ 6 ] , which defines 28 fine-grained English-only classes, and SemEval-2018 E-c [ 17 ] , which uses 11 classes across 3 languages, our taxonomy is designed to balance granularity with cross-lingual applicability. Importantly, we adopt a multi-label formulation: each sample may carry one or more emotion labels simultaneously, reflecting the well-documented co-occurrence of emotions in natural text [ 17 ] . 2.2 Language Selection The corpus covers 23 languages: Arabic, Bengali, Dutch, English, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Mandarin, Polish, Portuguese, Punjabi, Russian, Spanish, Swahili, Tamil, Turkish, Ukrainian, Urdu, and Vietnamese. These were chosen to maximise typological and geographic diversity, spanning 9 language families, 8 scripts, and all inhabited continents. The selection includes high-resource languages (English, Mandarin), medium-resource languages (Turkish, Vietnamese), and lower-resource languages (Swahili, Punjabi) to ensure broad coverage. 2.3 Culturally-Adapted Generation For each language, we generate 50k samples using a pipeline designed to produce culturally authentic emotional text rather than translated English. Cultural tailoring. Generation prompts are constructed per language to elicit culturally appropriate scenarios, social contexts, and idiomatic expressions. For instance, prompts for Japanese encourage references to hierarchical social obligations, while prompts for Spanish may elicit familial contexts. Emoji usage is included where culturally appropriate but is not a primary focus of the generation strategy. Script diversity. For languages where Latin-script writing is widespread in informal digital communication notably Hindi, Bengali, Tamil, Punjabi, Urdu, and Arabic—at least 10% of the generated samples are provided in romanised form. This reflects the common real-world practice of writing these languages in Latin characters on social media and in messaging apps [ 9 ] . Multi-label construction. Each sample is annotated with one or more emotion labels. The resulting label cardinality is 1.65 labels per sample on average: approximately 50% of samples carry a single label, 35% carry two labels, and 15% carry three. No empty-label rows exist in the corpus. Quality filtering. We apply programmatic checks to remove low-quality or mislabelled samples from our synthetic datasets following best practices. These include lexical diversity thresholds to discard near-duplicate or degenerate outputs, label text consistency verification using auxiliary classifiers, and multi-prompt generation strategies to increase stylistic and topical variety across the corpus. 2.4 Corpus Statistics The final corpus contains over 1.15M training samples (50k per language), with 500 validation and 500 test rows per language (11,500 each). Splits are stratified by language. The median text length is 209 characters (p95 \approx 650 characters); we truncate inputs to 192 tokens during training. Table 1 shows the class distribution. Frequencies range from sadness (19.0%) to neutral (8.0%), an imbalance ratio of approximately 2.4\times —mild enough to not require class re-weighting or oversampling. Table 1 : Emotion class distribution in the training set. Emotion Count Share (%) sadness 111,059 19.0 anger 107,058 18.3 frustration 104,456 17.9 surprise 99,942 17.1 disgust 92,825 15.9 love 84,870 14.5 fear 80,464 13.8 contempt 80,447 13.8 gratitude 78,812 13.5 joy 78,481 13.4 neutral 46,728 8.0 3 Experimental Setup 3.1 Models We train six configurations of multilingual transformer encoders, summarised in Table 2 . Table 2 : Models trained. All share the same head, loss, and hyperparameters. Model Params Epochs Train (min) DistilBERT-multi [ 19 ] 135M 3 14.0 mBERT [ 7 ] 178M 3 27.4 XLM-R-Base [ 4 ] 278M 3 31.5 Twitter-XLM-R [ 1 ] 278M 3 69.8 mDeBERTa-v3 [ 11 ] 278M 3 69.9 XLM-R-Large [ 4 ] 560M 3 130.8 3.2 Training Details All models are trained with binary cross-entropy loss ( BCEWithLogitsLoss ) using AdamW optimisation [ 16 ] , learning rate 2\times 10^{-5} , cosine schedule with