A Comparative Study on Affective Cues in Text Embeddings Across Psychological Emotion Theories Fabio Ciani, Harald Schweiger 0, Emilia Parada-Cabaleiro, Markus Schedl Abstract Text encoders are known for their utility in natural language processing, as they are able to efficiently compress inputs into dense vectors while preserving semantics. These models have been applied to affective computing, in particular to help with solving sentiment analysis and emotion recognition tasks. Nevertheless, it remains unclear to what extent the latent representations produced by modern text encoders capture well-defined psychological theories of affect. In this work, we investigate the affective capabilities of twelve recently released text encoders by probing their generated embeddings as input features for solving regression and classification tasks across three established emotion frameworks, using both word- and sentence-level data. Additionally, we apply a semantic data-leakage prevention technique to improve robustness in word-level evaluations. Our main findings show that the latent manifolds of the latest instruction-aware open-weight encoders enclose an equal or even a larger amount of affective information in comparison with proprietary counterparts when evaluated at word level. In contrast, embeddings of task-tuned and proprietary encoders reach the highest scores on sentence-level affective classification. Furthermore, a qualitative analysis of latent representations and their encoded affective cues is provided. 1 Introduction The advent of text encoders has transformed the way of converting textual content into numerical representations, known as embeddings, which now serve as core components for a variety of tasks, e.g., semantic similarity, retrieval, and reranking [ 8 ] . They have also proven to be valuable for text-based sentiment analysis, e.g., in emotion classification [ 11 ] or valence-arousal regression [ 20 ] . To solve these tasks, models such as BERT [ 12 , 42 ] have been tested under various configurations, mostly involving end-to-end fine-tuning. Since then, the performance of text encoders has been progressively improved by different techniques, including instruction-based queries and advanced training schemes [ 58 ] . Despite relevant prior works, the usage of text encoders as zero-shot feature extractors for emotion recognition tasks remains underexplored, especially with respect to the latest state-of-the-art and instruction-aware models. Moreover, it is uncertain whether the latent manifolds induced by these models enclose established emotion frameworks from psychology through a sufficient encoding of affective information. To fill this gap, our analysis compares recently released text encoders, without prior fine-tuning, across different emotion theories, i.e., the Pleasure-Arousal-Dominance model by Mehrabian and Russell [ 27 ] , the model of emotions by Plutchik [ 40 ] , and the “big six” by Ekman [ 13 ] . We use two structured lexica and one sentence-level dataset, i.e., NRC-VAD [ 33 ] , NRC-EIL [ 32 ] , and GoEmotions [ 11 ] , which respectively correspond to each of the emotion theories, together with a novel technique to limit leakage between splits. Accordingly, embeddings are computed and frozen to be exploited as input features to four downstream predictors and subsequently evaluated with respect to their affective cues. The quantitative results, together with a qualitative visual analysis, address the following research questions. RQ1 To what extent do latent manifolds from text encoders enclose emotional cues? RQ2 Are instruction-aware text encoders superior to task-tuned models or ones without explicit prompt support for generating optimized embeddings? RQ3 Are proprietary models better than open-weight ones? RQ4 Does model performance vary depending on the chosen emotion framework and downstream predictor? The paper proceeds as follows. In Section 2 , a background on emotion theories and affective language processing is presented. Section 3 describes the experimental setup, including datasets, encoders, and predictors. Finally, Section 4 reports the results, while Section 5 concludes the article. 1 1 1 Our code can be publicly accessed at the repository at the following hyperlink: https://github.com/hcai-mms/affective˙embeddings . 2 Related works 2.1 Emotion theories Consolidated research in psychology has led to a variety of frameworks explaining human emotions from both taxonomic and perceptual standpoints. One of the first categorical models was proposed by Ekman, in which the so called “big six” basic emotions ( anger , disgust , fear , happiness , sadness , and surprise ) were identified from facial expressions and considered biologically encoded and cross-cultural [ 13 ] . Another notable theory is the circumplex model by Russell, which represents affects in a two-dimensional space, where the horizontal and vertical axes respectively measure valence and arousal. While valence captures the pleasure ranging from negative to positive, arousal reflects the energy spanning from low to high [ 43 ] . This was preceded by the PAD (Pleasure-Arousal-Dominance) emotion model by Mehrabian, where a third bipolar dimension to quantify the evoked control or submissiveness was also included [ 27 ] . An additional framework, also relevant to affective computing, is the one by Plutchik, who designed a hybrid categorical-dimensional model with a resemblance to Russell’s theory, in which spatial proximity links to affect similarity. Eight primary emotions ( joy , trust , fear , surprise , sadness , disgust , anger , and anticipation ) are arranged in concentric circles corresponding to different levels of intensity, i.e., as a cone subdivided in sectors, with the possibility to mix adjacent and opposite emotions to form combined mood dyads [ 40 ] . 2.2 Affective language processing Early techniques to extract emotions from textual content and drawing from the distributional hypothesis in linguistics [ 19 , 17 ] included latent semantic analysis [ 3 ] , a matrix factorization procedure to learn compressed representations, that later evolved into popular self-supervised word embeddings [ 28 , 29 , 39 , 4 ] . Word vectors were retrained from scratch incorporating supervised affective contexts into the objective function [ 49 ] . Faruqui et al. [ 15 ] and Mrksic et al. [ 34 ] devised a method for adjusting pre-trained word embeddings with respect to lexical relationships and constraints, which was adopted by Yu et al. [ 55 ] and Seyeditabari et al. [ 44 ] on affective datasets to mitigate reported issues in vector similarity and arithmetic associated with general-purpose distributional embeddings [ 45 ] . Notable extensions built upon the Transformer architecture have been presented both at word level [ 9 , 10 ] and at sentence level [ 46 ] . The attention mechanism has also been adapted to enrich learnt representations by combining vectors and data from a knowledge base [ 48 ] . More broadly, it has been questioned whether language models (LMs) can effectively understand the multifaceted nature of emotions. Lee et al. [ 22 ] isolated low-level subcomponents focused on handling patterns deriving from specific affects, while Reichman et al. [ 41 ] continued the analysis underlining the presence of a complex redundancy scheme implemented by sets of specialized neurons and connections within the neural architecture. At a higher level, the neuropsychology of LMs has been studied by assessing whether their internal representations can be refined to align with established emotion theories [ 24 ] . It has also been observed that larger foundational models tend to exhibit emotional intelligence more accurately than smaller counterparts [ 53 ] and build increasingly detailed hierarchical taxonomies to organize emotions [ 57 ] . Lastly, connecting the affective expression in an emotion model to the definition in another theoretical framework has been achieved through annotated textual content, i.e., data with a series of assigned labels or numerical quantities, either directly bridging the categorical and dimensional families [ 38 ] or upon learning an agnostic intermediate representation space for conversions [ 5 ] . 3 Methodology To test the ability of text encoders to capture emotional cues, we performed evaluations on three corpora grounded in different emotion frameworks from psychology (cf. Section 3.1 ). We favored structured lexica, i.e., NRC-VAD [ 31 , 33 ] and NRC-EIL [ 32 ] to reduce ambiguity in syntax and better match the conditions under which the corresponding theories have been studied. Besides, GoEmotions [ 11 ] provides a more conservative perspective, as it contains full sentences instead of single- and multi-word samples. Figure 1: Pipeline demonstrating the fitting and evaluating procedure. All embeddings are calculated once and frozen for each dataset (blue section). For simplicity, the remaining control flow is depicted for one experiment only (yellow and purple sections), i.e., the regression task of NRC-VAD in combination with semantic leakage prevention and using KaLM v2 as text encoder. We evaluated twelve text encoders (cf. Section 3.2 ) using a two-step procedure. First, embeddings for all words and sentences in the datasets were computed and frozen. Second, a collection of downstream predictors was trained and its hyperparameters were tuned, with the generated latent features as input, to assess the predictive performance on the corresponding regression and classification tasks (cf. Figure 1 ). We selected four predictors owning distinct characteristics to map embeddings and emotions, allowing to test whether emotional cues are accessible linearly or via nonlinear transformations (cf. Section 3.3 ). To better measure the true generalization capabilities on the structured lexica, we applied two techniques to prevent morphological and semantic leakage across data splits, reducing evaluation biases on predictive models that rely on closely related lexical items to build their solutions. The results presented in the main body of this paper focus on the semantics-aware splitting strategy only, as it is inherently less biased. More comprehensive summaries, including those obtained with the morphology-aware approach, are reported in the Appendix (cf. Appendices 0.A and 0.D ). 3.1 Emotion datasets All corpora are freely accessible and in English. They present varying input granularity (single-word, multi-word, and sentence-level) as well as output formats (discrete and continuous). NRC-VAD [ 31 , 33 ] consists of around 55k single- and multi-word samples, annotated via crowdsourcing with real-valued valence, arousal, and dominance scores in the interval [-1,1] , following Mehrabian’s theory [ 27 ] . NRC-EIL [ 32 ] contains almost 6k single words with emotion intensities in the interval [0,1] , in line with Plutchik’s model [ 40 ] . The properties of the collection would make it suitable as both a regression and a classification task, since 62.4% of the entries are assigned a single emotion, 18.6% have a pair of nonzero intensities, and the remaining terms are characterized by three or more emotions. In our experiments, we focused on regression of the real-valued intensities. GoEmotions [ 11 ] comprises over 54k comments crawled from Reddit, paired with 27 labels and filtered according to inter-rater agreement. The dataset provides official documentation to map these categories to a subset of labels consistent with Ekman’s framework [ 13 ] . For our evaluations, we took the labeled sentences in combination with this projection to obtain a multi-label classification dataset with 7 classes, six for Ekman’s emotions and one for the neutral category, where 91.2% of the samples have one category and 8.8% at least two. Splitting strategy To train the predictive models, each dataset is partitioned into five folds for cross-validation and one holdout test set for final evaluation in a 80%/20% proportion. Given that GoEmotions has a predefined train-dev-test split, we combined the training and development splits and applied stratified sampling to balance the labels in the folds, while the test split was left unchanged to enable comparisons with the evaluations of the original work. For the two lexicon-based datasets, i.e., NRC-VAD and NRC-EIL, we used a novel technique for semantic leakage prevention, as detailed in the next paragraph. Leakage prevention Random train-test splits can overestimate generalization due to morphological or semantic leakage. For instance, NRC-VAD includes multiple inflected and derived forms of the same lexical root (e.g., pleasure , pleasures , pleasurable ) and semantically related terms (e.g., calm , chill , peaceful ). Text encoders tend to map these elements into nearby regions of the embedding space, which downstream predictive models could exploit by relying on nearest-neighbor similarity rather than learning to genuinely generalize. To prevent this, we created a graph representation of the dataset lexemes and clustered them with the Leiden algorithm [ 50 ] . Nodes correspond to lexemes, with edges that are present if exceeding a threshold and weighted by the Wu–Palmer semantic similarity [ 54 ] computed between the corresponding term synsets in WordNet [ 30 , 16 ] . Lexemes not covered by WordNet are excluded from the dataset. To assign the clusters to the cross-validation folds and the holdout set, we applied a greedy balancing algorithm that optimizes two criteria: (i) the split sizes, i.e., 16% for each fold and 20% for the test set; and (ii) the preservation of the mean of the distribution for each emotion dimension across splits. The procedure is executed multiple times with different initialization seeds and the best balanced split is kept. 3.2 Text encoders For a comprehensive analysis, we handpicked six open-weight and three proprietary models from the top entries of the Massive Text Embedding Benchmark (MTEB) 2 2 2 Online at https://huggingface.co/spaces/mteb/leaderboard . leaderboard curated by Hugging Face [ 35 , 14 ] . When multiple models shared the same base architecture, including updated versions, we selected the encoder with the best average score. Of the six considered open models, five of them ( KaLM Embedding v2 , Qwen3 Embedding 8B , Linq Embed Mistral , LLaMA Embed Nemotron 8B , and Multilingual E5 Large Instruct ) are instruction-aware, i.e., they were trained to generate embeddings optimized for user-defined specifications included as additional input [ 47 ] . Differently, EmbeddingGemma is the only one which is task-tuned, i.e., a relaxed instruction-aware model with predefined configurations for a set of use cases (e.g., classification or semantic text similarity). These were set up following the recommended prompts in their model cards together with the task to be solved, i.e., regression and classification, to detail the desired feature extraction (cf. Appendix 0.B ). As for proprietary models, we chose OpenAI Text Embedding v3 Large , Gemini Embedding 001 , and Voyage v3 Large , none of which is instruction-aware. To further diversify the model pool, we also included two very recent encoders, i.e., Jina Embeddings v4 and Nomic Embed v2 , whose earlier releases demonstrated good performance on the MTEB benchmark, but which have been assessed on a limited subset of the list of tasks in MTEB in their current versions. Additionally, Sentence T5 XXL , a popular sentence embedding model, is included to serve as a representative for text encoders without explicit support for custom prompts or task instructions. In total, twelve models are considered, spanning a wide range of parameter sizes, output dimensionalities, and training corpora, both English-only and multilingual (cf. Table 1 ). Table 1: Description of the analyzed text encoders. Licenses are subdivided into open-weight ( ) and proprietary (​ ​). The number of parameters for downloadable models and the dimensionality of the latent features are specified by p and d , respectively. Each entry is tagged with its type between no prompt support (▲), task-tuned (◼), and instruction-aware (⚫). All encoders are multilingual, unless their training corpora are mainly in English ( \dagger ). Name p d Access Reference Sentence T5 XXL ▲ \dagger 4.8B 768 [ 36 ] EmbeddingGemma ◼ 300M 768 [ 51 ] Nomic Embed v2 ◼ 305M 768 [ 37 ] Multilingual E5 Large Instruct ⚫ 560M 1024 [ 52 ] Jina Embeddings v4 ◼ 3.8B 2048 [ 18 ] KaLM Embedding v2 ⚫ 12B 3840 [ 58 ] Linq Embed Mistral ⚫ \dagger 7B 4096 [ 21 ] LLaMA Embed Nemotron 8B ⚫ 8B 4096 [ 2 ] Qwen3 Embedding 8B ⚫ 8B 4096 [ 56 ] Voyage v3 Large ▲, a a a https://blog.voyageai.com/2025/01/07/voyage-3-large . 2048 OpenAI Text Embedding v3 Large ▲, b b b https://platform.openai.com/docs/models/text-embedding-3-large . 3072 Gemini Embedding 001 ◼ 3072 [ 23 ] 3.3 Predictive models We employed four predictors, i.e., linear and logistic regression with elastic net regularization (LR), k -nearest neighbors ( k -NN), XGBoost (XGB), and multilayer perceptron (MLP), to leverage text embeddings for downstream emotion prediction tasks. On the regression datasets, i.e., NRC-VAD and NRC-EIL, all predictive models were trained to minimize the mean squared error (MSE). For multi-label classification, i.e., GoEmotions, performance was optimized maximizing the macro-averaged F_{1} , using a fixed decision threshold of 0.5 . Since LR and XGB do not natively support multi-label classification, one-vs-the-rest strategy was applied. The hyperparameters of the predictors were independently tuned for each dataset and generating encoder over multiple optimization trials with respect to R^{2} for regression and macro F_{1} for classification. More details on how hyperparameter tuning was carried out can be found in Appendix 0.C . 4 Experimental results 4.1 Quantitative regression analysis We evaluated the performance on the holdout test sets using three regression metrics, i.e., MSE, R^{2} , and concordance correlation coefficient \rho_{c}=\frac{2\rho\sigma_{x}\sigma_{y}}{\sigma_{x}^{2}+\sigma_{y}^{2}+(\mu_{x}-\mu_{y})^{2}}. (1) In Equation 1 , x and y denote true and predicted values respectively, whereas \rho equals to Pearson’s correlation coefficient. \rho_{c} was chosen because, as defined in its formula, it captures both correlation and agreement, penalizing scale mismatches [ 25 ] . To further support our results, the outcomes of paired difference tests, estimated via bootstrapping, are provided for each metric and encoder. The null hypothesis claims that, using a given predictor as backend, the candidate encoder leads to better performance than the best encoder. Tables 2 and 3 sum up the inference performance, with the best results in bold and highlighting the cases without statistical evidence ( p>0.05 ) with a double underline and statistically significant p -values falling within the interval [0.005,0.05] with a single underline. The absence of an underlined value refers to p -values below 0.005 . Table 2: Regression metrics at test time for NRC-VAD, sorted by R^{2} score of the MLP model in descending order. LR \bm{k} -NN XGB MLP MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 \mathbf{.066} \mathbf{.637} \mathbf{.790} \mathbf{.075} \mathbf{.591} \underline{\underline{.746}} \mathbf{.068} \mathbf{.630} \mathbf{.777} \mathbf{.059} \mathbf{.677} \mathbf{.811} Linq Mistral .067 .631 .786 \underline{.076} \underline{.586} \mathbf{.746} .069 .622 \underline{.773} .061 .667 .806 OpenAI Text v3 L .073 .602 .761 .079 .572 .704 .077 .582 .734 .062 .659 .797 Qwen3 8B .069 .624 .780 .077 .578 .733 .072 .607 .761 .063 .657 .798 LLaMA Nemotron 8B .075 .589 .745 .099 .463 .607 .079 .570 .729 .064 .653 .795 Gemini 001 .077 .578 .748 .080 .564 .710 .081 .560 .717 .066 .638 .782 ST5 XXL .080 .564 .735 .077 .581 .722 .080 .564 .715 .068 .631 .777 EmbeddingGemma .082 .553 .725 .085 .536 .697 .082 .553 .713 .073 .603 .757 Voyage v3 L .082 .556 .728 .098 .472 .624 .086 .534 .699 .073 .601 .755 Multilang E5 L Ins .084 .542 .717 .085 .537 .704 .083 .550 .713 .077 .582 .741 Jina v4 .088 .520 .694 .096 .479 .636 .091 .506 .671 .080 .563 .726 Nomic v2 .108 .413 .603 .112 .396 .540 .110 .403 .567 .097 .476 .647 NRC-VAD KaLM v2 achieves the highest scores across all predictive models with one exception, i.e., \rho_{c} for k -NN, where Linq Mistral is slightly more performant, though not in a statistically significant way (cf. double-underlined value in the first row of Table 2 ). OpenAI Text v3 L ranks third with the MLP backend, whereas other instruction-aware models, i.e., Qwen3 8B and LLaMA Nemotron 8B , show comparable performance, followed by the task-tuned Gemini 001 . In relation to RQ 3 , these insights indicate that some open-weight text encoders can significantly outperform proprietary alternatives (cf. no underlined scores for OpenAI Text v3 L ), likely due to instruction-awareness. This also provides initial evidence for RQ 2 . Interestingly, the remaining instructional Multilang E5 L Ins occupies one of the last positions. The maximum R^{2} score of .677 and correlation \rho of .811 indicate that the text encoders are capable of capturing affective cues, thereby addressing RQ 1 . In general, with the MLP predictor, all encoders except for Nomic v2 reach a R^{2}>.55 and \rho>.7 , hinting their ability to enclose affective signals. Concerning RQ 4 , the extraction of this affective information is more effective when using the MLP backend, with LR as runner-up. Table 3: Regression metrics at test time for NRC-EIL, sorted by R^{2} score of the MLP model in descending order. LR \bm{k} -NN XGB MLP MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 \underline{\underline{.023}} \underline{\underline{.516}} \underline{\underline{.705}} \mathbf{.024} \underline{\underline{.500}} \underline{.685} \underline{\underline{.023}} \underline{\underline{.515}} \underline{\underline{.697}} \mathbf{.022} \mathbf{.540} \mathbf{.730} Linq Mistral \mathbf{.023} \mathbf{.519} \mathbf{.710} \underline{\underline{.024}} \mathbf{.500} \mathbf{.695} \mathbf{.023} \mathbf{.517} \mathbf{.704} \underline{\underline{.022}} \underline{.528} \underline{\underline{.729}} Qwen3 8B \underline{\underline{.023}} \underline{\underline{.517}} \underline{\underline{.704}} \underline{\underline{.024}} \underline{\underline{.491}} \underline{.683} \underline{\underline{.023}} \underline{\underline{.513}} \underline{\underline{.698}} \underline{.022} \underline{.524} \underline{\underline{.725}} EmbeddingGemma .024 .491 .683 \underline{.024} \underline{.486} .672 .024 .485 .667 \underline{.022} \underline{.524} .717 LLaMA Nemotron 8B .025 .475 .673 .026 .446 .647 .026 .456 .637 \underline{.023} \underline{.523} \underline{.720} OpenAI Text v3 L .025 .471 .658 .026 .454 .631 .026 .445 .616 .023 .516 .704 ST5 XXL .025 .471 .659 .025 .474 .659 .026 .449 .626 .023 .513 .712 Multilang E5 L Ins .025 .472 .671 .025 .472 .665 .024 .489 .676 .023 .508 .709 Gemini 001 .026 .460 .654 .026 .458 .648 .027 .436 .613 .024 .499 .699 Voyage v3 L .026 .446 .645 .029 .401 .578 .027 .427 .605 .024 .488 .688 Jina v4 .028 .408 .600 .030 .372 .558 .029 .384 .572 .027 .432 .631 Nomic v2 .032 .336 .529 .034 .293 .457 .033 .313 .479 .030 .369 .578 NRC-EIL Similar patterns emerge with respect to NRC-VAD. KaLM v2 and Linq Mistral consistently occupy the top positions. KaLM v2 benefits again from the MLP predictor (cf. top-right corner of Table 3 ), while Linq Mistral performs better across almost all other predictive models (cf. bold scores on second row). In contrast to the ranking on NRC-VAD, Qwen3 8B and EmbeddingGemma rise to the third and the fourth places, while OpenAI Text v3 L drops to the sixth position. This addresses RQ 4 . Performance differences between Linq Mistral and KaLM v2 , Qwen3 8B , and EmbeddingGemma have p -values in [0.005,0.05] , indicating that the ranking is not statistically decisive (cf. single-underlined values in the corresponding rows). This suggests that task-tuned encoders such as EmbeddingGemma are competitive with respect to instruction-aware alternatives, providing further insight into RQ 2 . With regard to RQ 3 , four instruction-aware models and one task-tuned encoder outperform all proprietary models (cf. top-5 ranking and rows below), hence the support for generating optimized embeddings via instructions appears to be generally beneficial. As for RQ 1 , the highest R^{2} reaches a score of .540 and \rho a correlation of .730 , showing a moderate encoding of affective information for the top performing embedding model. Recent open encoders, i.e., Jina v4 and Nomic v2 , seem to weakly enclose affective cues, as hinted by their low R^{2} scores of .432 and .369 (cf. last rows). Table 4: Summary of the macro-averaged classification metrics at test time for GoEmotions, sorted by F_{1} score of the MLP model in descending order. LR \bm{k} -NN XGB MLP p r F_{1} p r F_{1} p r F_{1} p r F_{1} Gemini 001 \mathbf{.716} \mathbf{.517} \mathbf{.594} \mathbf{.624} \mathbf{.543} \mathbf{.575} \underline{\underline{.674}} \mathbf{.469} \mathbf{.546} \mathbf{.713} \mathbf{.529} \mathbf{.600} EmbeddingGemma \underline{.688} .493 .568 \underline{.599} .512 .546 \underline{\underline{.678}} \underline{.447} \underline{\underline{.533}} \underline{\underline{.700}} \underline{\underline{.517}} \underline{\underline{.590}} OpenAI Text v3 L \underline{.687} .489 .564 .472 .423 .437 \underline{.645} .397 .483 .683 \underline{\underline{.518}} \underline{.579} Linq Mistral \underline{.685} \underline{.494} .568 .565 .461 .500 \underline{\underline{.666}} .408 .497 \underline{\underline{.700}} .494 .575 Qwen3 8B \underline{\underline{.700}} \underline{.496} \underline{.573} .582 .490 .525 \underline{.651} .425 .507 \underline{\underline{.703}} .498 .574 KaLM v2 \underline{\underline{.705}} \underline{\underline{.515}} \underline{\underline{.588}} \underline{.600} .511 .544 \underline{\underline{.682}} \underline{\underline{.453}} \underline{\underline{.535}} \underline{\underline{.704}} .487 .565 Multilang E5 L Ins .674 .474 .549 .511 .496 .499 \underline{\underline{.674}} .392 .481 .675 .482 .550 LLaMA Nemotron 8B .637 .454 .521 .476 .405 .425 \mathbf{.683} .388 .480 .607 \underline{\underline{.512}} .549 Nomic v2 .669 .428 .514 .476 .410 .429 \underline{.639} .348 .438 .669 .460 .537 Jina v4 .663 .423 .508 .454 .395 .412 .631 .346 .434 .660 .436 .515 Voyage v3 L .635 .420 .498 .442 .382 .399 .568 .327 .406 .621 .447 .513 ST5 XXL .649 .424 .506 .449 .408 .420 .618 .350 .436 .668 .427 .510 4.2 Quantitative classification analysis We used three common measures, i.e., precision, recall, and F_{1} score, aggregating with respect to multiple classes by calculating the macro average of these metrics. Table 4 summarizes the aggregated results. As in Section 4.1 , results are supported by significance checks. In particular, paired permutation tests were applied. GoEmotions Results reveal a substantially different ranking in comparison with the regression analysis. The proprietary Gemini 001 and the open-weight EmbeddingGemma achieve the top places on all predictive backends with their F_{1} scores, followed by OpenAI Text v3 L (cf. top-3 ranking of Table 4 ). Interestingly, KaLM v2 achieves an F_{1} score of .588 with LR, which is the only occurrence where MLP is outperformed (cf. F_{1} score of .565 in the sixth row). This makes KaLM v2 the model with the third best F_{1} , giving new insights on RQ 4 . Two open encoders without explicit prompt support for optimized embeddings, i.e., Voyage v3 L and ST5 XXL , occupy the last positions (cf. last rows). Considering RQ 3 , in contrast to the analyses on NRC-VAD and NRC-EIL, the top ranker is a proprietary text encoder, i.e., Gemini 001 , rather than an open-weight model. However, this superior performance is not statistically significant (cf. underlined F_{1} scores for EmbeddingGemma ). In relation to RQ 2 , this highlights that instruction-aware encoders, as well as models with large parameter size and embedding dimensionality, are not always better than task-tuned or proprietary alternatives, in particular when sentence-level samples are used instead of word-level data. Concerning RQ 1 , a maximum F_{1} of 0.60 is achieved (cf. MLP column in the top-right corner). Instead, the authors of the dataset report a score of 0.64 by fine-tuning BERT [ 11 ] , though without specifying the selected classification threshold. This comparison suggests that exploiting fine-tuning could be advantageous. As for RQ 4 , in terms of F_{1} scores, the MLP backend consistently outperforms all other predictive models, with only one exception (cf. LR backend for KaLM v2 ). LR ranks second, whereas k -NN and XGB exhibit more variable performance depending on the used encoder. These trends are in line with those observed in the experiments on regression. 4.3 Qualitative visual analysis Figure 2: UMAP visualization of the full embeddings with color-coded labels. We report a series of visualizations with the intent to discover whether generated vectors are expressive enough to imply a clustering of similar elements with respect to their affect. For this analysis, we focus on the best performing open-weight representatives for each of the three types of text encoder, i.e., without prompt support, task-tuned, and instruction-aware, as specified in Table 1 . The embeddings calculated on the full datasets were transformed into a 2D representation with UMAP [ 26 ] set up with cosine similarity as metric to compare vectors. Depending on the data format, the output variable information was converted through a color encoding as follows. • 3D emotion points from NRC-VAD were read as RGB triples and their hue value was used as parametrization of a cyclic colormap. Samples with pure valence (R), arousal (G), or dominance (B) signals are equidistant, with respect to the color space, to other pure points. • Entries of NRC-EIL with more than one active affect intensity were filtered out and the remaining elements were transmuted into members of the positive emotions ( joy , trust , anger , and anticipation ) or negative counterparts ( sadness , disgust , fear , and surprise ) following Plutchik’s original statement, where the two groups render the opposite ends of a diverging colormap. • Samples from GoEmotions either having multiple labels or tagged as neutral were dropped and linked to Ekman’s taxonomy through the official dataset lookup table, with each category corresponding to a distinct color. For the sake of readability, Figure 2 is limited to 500 points per dataset, where stratified sampling was applied to NRC-EIL and GoEmotions. In the first row, it can be seen that none of the encoders is able to entail the creation of clusters of consistent entries for NRC-VAD. In fact, an equilateral triangle of uniformly distributed samples, with pure points of a component at its vertices, should ideally form. Among the three affective axes, the one for arousal looks to be the most easily separable when considered alone. Instead, all text encoders can sharply divide the elements of NRC-EIL into two groupings, as evident in the second line of plots. In addition, the instruction-aware model is particularly skilled at avoiding poisoning the cluster of positive samples with intense negative points. As for GoEmotions in the third row, while disgust-, fear-, and sadness-labeled entries tend to gather together, especially in the task-tuned and instruction-aware encoders, elements tagged with the anger, joy, and surprise labels spread out and hardly group. The difficulty might be possibly due to the fact that semantically broader labels are intuitively associated with the tendency to be more frequently selected by a human rater. Since a bigger cardinality for the subset of a category can imply a higher variance in its embedding vectors, samples marked with the most occurring tags have a higher risk of being characterized by inconsistent latent features with respect to a representative of their cluster of belongingness. 5 Conclusions and future works To conclude, our analyses show that affective information is present to a varying degree within twelve text encoders and across three emotion frameworks. Addressing RQ 1 , the highest R^{2} score achieved by the best encoder on VAD regression has been of .677 , highlighting that affective information is well-represented by the embeddings within this framework. In contrast, for Plutchik regression, the maximum score has been of .540 , likely due to the higher dimensionality (i.e., eight versus three) and the smaller dataset size. As for the sentence-based multi-label classification over Ekman’s six emotions plus a neutral one, the highest F_{1} score has been of .600 . In addition, visualizing the down projections of the embeddings reveals similar patterns. Concerning RQ 4 , affective cues appear to be more readily accessible through nonlinear downstream predictors, even though linear transformations can achieve comparable results. On the lexicon datasets, instruction-aware models rank the highest and outperform other candidates, including proprietary ones. However, this trend does not hold for sentence-level data, where task-tuned models are the best performing, giving insights into RQ 2 and RQ 3 . Regarding future works, we pivoted on three emotion theories, considering both the categorical and the dimensional families. However, other frameworks motivated by psychology and specifically developed for affective computing have been presented [ 7 ] . Therefore, it would be desirable to extend our overview to them. Additionally, we focused on discovering emotional cues from text. Nevertheless, most of the latest LMs have been trained with multimodality in mind. It has already been observed that incorporating different types of data can improve the understanding capabilities on a vast series of downstream tasks. 3 3 3 For instance, see the technical report at https://www.anthropic.com/claude-3-model-card . Consequently, adapting our methodology and applying it to multimodal resources [ 6 ] could highlight even more which embedding models are the best in distinguishing the nuanced facets of emotions. Limitations We should acknowledge that the prompts of the task-based and instruction-aware text encoders under examination were set up at our own discretion. To enable fair evaluation and comparison, we tried to configure these models as much as possible with matching settings for feature extraction. However, it is reasonable to suppose that, for each specific encoder and task between regression and classification, there might exist other instructions which would imply better predictive performance. Appendix 0.A Morphological split For NRC-VAD and NRC-EIL, along with the semantics-aware splitting strategy, we also applied a simpler approach to prevent morphological leakage between the 5-folds for cross-validation and the holdout test set. By comparing the results of both strategies, the impact of allowing semantic information exchange across splits can be shown. The morphological split was created by using stop word removal and Snowball stemming, assigning words with the same stem to the same group. All words in a group need to be in the same split. This is accomplished by the greedy algorithm in Section 3.1 . Appendix 0.B Prompts In Appendix 0.B , we detail a subset of the prompts used to instantiate the analyzed task-tuned and instruction-aware text encoders. As already mentioned in Section 3.2 , these prompts specify the downstream problem under examination, i.e., regression or classification, taking inspiration from the recommended prompts of each model card. To be precise, for regression, we slightly modified default prompts tailored to either semantic text similarity (STS) or retrieval if no STS option was available. {listing*} [!tb] Configured prompts for EmbeddingGemma , KaLM v2 , and Linq Mistral . The other text encoders are set up in an analogous way, with similar prompts with respect to this sample list. prompt_name and prompt are arguments passed to the model instantiation via the Hugging Face API in Python. ⬇ { " google@embeddinggemma -300 m ": { " regression_configs ": { " prompt_name ": " STS " }, " classification_configs ": { " prompt_name ": " Classification " } }, " tencent@KaLM - Embedding - Gemma3 -12 B -2511": { " regression_configs ": { " prompt ": " Instruct : Retrieving emotion .\ nQuery :" }, " classification_configs ": { " prompt ": " Instruct : Classifying emotion .\ nQuery :" } }, " Linq - AI - Research@Linq - Embed - Mistral ": { " regression_configs ": { " prompt ": " Instruct : Retrieve the emotion expressed in the text \ nQuery : " }, " classification_configs ": { " prompt ": " Instruct : Classify the emotion expressed in the text \ nQuery : " } } } Appendix 0.C Hyperparameter tuning Hyperparameters were tuned for each predictive model ( 4 ), dataset and splitting strategy ( 5 ), and encoder ( 12 ), leading to 240 experiments in total. We chose Optuna [ 1 ] as Bayesian optimization procedure. The set of tuneable hyperparameters excluded those that we expected to uniformly influence the performance of a given predictor. For instance, the batch size of the MLP was kept fixed, as it mainly refers to training efficiency and is unlikely to influence how well affective cues are extracted from text embeddings. The number of runs was heuristically determined with respect to the complexity of the search space under examination (e.g., number of hyperparameters, continuous or categorical variables), while ensuring that all predictive models had sufficient chances to find their optimal configuration. The final hyperparameters are listed from Table 5 to Table 24 . Most of them are self-explanatory, except for the complexity parameter of the MLP, which controls the network architecture in terms of number and size of its hidden layers. Each complexity level is a categorical variable, with twelve possible values, that corresponds to a predefined configuration, from shallow architectures with a single hidden layer (e.g., 64 neurons for complexity #0 and 1024 for #4) to deeper networks (e.g., one layer for levels from #0 to #4, two from #5 to #8, and three from #9 to #11). Appendix 0.D Full reports As supplementary documentation, we attach more detailed quantitative results. Considering NRC-VAD (cf. Table 27 ) and NRC-EIL (cf. Table 30 ), the outcomes with the morphology-aware splitting strategy are added. For GoEmotions (cf. Table 33 ), we include weighted- and micro-averaged metrics as auxiliary measures for a more complete view. Furthermore, we report mean and standard deviation of the cross-validation scores of the training procedure to give insights into the variability across folds (cf. Tables 25 , 26 , 28 , 29 , 31 and 32 ). Table 5: Summary of the best hyperparameters for LR on NRC-VAD, split with the semantics-aware strategy. \bm{R^{2}} alpha l1_ratio ST5 XXL 0.562 9.7e-04 7.5e-04 EmbeddingGemma 0.546 1.6e-03 0.231 Nomic v2 0.407 2.7e-03 0.162 Multilang E5 L Ins 0.526 4.0e-04 0.286 Jina v4 0.516 1.7e-03 0.475 Linq Mistral 0.626 6.4e-03 0.094 KaLM v2 0.631 6.7e-04 0.840 LLaMA Nemotron 8B 0.582 0.030 0.032 Qwen3 8B 0.617 3.9e-03 0.153 Voyage v3 L 0.548 6.8e-04 0.354 OpenAI Text v3 L 0.597 1.2e-03 0.729 Gemini 001 0.575 2.6e-04 0.403 Table 6: Summary of the best hyperparameters for LR on NRC-VAD, split with the morphology-aware strategy. \bm{R^{2}} alpha l1_ratio ST5 XXL 0.567 4.3e-04 5.3e-03 EmbeddingGemma 0.544 1.0e-04 0.026 Nomic v2 0.407 1.7e-03 0.216 Multilang E5 L Ins 0.531 1.1e-04 0.910 Jina v4 0.518 1.6e-03 0.480 Linq Mistral 0.637 5.4e-04 0.743 KaLM v2 0.640 0.027 9.4e-03 LLaMA Nemotron 8B 0.593 7.9e-03 0.113 Qwen3 8B 0.623 0.010 0.038 Voyage v3 L 0.549 2.0e-04 0.800 OpenAI Text v3 L 0.602 7.5e-04 0.934 Gemini 001 0.581 1.5e-03 0.025 Table 7: Summary of the best hyperparameters for LR on NRC-EIL, split with the semantics-aware strategy. \bm{R^{2}} alpha l1_ratio ST5 XXL 0.455 0.027 0.140 EmbeddingGemma 0.484 6.0e-03 0.563 Nomic v2 0.331 8.3e-03 0.605 Multilang E5 L Ins 0.469 0.340 3.3e-03 Jina v4 0.401 0.200 0.019 Linq Mistral 0.515 6.8e-03 0.610 KaLM v2 0.505 4.8e-03 0.864 LLaMA Nemotron 8B 0.465 6.3e-03 0.768 Qwen3 8B 0.500 5.8e-03 0.730 Voyage v3 L 0.433 8.2e-03 0.300 OpenAI Text v3 L 0.461 0.011 0.423 Gemini 001 0.455 6.1e-03 0.838 Table 8: Summary of the best hyperparameters for LR on NRC-EIL, split with the morphology-aware strategy. \bm{R^{2}} alpha l1_ratio ST5 XXL 0.465 7.1e-03 0.496 EmbeddingGemma 0.488 0.043 0.060 Nomic v2 0.339 0.011 0.412 Multilang E5 L Ins 0.469 6.5e-03 0.371 Jina v4 0.404 7.1e-03 0.529 Linq Mistral 0.524 0.048 0.072 KaLM v2 0.513 3.8e-03 0.947 LLaMA Nemotron 8B 0.465 0.026 0.171 Qwen3 8B 0.509 0.011 0.358 Voyage v3 L 0.444 0.034 0.057 OpenAI Text v3 L 0.475 0.015 0.293 Gemini 001 0.464 6.6e-03 0.689 Table 9: Summary of the best hyperparameters for LR on GoEmotions. \bm{F_{1}} C l1_ratio ST5 XXL 0.476 2.210 0.405 EmbeddingGemma 0.559 0.168 0.625 Nomic v2 0.496 0.288 0.241 Multilang E5 L Ins 0.517 43.115 0.693 Jina v4 0.489 0.182 0.840 Linq Mistral 0.544 0.074 0.215 KaLM v2 0.566 0.012 2.1e-03 LLaMA Nemotron 8B 0.509 0.178 0.767 Qwen3 8B 0.551 0.111 0.962 Voyage v3 L 0.490 0.724 0.806 OpenAI Text v3 L 0.548 0.059 0.520 Gemini 001 0.573 9.0e-03 0.087 Table 10: Summary of the best hyperparameters for k -NN on NRC-VAD, split with the semantics-aware strategy. \bm{R^{2}} n_neighbors weights ST5 XXL 0.566 29 distance EmbeddingGemma 0.527 27 distance Nomic v2 0.374 19 distance Multilang E5 L Ins 0.520 27 distance Jina v4 0.470 28 distance Linq Mistral 0.577 32 distance KaLM v2 0.583 32 distance LLaMA Nemotron 8B 0.449 64 distance Qwen3 8B 0.570 35 distance Voyage v3 L 0.469 23 distance OpenAI Text v3 L 0.560 24 distance Gemini 001 0.554 25 distance Table 11: Summary of the best hyperparameters for k -NN on NRC-VAD, split with the morphology-aware strategy. \bm{R^{2}} n_neighbors weights ST5 XXL 0.578 27 distance EmbeddingGemma 0.528 32 distance Nomic v2 0.368 27 distance Multilang E5 L Ins 0.524 27 distance Jina v4 0.468 34 distance Linq Mistral 0.587 29 distance KaLM v2 0.594 27 distance LLaMA Nemotron 8B 0.436 70 distance Qwen3 8B 0.574 30 distance Voyage v3 L 0.481 21 distance OpenAI Text v3 L 0.569 27 distance Gemini 001 0.562 21 distance Table 12: Summary of the best hyperparameters for k -NN on NRC-EIL, split with the semantics-aware strategy. \bm{R^{2}} n_neighbors weights ST5 XXL 0.464 18 distance EmbeddingGemma 0.484 30 distance Nomic v2 0.291 24 distance Multilang E5 L Ins 0.475 25 distance Jina v4 0.367 25 distance Linq Mistral 0.495 24 distance KaLM v2 0.491 28 distance LLaMA Nemotron 8B 0.456 22 distance Qwen3 8B 0.494 32 distance Voyage v3 L 0.381 21 distance OpenAI Text v3 L 0.445 16 distance Gemini 001 0.456 25 distance Table 13: Summary of the best hyperparameters for k -NN on NRC-EIL, split with the morphology-aware strategy. \bm{R^{2}} n_neighbors weights ST5 XXL 0.488 19 distance EmbeddingGemma 0.490 23 distance Nomic v2 0.305 20 distance Multilang E5 L Ins 0.479 24 distance Jina v4 0.380 25 distance Linq Mistral 0.504 23 distance KaLM v2 0.502 31 distance LLaMA Nemotron 8B 0.457 23 distance Qwen3 8B 0.502 27 distance Voyage v3 L 0.416 22 distance OpenAI Text v3 L 0.471 16 distance Gemini 001 0.470 20 distance Table 14: Summary of the best hyperparameters for k -NN on GoEmotions. \bm{F_{1}} n_neighbors weights ST5 XXL 0.410 4 uniform EmbeddingGemma 0.535 6 uniform Nomic v2 0.418 4 uniform Multilang E5 L Ins 0.472 4 uniform Jina v4 0.387 4 uniform Linq Mistral 0.487 6 uniform KaLM v2 0.524 6 uniform LLaMA Nemotron 8B 0.418 4 uniform Qwen3 8B 0.512 6 uniform Voyage v3 L 0.398 4 uniform OpenAI Text v3 L 0.440 4 uniform Gemini 001 0.557 6 uniform Table 15: Summary of the best hyperparameters for XGB on NRC-VAD, split with the semantics-aware strategy. \bm{R^{2}} lr max_depth reg_lambda ST5 XXL 0.558 0.048 7 1.5e-03 EmbeddingGemma 0.548 0.047 8 0.368 Nomic v2 0.391 0.064 7 5.7e-05 Multilang E5 L Ins 0.538 0.041 8 1.2e-05 Jina v4 0.501 0.056 7 7.2e-06 Linq Mistral 0.617 0.054 7 1.6e-08 KaLM v2 0.623 0.047 7 5.2e-07 LLaMA Nemotron 8B 0.562 0.084 6 6.0e-08 Qwen3 8B 0.600 0.050 7 4.1e-08 Voyage v3 L 0.529 0.073 6 0.531 OpenAI Text v3 L 0.570 0.077 6 0.062 Gemini 001 0.554 0.068 6 1.3e-03 Table 16: Summary of the best hyperparameters for XGB on NRC-VAD, split with the morphology-aware strategy. \bm{R^{2}} lr max_depth reg_lambda ST5 XXL 0.568 0.066 7 0.010 EmbeddingGemma 0.547 0.040 8 0.308 Nomic v2 0.388 0.063 7 4.1e-08 Multilang E5 L Ins 0.541 0.049 8 0.393 Jina v4 0.502 0.055 7 0.027 Linq Mistral 0.627 0.058 7 2.2e-08 KaLM v2 0.633 0.060 7 0.064 LLaMA Nemotron 8B 0.564 0.073 6 4.8e-08 Qwen3 8B 0.606 0.047 7 2.1e-04 Voyage v3 L 0.535 0.059 7 4.8e-05 OpenAI Text v3 L 0.579 0.070 6 1.1e-04 Gemini 001 0.562 0.073 6 1.2e-07 Table 17: Summary of the best hyperparameters for XGB on NRC-EIL, split with the semantics-aware strategy. \bm{R^{2}} lr max_depth reg_lambda ST5 XXL 0.432 0.052 4 5.7e-05 EmbeddingGemma 0.474 0.041 4 0.504 Nomic v2 0.301 0.046 4 0.712 Multilang E5 L Ins 0.490 0.028 5 0.710 Jina v4 0.379 0.054 3 0.839 Linq Mistral 0.516 0.029 4 1.3e-04 KaLM v2 0.506 0.029 5 7.0e-06 LLaMA Nemotron 8B 0.460 0.046 4 0.628 Qwen3 8B 0.501 0.046 4 0.980 Voyage v3 L 0.423 0.038 5 0.201 OpenAI Text v3 L 0.425 0.056 4 1.4e-03 Gemini 001 0.426 0.040 4 1.1e-08 Table 18: Summary of the best hyperparameters for XGB on NRC-EIL, split with the morphology-aware strategy. \bm{R^{2}} lr max_depth reg_lambda ST5 XXL 0.452 0.048 4 0.987 EmbeddingGemma 0.484 0.030 5 0.631 Nomic v2 0.306 0.048 4 0.490 Multilang E5 L Ins 0.489 0.026 5 0.462 Jina v4 0.391 0.048 4 0.962 Linq Mistral 0.523 0.030 4 0.985 KaLM v2 0.518 0.030 5 1.9e-05 LLaMA Nemotron 8B 0.463 0.071 3 1.2e-04 Qwen3 8B 0.508 0.026 5 0.024 Voyage v3 L 0.440 0.044 4 1.8e-04 OpenAI Text v3 L 0.446 0.052 4 4.8e-04 Gemini 001 0.442 0.044 4 0.122 Table 19: Summary of the best hyperparameters for XGB on GoEmotions. \bm{F_{1}} lr max_depth reg_lambda ST5 XXL 0.408 0.298 3 3.7e-08 EmbeddingGemma 0.520 0.189 3 0.996 Nomic v2 0.425 0.299 3 0.111 Multilang E5 L Ins 0.464 0.214 5 0.683 Jina v4 0.412 0.293 3 0.982 Linq Mistral 0.478 0.178 4 2.3e-06 KaLM v2 0.521 0.141 5 0.658 LLaMA Nemotron 8B 0.452 0.256 3 0.954 Qwen3 8B 0.507 0.180 4 0.050 Voyage v3 L 0.401 0.279 4 1.2e-07 OpenAI Text v3 L 0.469 0.281 3 3.1e-04 Gemini 001 0.539 0.266 3 0.672 Table 20: Summary of the best hyperparameters for MLP on NRC-VAD, split with the semantics-aware strategy. \bm{R^{2}} complexity activation lr decay ST5 XXL 0.624 4 relu 1.6e-04 8.8e-04 EmbeddingGemma 0.596 4 relu 1.4e-04 1.2e-03 Nomic v2 0.462 4 relu 4.4e-05 1.1e-03 Multilang E5 L Ins 0.569 4 relu 6.0e-05 9.8e-04 Jina v4 0.555 4 relu 6.8e-05 1.9e-03 Linq Mistral 0.663 4 relu 4.8e-05 1.5e-03 KaLM v2 0.673 4 relu 5.8e-05 2.1e-03 LLaMA Nemotron 8B 0.645 4 relu 1.0e-04 1.7e-03 Qwen3 8B 0.648 2 relu 1.0e-04 2.2e-03 Voyage v3 L 0.595 4 relu 4.7e-05 1.3e-03 OpenAI Text v3 L 0.651 4 relu 5.4e-05 2.0e-03 Gemini 001 0.635 4 relu 5.5e-05 1.5e-03 Table 21: Summary of the best hyperparameters for MLP on NRC-VAD, split with the morphology-aware strategy. \bm{R^{2}} complexity activation lr decay ST5 XXL 0.632 4 relu 3.2e-05 7.9e-04 EmbeddingGemma 0.594 4 relu 7.0e-05 8.9e-04 Nomic v2 0.459 4 relu 2.0e-05 1.5e-03 Multilang E5 L Ins 0.574 4 relu 3.3e-05 7.3e-04 Jina v4 0.558 3 relu 8.6e-05 1.7e-03 Linq Mistral 0.676 4 relu 4.7e-05 1.5e-03 KaLM v2 0.681 4 relu 4.8e-05 2.0e-03 LLaMA Nemotron 8B 0.656 4 relu 5.6e-05 1.3e-03 Qwen3 8B 0.658 4 relu 2.2e-05 2.3e-03 Voyage v3 L 0.599 4 relu 3.5e-05 1.3e-03 OpenAI Text v3 L 0.659 4 relu 5.2e-05 2.3e-03 Gemini 001 0.642 4 relu 5.4e-05 2.0e-03 Table 22: Summary of the best hyperparameters for MLP on NRC-EIL, split with the semantics-aware strategy. \bm{R^{2}} complexity activation lr decay ST5 XXL 0.506 8 relu 2.0e-05 2.0e-05 EmbeddingGemma 0.523 8 relu 3.4e-05 1.6e-05 Nomic v2 0.374 4 relu 5.2e-05 1.3e-05 Multilang E5 L Ins 0.504 8 relu 3.7e-05 1.7e-05 Jina v4 0.431 1 logistic 2.9e-05 1.3e-05 Linq Mistral 0.539 8 relu 1.2e-05 1.0e-05 KaLM v2 0.541 8 relu 1.0e-05 2.5e-05 LLaMA Nemotron 8B 0.521 8 relu 1.2e-05 3.6e-05 Qwen3 8B 0.528 8 relu 4.1e-05 5.2e-05 Voyage v3 L 0.481 4 relu 7.6e-05 4.4e-05 OpenAI Text v3 L 0.510 4 relu 5.8e-05 2.4e-05 Gemini 001 0.501 8 relu 2.0e-05 1.0e-05 Table 23: Summary of the best hyperparameters for MLP on NRC-EIL, split with the morphology-aware strategy. \bm{R^{2}} complexity activation lr decay ST5 XXL 0.522 8 relu 7.5e-05 1.5e-05 EmbeddingGemma 0.524 8 relu 6.6e-05 1.0e-05 Nomic v2 0.378 4 relu 4.6e-05 4.6e-05 Multilang E5 L Ins 0.503 4 relu 1.2e-04 2.2e-05 Jina v4 0.436 1 logistic 3.1e-05 1.0e-05 Linq Mistral 0.544 8 relu 3.4e-05 5.8e-05 KaLM v2 0.544 7 relu 7.5e-05 6.6e-05 LLaMA Nemotron 8B 0.528 8 relu 3.5e-05 1.7e-05 Qwen3 8B 0.528 7 relu 3.8e-05 2.1e-04 Voyage v3 L 0.491 8 relu 5.8e-05 1.7e-05 OpenAI Text v3 L 0.525 8 relu 1.5e-05 6.1e-05 Gemini 001 0.507 3 logistic 2.4e-04 1.0e-05 Table 24: Summary of the best hyperparameters for MLP on GoEmotions. \bm{F_{1}} complexity activation lr decay ST5 XXL 0.501 7 tanh 9.7e-04 1.0e-05 EmbeddingGemma 0.571 7 tanh 2.5e-03 2.9e-05 Nomic v2 0.520 10 tanh 6.7e-04 2.2e-05 Multilang E5 L Ins 0.531 8 tanh 1.9e-03 2.2e-05 Jina v4 0.500 3 tanh 1.9e-03 1.2e-05 Linq Mistral 0.551 3 tanh 1.3e-03 1.0e-05 KaLM v2 0.571 4 tanh 8.4e-04 3.1e-05 LLaMA Nemotron 8B 0.532 8 tanh 7.7e-04 3.6e-05 Qwen3 8B 0.556 1 tanh 1.5e-03 2.8e-05 Voyage v3 L 0.500 8 tanh 8.1e-04 1.1e-05 OpenAI Text v3 L 0.559 8 tanh 5.3e-04 1.2e-05 Gemini 001 0.579 7 relu 2.5e-04 2.1e-05 Table 25: Full summary of the 5-fold cross-validation scores for LR and k -NN on NRC-VAD. For each encoder, top and bottom rows refer to data splits based on semantic and morphological leakage prevention, respectively. LR \bm{k} -NN MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 0.070_{\pm.003} .631_{\pm.014} .772_{\pm.009} 0.079_{\pm.004} .583_{\pm.014} .720_{\pm.010} 0.071_{\pm.001} .640_{\pm.008} .780_{\pm.005} 0.080_{\pm.001} .594_{\pm.007} .731_{\pm.004} Linq Mistral 0.071_{\pm.003} .626_{\pm.014} .768_{\pm.009} 0.080_{\pm.004} .577_{\pm.013} .717_{\pm.011} 0.072_{\pm.001} .637_{\pm.009} .778_{\pm.006} 0.082_{\pm.001} .587_{\pm.007} .725_{\pm.004} OpenAI Text v3 L 0.077_{\pm.004} .597_{\pm.010} .745_{\pm.008} 0.084_{\pm.004} .560_{\pm.010} .682_{\pm.008} 0.079_{\pm.001} .602_{\pm.006} .750_{\pm.003} 0.086_{\pm.002} .569_{\pm.005} .688_{\pm.004} Qwen3 8B 0.073_{\pm.003} .617_{\pm.014} .761_{\pm.008} 0.082_{\pm.004} .570_{\pm.012} .706_{\pm.010} 0.075_{\pm.001} .623_{\pm.009} .767_{\pm.005} 0.085_{\pm.002} .574_{\pm.008} .710_{\pm.005} LLaMA Nemotron 8B 0.079_{\pm.003} .582_{\pm.017} .733_{\pm.012} 0.105_{\pm.007} .449_{\pm.018} .575_{\pm.016} 0.081_{\pm.001} .593_{\pm.008} .743_{\pm.006} 0.112_{\pm.002} .436_{\pm.008} .560_{\pm.006} Gemini 001 0.081_{\pm.003} .575_{\pm.011} .733_{\pm.007} 0.085_{\pm.005} .554_{\pm.009} .686_{\pm.009} 0.083_{\pm.001} .581_{\pm.007} .738_{\pm.004} 0.087_{\pm.002} .562_{\pm.006} .696_{\pm.004} ST5 XXL 0.083_{\pm.004} .562_{\pm.013} .719_{\pm.010} 0.083_{\pm.004} .566_{\pm.011} .696_{\pm.009} 0.086_{\pm.001} .567_{\pm.007} .724_{\pm.004} 0.084_{\pm.002} .578_{\pm.006} .708_{\pm.004} EmbeddingGemma 0.086_{\pm.006} .546_{\pm.015} .704_{\pm.015} 0.090_{\pm.006} .527_{\pm.013} .671_{\pm.014} 0.091_{\pm.002} .544_{\pm.010} .705_{\pm.006} 0.094_{\pm.002} .528_{\pm.010} .669_{\pm.007} Voyage v3 L 0.086_{\pm.004} .548_{\pm.008} .708_{\pm.007} 0.102_{\pm.006} .469_{\pm.003} .609_{\pm.005} 0.090_{\pm.002} .549_{\pm.008} .710_{\pm.005} 0.103_{\pm.002} .481_{\pm.006} .622_{\pm.005} Multilang E5 L Ins 0.090_{\pm.006} .526_{\pm.015} .689_{\pm.014} 0.091_{\pm.006} .520_{\pm.017} .670_{\pm.015} 0.093_{\pm.002} .531_{\pm.011} .693_{\pm.008} 0.095_{\pm.002} .524_{\pm.012} .674_{\pm.008} Jina v4 0.092_{\pm.005} .516_{\pm.012} .680_{\pm.011} 0.101_{\pm.005} .470_{\pm.011} .613_{\pm.008} 0.096_{\pm.002} .518_{\pm.005} .682_{\pm.003} 0.106_{\pm.002} .468_{\pm.007} .606_{\pm.005} Nomic v2 0.114_{\pm.009} .407_{\pm.018} .577_{\pm.023} 0.120_{\pm.009} .374_{\pm.020} .508_{\pm.019} 0.118_{\pm.003} .407_{\pm.015} .578_{\pm.013} 0.127_{\pm.004} .368_{\pm.011} .496_{\pm.010} Table 26: Full summary of the 5-fold cross-validation scores for XGB and MLP on NRC-VAD. For each encoder, top and bottom rows refer to data splits based on semantic and morphological leakage prevention, respectively. XGB MLP MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 0.072_{\pm.003} .623_{\pm.014} .756_{\pm.009} 0.062_{\pm.003} .672_{\pm.013} .799_{\pm.009} 0.073_{\pm.001} .633_{\pm.007} .767_{\pm.004} 0.063_{\pm.001} .681_{\pm.007} .805_{\pm.004} Linq Mistral 0.073_{\pm.003} .617_{\pm.014} .753_{\pm.010} 0.064_{\pm.003} .662_{\pm.015} .791_{\pm.011} 0.074_{\pm.001} .627_{\pm.007} .761_{\pm.004} 0.065_{\pm.001} .675_{\pm.009} .803_{\pm.005} OpenAI Text v3 L 0.082_{\pm.004} .570_{\pm.013} .712_{\pm.010} 0.066_{\pm.003} .651_{\pm.009} .783_{\pm.006} 0.084_{\pm.002} .579_{\pm.006} .720_{\pm.004} 0.068_{\pm.001} .659_{\pm.006} .790_{\pm.004} Qwen3 8B 0.076_{\pm.003} .600_{\pm.014} .739_{\pm.009} 0.067_{\pm.003} .648_{\pm.013} .782_{\pm.009} 0.078_{\pm.001} .606_{\pm.007} .743_{\pm.005} 0.068_{\pm.001} .658_{\pm.008} .789_{\pm.005} LLaMA Nemotron 8B 0.083_{\pm.004} .562_{\pm.015} .708_{\pm.012} 0.068_{\pm.003} .642_{\pm.016} .778_{\pm.011} 0.087_{\pm.001} .564_{\pm.008} .709_{\pm.005} 0.069_{\pm.001} .654_{\pm.008} .788_{\pm.005} Gemini 001 0.085_{\pm.004} .554_{\pm.009} .695_{\pm.008} 0.069_{\pm.003} .635_{\pm.009} .773_{\pm.006} 0.087_{\pm.002} .562_{\pm.007} .705_{\pm.005} 0.071_{\pm.001} .641_{\pm.007} .778_{\pm.004} ST5 XXL 0.084_{\pm.004} .558_{\pm.014} .696_{\pm.011} 0.071_{\pm.004} .625_{\pm.011} .765_{\pm.009} 0.086_{\pm.001} .568_{\pm.005} .712_{\pm.004} 0.073_{\pm.001} .632_{\pm.006} .771_{\pm.003} EmbeddingGemma 0.086_{\pm.006} .548_{\pm.015} .691_{\pm.014} 0.077_{\pm.006} .595_{\pm.015} .741_{\pm.012} 0.090_{\pm.002} .547_{\pm.010} .690_{\pm.006} 0.081_{\pm.002} .593_{\pm.010} .741_{\pm.006} Voyage v3 L 0.090_{\pm.005} .529_{\pm.009} .680_{\pm.008} 0.077_{\pm.004} .595_{\pm.008} .741_{\pm.006} 0.093_{\pm.002} .535_{\pm.005} .683_{\pm.003} 0.080_{\pm.002} .598_{\pm.007} .744_{\pm.005} Multilang E5 L Ins 0.088_{\pm.006} .538_{\pm.016} .686_{\pm.014} 0.082_{\pm.005} .568_{\pm.015} .720_{\pm.013} 0.091_{\pm.002} .541_{\pm.009} .690_{\pm.006} 0.085_{\pm.002} .573_{\pm.011} .725_{\pm.007} Jina v4 0.095_{\pm.005} .501_{\pm.010} .652_{\pm.009} 0.085_{\pm.005} .554_{\pm.012} .708_{\pm.010} 0.099_{\pm.002} .502_{\pm.005} .654_{\pm.003} 0.088_{\pm.001} .556_{\pm.006} .712_{\pm.006} Nomic v2 0.117_{\pm.009} .391_{\pm.018} .535_{\pm.019} 0.103_{\pm.009} .462_{\pm.020} .627_{\pm.021} 0.122_{\pm.003} .388_{\pm.013} .536_{\pm.011} 0.108_{\pm.003} .457_{\pm.014} .624_{\pm.012} Table 27: Full summary of the regression metrics at test time for NRC-VAD, sorted by R^{2} score of the MLP model in descending order. For each encoder, top and bottom rows refer to data splits based on semantic and morphological leakage prevention, respectively. LR \bm{k} -NN XGB MLP MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 \mathbf{.066} \mathbf{.637} \mathbf{.790} \mathbf{.075} \mathbf{.591} \underline{\underline{.746}} \mathbf{.068} \mathbf{.630} \mathbf{.777} \mathbf{.059} \mathbf{.677} \mathbf{.811} \mathbf{.074} \mathbf{.665} \underline{\underline{.797}} \mathbf{.083} \mathbf{.620} \mathbf{.756} \mathbf{.075} \mathbf{.659} \mathbf{.788} \mathbf{.065} \mathbf{.705} \mathbf{.825} Linq Mistral .067 .631 .786 \underline{.076} \underline{.586} \mathbf{.746} .069 .622 \underline{.773} .061 .667 .806 \underline{\underline{.074}} \underline{\underline{.665}} \mathbf{.797} \underline{\underline{.084}} \underline{\underline{.617}} \underline{\underline{.754}} \underline{\underline{.075}} \underline{\underline{.658}} \underline{\underline{.786}} \underline{\underline{.065}} \underline{\underline{.703}} \underline{\underline{.823}} OpenAI Text v3 L .073 .602 .761 .079 .572 .704 .077 .582 .734 .062 .659 .797 .082 .627 .768 .091 .589 .708 .087 .604 .741 .070 .682 .806 Qwen3 8B .069 .624 .780 .077 .578 .733 .072 .607 .761 .063 .657 .798 .077 .652 .786 .088 .599 .736 .081 .634 .767 .070 .685 .808 LLaMA Nemotron 8B .075 .589 .745 .099 .463 .607 .079 .570 .729 .064 .653 .795 .084 .618 .762 .119 .459 .587 .090 .593 .734 .070 .681 .807 Gemini 001 .077 .578 .748 .080 .564 .710 .081 .560 .717 .066 .638 .782 .086 .611 .760 .091 .589 .720 .091 .589 .730 .073 .670 .799 ST5 XXL .080 .564 .735 .077 .581 .722 .080 .564 .715 .068 .631 .777 .089 .596 .744 .087 .603 .732 .089 .595 .736 .075 .659 .790 EmbeddingGemma .082 .553 .725 .085 .536 .697 .082 .553 .713 .073 .603 .757 .095 .569 .724 .099 .549 .691 .095 .571 .712 .084 .618 .760 Voyage v3 L .082 .556 .728 .098 .472 .624 .086 .534 .699 .073 .601 .755 .092 .581 .732 .109 .505 .644 .097 .558 .703 .082 .626 .765 Multilang E5 L Ins .084 .542 .717 .085 .537 .704 .083 .550 .713 .077 .582 .741 .097 .558 .713 .099 .552 .700 .095 .569 .714 .088 .601 .747 Jina v4 .088 .520 .694 .096 .479 .636 .091 .506 .671 .080 .563 .726 .101 .543 .700 .113 .490 .628 .104 .527 .676 .092 .581 .729 Nomic v2 .108 .413 .603 .112 .396 .540 .110 .403 .567 .097 .476 .647 .127 .426 .590 .136 .385 .507 .132 .404 .550 .114 .482 .639 Table 28: Full summary of the 5-fold cross-validation scores for LR and k -NN on NRC-EIL. For each encoder, top and bottom rows refer to data splits based on semantic and morphological leakage prevention, respectively. LR \bm{k} -NN MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 0.023_{\pm.000} .505_{\pm.012} .658_{\pm.010} 0.024_{\pm.000} .491_{\pm.010} .630_{\pm.007} 0.022_{\pm.000} .513_{\pm.010} .667_{\pm.006} 0.023_{\pm.000} .502_{\pm.011} .638_{\pm.007} Linq Mistral 0.023_{\pm.000} .515_{\pm.012} .666_{\pm.009} 0.023_{\pm.000} .495_{\pm.010} .639_{\pm.004} 0.022_{\pm.000} .524_{\pm.006} .675_{\pm.005} 0.022_{\pm.000} .504_{\pm.007} .651_{\pm.005} Qwen3 8B 0.023_{\pm.000} .500_{\pm.010} .653_{\pm.008} 0.024_{\pm.000} .494_{\pm.013} .635_{\pm.008} 0.022_{\pm.000} .509_{\pm.005} .661_{\pm.003} 0.023_{\pm.000} .502_{\pm.006} .646_{\pm.003} EmbeddingGemma 0.024_{\pm.000} .484_{\pm.010} .640_{\pm.007} 0.024_{\pm.000} .484_{\pm.009} .621_{\pm.006} 0.023_{\pm.000} .488_{\pm.010} .645_{\pm.007} 0.023_{\pm.000} .490_{\pm.013} .637_{\pm.009} LLaMA Nemotron 8B 0.025_{\pm.001} .465_{\pm.011} .623_{\pm.008} 0.026_{\pm.001} .456_{\pm.006} .600_{\pm.005} 0.024_{\pm.000} .465_{\pm.006} .624_{\pm.007} 0.025_{\pm.000} .457_{\pm.007} .602_{\pm.005} OpenAI Text v3 L 0.025_{\pm.000} .461_{\pm.012} .616_{\pm.008} 0.026_{\pm.000} .445_{\pm.008} .580_{\pm.005} 0.024_{\pm.000} .475_{\pm.009} .631_{\pm.006} 0.024_{\pm.000} .471_{\pm.008} .608_{\pm.005} ST5 XXL 0.026_{\pm.000} .455_{\pm.011} .612_{\pm.008} 0.025_{\pm.000} .464_{\pm.008} .606_{\pm.004} 0.024_{\pm.000} .465_{\pm.008} .622_{\pm.007} 0.023_{\pm.000} .488_{\pm.011} .628_{\pm.008} Multilang E5 L Ins 0.025_{\pm.000} .469_{\pm.011} .627_{\pm.006} 0.025_{\pm.000} .475_{\pm.012} .617_{\pm.007} 0.024_{\pm.000} .469_{\pm.012} .629_{\pm.009} 0.024_{\pm.000} .479_{\pm.013} .622_{\pm.010} Gemini 001 0.026_{\pm.000} .455_{\pm.007} .610_{\pm.004} 0.026_{\pm.001} .456_{\pm.003} .598_{\pm.005} 0.024_{\pm.000} .464_{\pm.007} .619_{\pm.006} 0.024_{\pm.001} .470_{\pm.012} .618_{\pm.009} Voyage v3 L 0.027_{\pm.000} .433_{\pm.011} .600_{\pm.008} 0.029_{\pm.001} .381_{\pm.004} .520_{\pm.004} 0.026_{\pm.000} .444_{\pm.006} .611_{\pm.005} 0.027_{\pm.000} .416_{\pm.009} .551_{\pm.007} Jina v4 0.028_{\pm.001} .401_{\pm.011} .554_{\pm.010} 0.030_{\pm.001} .367_{\pm.008} .504_{\pm.009} 0.027_{\pm.000} .404_{\pm.006} .566_{\pm.006} 0.029_{\pm.000} .380_{\pm.008} .516_{\pm.007} Nomic v2 0.032_{\pm.001} .331_{\pm.006} .485_{\pm.004} 0.034_{\pm.001} .291_{\pm.012} .404_{\pm.012} 0.031_{\pm.000} .339_{\pm.009} .492_{\pm.008} 0.033_{\pm.000} .305_{\pm.011} .428_{\pm.009} Table 29: Full summary of the 5-fold cross-validation scores for XGB and MLP on NRC-EIL. For each encoder, top and bottom rows refer to data splits based on semantic and morphological leakage prevention, respectively. XGB MLP MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 0.023_{\pm.001} .506_{\pm.010} .646_{\pm.009} 0.021_{\pm.000} .540_{\pm.015} .694_{\pm.015} 0.022_{\pm.001} .518_{\pm.011} .657_{\pm.007} 0.021_{\pm.000} .543_{\pm.012} .698_{\pm.008} Linq Mistral 0.022_{\pm.000} .516_{\pm.010} .657_{\pm.008} 0.022_{\pm.001} .536_{\pm.017} .693_{\pm.009} 0.022_{\pm.000} .523_{\pm.009} .663_{\pm.008} 0.021_{\pm.000} .542_{\pm.008} .696_{\pm.006} Qwen3 8B 0.023_{\pm.001} .501_{\pm.015} .646_{\pm.012} 0.022_{\pm.001} .526_{\pm.015} .681_{\pm.011} 0.022_{\pm.000} .508_{\pm.004} .647_{\pm.003} 0.021_{\pm.000} .527_{\pm.009} .680_{\pm.006} EmbeddingGemma 0.025_{\pm.000} .474_{\pm.010} .616_{\pm.009} 0.022_{\pm.000} .523_{\pm.013} .681_{\pm.009} 0.024_{\pm.000} .484_{\pm.010} .621_{\pm.010} 0.022_{\pm.000} .522_{\pm.014} .686_{\pm.011} LLaMA Nemotron 8B 0.025_{\pm.001} .460_{\pm.008} .595_{\pm.005} 0.022_{\pm.001} .519_{\pm.017} .675_{\pm.011} 0.025_{\pm.000} .463_{\pm.009} .610_{\pm.007} 0.022_{\pm.001} .527_{\pm.015} .687_{\pm.013} OpenAI Text v3 L 0.027_{\pm.000} .425_{\pm.008} .553_{\pm.006} 0.023_{\pm.001} .510_{\pm.017} .667_{\pm.014} 0.025_{\pm.001} .446_{\pm.011} .575_{\pm.007} 0.022_{\pm.001} .524_{\pm.013} .678_{\pm.012} ST5 XXL 0.027_{\pm.000} .432_{\pm.008} .569_{\pm.005} 0.023_{\pm.001} .505_{\pm.019} .670_{\pm.012} 0.025_{\pm.000} .452_{\pm.006} .586_{\pm.006} 0.022_{\pm.000} .521_{\pm.010} .683_{\pm.011} Multilang E5 L Ins 0.024_{\pm.000} .490_{\pm.011} .634_{\pm.006} 0.023_{\pm.000} .503_{\pm.014} .661_{\pm.011} 0.023_{\pm.000} .489_{\pm.010} .632_{\pm.007} 0.023_{\pm.001} .502_{\pm.014} .662_{\pm.011} Gemini 001 0.027_{\pm.001} .426_{\pm.005} .557_{\pm.006} 0.023_{\pm.000} .499_{\pm.010} .659_{\pm.012} 0.025_{\pm.001} .442_{\pm.011} .574_{\pm.010} 0.023_{\pm.000} .506_{\pm.007} .654_{\pm.007} Voyage v3 L 0.027_{\pm.000} .423_{\pm.007} .559_{\pm.008} 0.024_{\pm.001} .480_{\pm.016} .649_{\pm.012} 0.026_{\pm.000} .440_{\pm.006} .583_{\pm.006} 0.023_{\pm.001} .490_{\pm.011} .660_{\pm.008} Jina v4 0.029_{\pm.001} .379_{\pm.010} .518_{\pm.012} 0.027_{\pm.001} .430_{\pm.013} .576_{\pm.011} 0.028_{\pm.000} .391_{\pm.010} .528_{\pm.010} 0.026_{\pm.000} .435_{\pm.007} .585_{\pm.008} Nomic v2 0.033_{\pm.001} .301_{\pm.008} .424_{\pm.013} 0.030_{\pm.000} .373_{\pm.006} .540_{\pm.006} 0.032_{\pm.000} .306_{\pm.007} .431_{\pm.006} 0.029_{\pm.001} .377_{\pm.014} .540_{\pm.016} Table 30: Full summary of the regression metrics at test time for NRC-EIL, sorted by R^{2} score of the MLP model in descending order. For each encoder, top and bottom rows refer to data splits based on semantic and morphological leakage prevention, respectively. LR \bm{k} -NN XGB MLP MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} MSE R^{2} \rho_{c} KaLM v2 \underline{\underline{.023}} \underline{\underline{.516}} \underline{\underline{.705}} \mathbf{.024} \underline{\underline{.500}} \underline{.685} \underline{\underline{.023}} \underline{\underline{.515}} \underline{\underline{.697}} \mathbf{.022} \mathbf{.540} \mathbf{.730} \underline{\underline{.021}} \mathbf{.531} \underline{\underline{.718}} \underline{\underline{.022}} \underline{\underline{.514}} .695 \underline{\underline{.021}} \underline{\underline{.528}} \underline{.709} \mathbf{.020} \mathbf{.554} \underline{\underline{.740}} Linq Mistral \mathbf{.023} \mathbf{.519} \mathbf{.710} \underline{\underline{.024}} \mathbf{.500} \mathbf{.695} \mathbf{.023} \mathbf{.517} \mathbf{.704} \underline{\underline{.022}} \underline{.528} \underline{\underline{.729}} \mathbf{.021} \underline{\underline{.531}} \mathbf{.719} \mathbf{.022} \mathbf{.514} \mathbf{.706} \mathbf{.021} \mathbf{.533} \mathbf{.717} \underline{\underline{.021}} \underline{\underline{.551}} \mathbf{.744} Qwen3 8B \underline{\underline{.023}} \underline{\underline{.517}} \underline{\underline{.704}} \underline{\underline{.024}} \underline{\underline{.491}} \underline{.683} \underline{\underline{.023}} \underline{\underline{.513}} \underline{\underline{.698}} \underline{.022} \underline{.524} \underline{\underline{.725}} \underline{\underline{.022}} \underline{\underline{.523}} \underline{.709} \underline{\underline{.022}} \underline{\underline{.508}} \underline{.696} .022 .517 .697 \underline{\underline{.021}} \underline{.541} \underline{.735} EmbeddingGemma .024 .491 .683 \underline{.024} \underline{.486} .672 .024 .485 .667 \underline{.022} \underline{.524} .717 .022 .507 .696 .023 .492 .680 .023 .496 .673 \underline{.021} \underline{.537} \underline{.732} LLaMA Nemotron 8B .025 .475 .673 .026 .446 .647 .026 .456 .637 \underline{.023} \underline{.523} \underline{.720} .023 .485 .675 .024 .466 .658 .024 .482 .666 \underline{.021} .535 \underline{.733} OpenAI Text v3 L .025 .471 .658 .026 .454 .631 .026 .445 .616 .023 .516 .704 .023 .494 .683 .023 .486 .660 .025 .459 .638 .021 .532 .723 ST5 XXL .025 .471 .659 .025 .474 .659 .026 .449 .626 .023 .513 .712 .024 .482 .671 \underline{.023} \underline{.501} .681 .024 .472 .644 .022 .526 .725 Multilang E5 L Ins .025 .472 .671 .025 .472 .665 .024 .489 .676 .023 .508 .709 .024 .478 .675 .023 .490 .676 .023 .499 .682 .022 .514 .708 Gemini 001 .026 .460 .654 .026 .458 .648 .027 .436 .613 .024 .499 .699 .024 .486 .674 .024 .481 .671 .025 .462 .638 .022 .519 .706 Voyage v3 L .026 .446 .645 .029 .401 .578 .027 .427 .605 .024 .488 .688 .025 .460 .660 .027 .421 .602 .025 .456 .640 .023 .509 .707 Jina v4 .028 .408 .600 .030 .372 .558 .029 .384 .572 .027 .432 .631 .026 .427 .623 .027 .411 .589 .027 .418 .597 .025 .455 .654 Nomic v2 .032 .336 .529 .034 .293 .457 .033 .313 .479 .030 .369 .578 .030 .348 .543 .032 .318 .484 .032 .316 .493 .029 .382 .594 Table 31: Full summary of the 5-fold cross-validation scores for LR and k -NN on GoEmotions. For each encoder, its rows refer to macro, weighted, and micro averages, respectively. LR \bm{k} -NN p r F_{1} p r F_{1} Gemini 001 .698_{\pm.004} .495_{\pm.005} .573_{\pm.005} .596_{\pm.008} .529_{\pm.006} .557_{\pm.005} .731_{\pm.004} .608_{\pm.004} .660_{\pm.003} .646_{\pm.002} .630_{\pm.004} .636_{\pm.003} .739_{\pm.004} .608_{\pm.004} .667_{\pm.003} .652_{\pm.002} .630_{\pm.004} .640_{\pm.003} EmbeddingGemma .679_{\pm.006} .483_{\pm.007} .559_{\pm.006} .586_{\pm.009} .501_{\pm.004} .535_{\pm.004} .724_{\pm.001} .590_{\pm.004} .647_{\pm.002} .629_{\pm.002} .606_{\pm.003} .614_{\pm.002} .734_{\pm.001} .590_{\pm.004} .654_{\pm.002} .635_{\pm.002} .606_{\pm.003} .620_{\pm.002} OpenAI Text v3 L .667_{\pm.007} .474_{\pm.010} .548_{\pm.009} .474_{\pm.004} .422_{\pm.004} .439_{\pm.003} .710_{\pm.002} .589_{\pm.003} .640_{\pm.002} .518_{\pm.001} .562_{\pm.002} .534_{\pm.001} .719_{\pm.002} .589_{\pm.003} .648_{\pm.002} .532_{\pm.001} .562_{\pm.002} .547_{\pm.001} Linq Mistral .643_{\pm.003} .479_{\pm.008} .544_{\pm.007} .549_{\pm.009} .449_{\pm.007} .487_{\pm.007} .692_{\pm.002} .592_{\pm.003} .635_{\pm.002} .594_{\pm.003} .566_{\pm.003} .576_{\pm.003} .701_{\pm.002} .592_{\pm.003} .642_{\pm.002} .605_{\pm.004} .566_{\pm.003} .585_{\pm.003} Qwen3 8B .676_{\pm.005} .476_{\pm.007} .550_{\pm.006} .573_{\pm.007} .474_{\pm.005} .512_{\pm.005} .713_{\pm.001} .592_{\pm.002} .643_{\pm.002} .605_{\pm.003} .575_{\pm.004} .586_{\pm.003} .723_{\pm.001} .592_{\pm.002} .651_{\pm.002} .614_{\pm.003} .575_{\pm.004} .594_{\pm.003} KaLM v2 .675_{\pm.007} .496_{\pm.006} .566_{\pm.006} .579_{\pm.007} .490_{\pm.008} .524_{\pm.006} .714_{\pm.003} .603_{\pm.004} .651_{\pm.003} .620_{\pm.004} .594_{\pm.004} .603_{\pm.004} .723_{\pm.003} .603_{\pm.004} .658_{\pm.003} .625_{\pm.003} .594_{\pm.004} .609_{\pm.004} Multilang E5 L Ins .636_{\pm.004} .443_{\pm.006} .517_{\pm.004} .485_{\pm.011} .469_{\pm.009} .472_{\pm.008} .703_{\pm.003} .563_{\pm.003} .620_{\pm.001} .544_{\pm.002} .594_{\pm.004} .566_{\pm.003} .716_{\pm.004} .563_{\pm.003} .630_{\pm.001} .548_{\pm.002} .594_{\pm.004} .570_{\pm.002} LLaMA Nemotron 8B .615_{\pm.005} .444_{\pm.004} .509_{\pm.003} .466_{\pm.007} .399_{\pm.004} .418_{\pm.005} .676_{\pm.002} .577_{\pm.002} .620_{\pm.002} .530_{\pm.002} .573_{\pm.002} .545_{\pm.001} .686_{\pm.002} .577_{\pm.002} .627_{\pm.002} .543_{\pm.002} .573_{\pm.002} .557_{\pm.001} Nomic v2 .639_{\pm.010} .416_{\pm.008} .496_{\pm.006} .451_{\pm.009} .405_{\pm.004} .418_{\pm.005} .691_{\pm.002} .530_{\pm.003} .593_{\pm.002} .495_{\pm.003} .527_{\pm.005} .508_{\pm.003} .705_{\pm.002} .530_{\pm.003} .605_{\pm.002} .499_{\pm.002} .527_{\pm.005} .512_{\pm.003} Jina v4 .623_{\pm.009} .411_{\pm.004} .489_{\pm.004} .419_{\pm.005} .373_{\pm.003} .387_{\pm.003} .682_{\pm.003} .534_{\pm.003} .594_{\pm.003} .480_{\pm.002} .521_{\pm.001} .494_{\pm.001} .696_{\pm.003} .534_{\pm.003} .604_{\pm.003} .499_{\pm.002} .521_{\pm.001} .509_{\pm.001} Voyage v3 L .609_{\pm.003} .417_{\pm.005} .490_{\pm.003} .430_{\pm.009} .383_{\pm.005} .398_{\pm.006} .683_{\pm.003} .538_{\pm.002} .597_{\pm.002} .494_{\pm.003} .533_{\pm.003} .509_{\pm.003} .697_{\pm.003} .538_{\pm.002} .607_{\pm.002} .508_{\pm.004} .533_{\pm.003} .520_{\pm.003} ST5 XXL .619_{\pm.007} .396_{\pm.006} .475_{\pm.005} .434_{\pm.005} .399_{\pm.004} .409_{\pm.003} .687_{\pm.003} .520_{\pm.002} .585_{\pm.002} .496_{\pm.002} .542_{\pm.002} .514_{\pm.002} .704_{\pm.004} .520_{\pm.002} .598_{\pm.002} .513_{\pm.002} .542_{\pm.002} .527_{\pm.002} Table 32: Full summary of the 5-fold cross-validation scores for XGB and MLP on GoEmotions. For each encoder, its rows refer to macro, weighted, and micro averages, respectively. XGB MLP p r F_{1} p r F_{1} Gemini 001 .656_{\pm.009} .464_{\pm.005} .538_{\pm.005} .688_{\pm.009} .508_{\pm.006} .577_{\pm.006} .712_{\pm.004} .577_{\pm.003} .634_{\pm.002} .721_{\pm.005} .622_{\pm.005} .665_{\pm.005} .722_{\pm.004} .577_{\pm.003} .641_{\pm.002} .728_{\pm.004} .622_{\pm.005} .671_{\pm.004} EmbeddingGemma .672_{\pm.004} .432_{\pm.006} .520_{\pm.005} .671_{\pm.014} .493_{\pm.010} .559_{\pm.009} .716_{\pm.002} .543_{\pm.002} .613_{\pm.002} .715_{\pm.003} .608_{\pm.013} .652_{\pm.008} .729_{\pm.002} .543_{\pm.002} .622_{\pm.002} .720_{\pm.005} .608_{\pm.013} .659_{\pm.007} OpenAI Text v3 L .633_{\pm.008} .383_{\pm.005} .469_{\pm.006} .649_{\pm.006} .493_{\pm.008} .555_{\pm.007} .692_{\pm.003} .510_{\pm.004} .580_{\pm.004} .699_{\pm.004} .605_{\pm.007} .646_{\pm.005} .709_{\pm.003} .510_{\pm.004} .593_{\pm.003} .707_{\pm.004} .605_{\pm.007} .652_{\pm.004} Linq Mistral .641_{\pm.008} .392_{\pm.005} .477_{\pm.005} .674_{\pm.012} .469_{\pm.011} .543_{\pm.010} .696_{\pm.002} .524_{\pm.003} .591_{\pm.002} .706_{\pm.006} .590_{\pm.011} .639_{\pm.006} .712_{\pm.002} .524_{\pm.003} .604_{\pm.002} .713_{\pm.008} .590_{\pm.011} .645_{\pm.004} Qwen3 8B .656_{\pm.010} .423_{\pm.008} .507_{\pm.008} .688_{\pm.010} .476_{\pm.010} .551_{\pm.009} .706_{\pm.004} .538_{\pm.004} .606_{\pm.004} .714_{\pm.002} .597_{\pm.005} .646_{\pm.003} .719_{\pm.003} .538_{\pm.004} .616_{\pm.004} .721_{\pm.003} .597_{\pm.005} .653_{\pm.002} KaLM v2 .681_{\pm.009} .435_{\pm.009} .521_{\pm.010} .691_{\pm.012} .497_{\pm.012} .567_{\pm.010} .716_{\pm.004} .555_{\pm.003} .619_{\pm.003} .717_{\pm.003} .605_{\pm.005} .652_{\pm.004} .727_{\pm.004} .555_{\pm.003} .629_{\pm.003} .723_{\pm.003} .605_{\pm.005} .659_{\pm.004} Multilang E5 L Ins .639_{\pm.012} .378_{\pm.003} .463_{\pm.004} .673_{\pm.026} .436_{\pm.022} .504_{\pm.013} .684_{\pm.001} .508_{\pm.003} .575_{\pm.002} .699_{\pm.004} .572_{\pm.014} .617_{\pm.010} .699_{\pm.001} .508_{\pm.003} .589_{\pm.002} .704_{\pm.008} .572_{\pm.014} .631_{\pm.006} LLaMA Nemotron 8B .634_{\pm.012} .365_{\pm.002} .452_{\pm.003} .626_{\pm.009} .462_{\pm.012} .521_{\pm.008} .690_{\pm.003} .505_{\pm.003} .576_{\pm.003} .679_{\pm.006} .597_{\pm.002} .630_{\pm.004} .706_{\pm.003} .505_{\pm.003} .589_{\pm.003} .683_{\pm.003} .597_{\pm.002} .637_{\pm.002} Nomic v2 .603_{\pm.011} .340_{\pm.004} .425_{\pm.005} .641_{\pm.011} .444_{\pm.008} .517_{\pm.007} .664_{\pm.004} .467_{\pm.005} .539_{\pm.003} .687_{\pm.004} .556_{\pm.006} .610_{\pm.005} .685_{\pm.004} .467_{\pm.005} .556_{\pm.003} .698_{\pm.003} .556_{\pm.006} .619_{\pm.005} Jina v4 .604_{\pm.010} .326_{\pm.003} .412_{\pm.002} .635_{\pm.009} .422_{\pm.007} .497_{\pm.006} .666_{\pm.004} .458_{\pm.004} .532_{\pm.004} .681_{\pm.006} .549_{\pm.005} .603_{\pm.001} .688_{\pm.004} .458_{\pm.004} .550_{\pm.003} .693_{\pm.006} .549_{\pm.005} .613_{\pm.002} Voyage v3 L .573_{\pm.006} .321_{\pm.003} .400_{\pm.004} .616_{\pm.010} .428_{\pm.012} .497_{\pm.008} .646_{\pm.003} .463_{\pm.002} .531_{\pm.002} .674_{\pm.009} .552_{\pm.008} .602_{\pm.004} .668_{\pm.003} .463_{\pm.002} .547_{\pm.002} .685_{\pm.007} .552_{\pm.008} .611_{\pm.004} ST5 XXL .575_{\pm.008} .326_{\pm.006} .408_{\pm.007} .638_{\pm.011} .418_{\pm.010} .496_{\pm.010} .654_{\pm.003} .452_{\pm.004} .526_{\pm.003} .690_{\pm.004} .541_{\pm.005} .601_{\pm.003} .679_{\pm.004} .452_{\pm.004} .543_{\pm.003} .703_{\pm.005} .541_{\pm.005} .611_{\pm.002} Table 33: Full summary of the classification metrics at test time for GoEmotions, sorted by F_{1} score of the MLP model in descending order. For each encoder, its rows refer to macro, weighted, and micro averages, respectively. LR \bm{k} -NN XGB MLP p r F_{1} p r F_{1} p r F_{1} p r F_{1} Gemini 001 \mathbf{.716} \mathbf{.517} \mathbf{.594} \mathbf{.624} \mathbf{.543} \mathbf{.575} \underline{\underline{.674}} \mathbf{.469} \mathbf{.546} \mathbf{.713} \mathbf{.529} \mathbf{.600} \mathbf{.735} \mathbf{.612} \mathbf{.664} \mathbf{.649} \mathbf{.631} \mathbf{.637} \underline{\underline{.714}} \mathbf{.579} \mathbf{.636} \underline{\underline{.719}} \mathbf{.623} \mathbf{.663} \mathbf{.743} \mathbf{.612} \mathbf{.671} \mathbf{.654} \mathbf{.631} \mathbf{.642} \underline{\underline{.724}} \mathbf{.579} \mathbf{.643} \underline{\underline{.723}} \mathbf{.623} \mathbf{.670} EmbeddingGemma \underline{.688} .493 .568 \underline{.599} .512 .546 \underline{\underline{.678}} \underline{.447} \underline{\underline{.533}} \underline{\underline{.700}} \underline{\underline{.517}} \underline{\underline{.590}} \underline{.722} .590 .645 .628 .606 .613 \underline{.710} .542 .610 \mathbf{.725} \underline{.610} \underline{\underline{.659}} \underline{.732} .590 .653 .635 .606 .620 \underline{\underline{.724}} .542 .620 \mathbf{.732} \underline{.610} \underline{\underline{.665}} OpenAI Text v3 L \underline{.687} .489 .564 .472 .423 .437 \underline{.645} .397 .483 .683 \underline{\underline{.518}} \underline{.579} .710 .587 .638 .512 .557 .528 .687 .502 .573 .705 .604 .647 .718 .587 .646 .529 .557 .542 .704 .502 .586 .706 .604 .651 Linq Mistral \underline{.685} \underline{.494} .568 .565 .461 .500 \underline{\underline{.666}} .408 .497 \underline{\underline{.700}} .494 .575 .707 .595 .643 .592 .566 .575 .704 .530 .597 \underline{\underline{.718}} .577 .637 .715 .595 .649 .603 .566 .584 \underline{.717} .530 .610 \underline{\underline{.727}} .577 .644 Qwen3 8B \underline{\underline{.700}} \underline{.496} \underline{.573} .582 .490 .525 \underline{.651} .425 .507 \underline{\underline{.703}} .498 .574 .718 \underline{.599} .649 .604 .574 .585 .701 .541 .605 \underline{\underline{.718}} .604 \underline{.651} .727 \underline{.599} .657 .614 .574 .593 .714 .541 .616 \underline{.721} .604 \underline{.658} KaLM v2 \underline{\underline{.705}} \underline{\underline{.515}} \underline{\underline{.588}} \underline{.600} .511 .544 \underline{\underline{.682}} \underline{\underline{.453}} \underline{\underline{.535}} \underline{\underline{.704}} .487 .565 \underline{.725} \underline{\underline{.608}} \underline{\underline{.658}} .627 .606 .611 \mathbf{.722} \underline{.563} \underline{\underline{.627}} \underline{\underline{.715}} .597 .642 \underline{.733} \underline{\underline{.608}} \underline{\underline{.665}} .632 .606 .618 \mathbf{.733} \underline{.563} \underline{\underline{.637}} \underline{\underline{.723}} .597 .654 Multilang E5 L Ins .674 .474 .549 .511 .496 .499 \underline{\underline{.674}} .392 .481 .675 .482 .550 .718 .568 .627 .546 .603 .571 .688 .506 .574 .702 .568 .616 \underline{.728} .568 .638 .552 .603 .576 .702 .506 .588 .711 .568 .631 LLaMA Nemotron 8B .637 .454 .521 .476 .405 .425 \mathbf{.683} .388 .480 .607 \underline{\underline{.512}} .549 .677 .580 .620 .523 .563 .536 .700 .513 .583 .676 .587 .623 .687 .580 .629 .535 .563 .549 .713 .513 .597 .674 .587 .628 Nomic v2 .669 .428 .514 .476 .410 .429 \underline{.639} .348 .438 .669 .460 .537 .703 .538 .602 .506 .536 .517 .672 .472 .543 .692 .559 .610 .717 .538 .615 .511 .536 .524 .690 .472 .560 .706 .559 .624 Jina v4 .663 .423 .508 .454 .395 .412 .631 .346 .434 .660 .436 .515 .695 .534 .598 .481 .524 .496 .673 .461 .536 .687 .547 .602 .708 .534 .609 .498 .524 .511 .696 .461 .555 .702 .547 .615 Voyage v3 L .635 .420 .498 .442 .382 .399 .568 .327 .406 .621 .447 .513 .685 .535 .594 .489 .529 .504 .648 .463 .530 .670 .556 .604 .701 .535 .607 .504 .529 .516 .671 .463 .547 .671 .556 .608 ST5 XXL .649 .424 .506 .449 .408 .420 .618 .350 .436 .668 .427 .510 .688 .520 .585 .496 .545 .514 .665 .455 .529 .678 .533 .589 .703 .520 .598 .512 .545 .528 .688 .455 .548 .691 .533 .602