AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis

Dong She, Xianrong YaoLiqun ChenJinghe YuYang Gao, Zhanpeng JinSchool of Future Technology,South China University of Technology,Guangzhou {ftdshe,ftxryao,ftchenliqvn,ftyujinghe}@mail.scut.edu.cn{gaoyang2025,zjin}@scut.edu.cn · 2026

Read AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis by Dong She, Xianrong YaoLiqun ChenJinghe YuYang Gao, Zhanpeng JinSchool of Future Technology,South China University of Technology,Guangzhou {ftdshe,ftxryao,ftchenliqvn,ftyujinghe}@mail.scut.edu.cn{gaoyang2025,zjin}@scut.edu.cn (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-05900.txt.

Introduction

She and five colleagues test vision-language models on feeling in images with AICA-Bench, covering feeling grasp, reasoning, and guided image making. Trials of 23 models show weak intensity tuning and thin open answers. Their no-training Grounded Affective Tree prompts cut strength slips and add depth to answers, giving a strong base mark.

Catalog record

CREATORDong She, Xianrong YaoLiqun ChenJinghe YuYang Gao, Zhanpeng JinSchool of Future Technology,South China University of Technology,Guangzhou {ftdshe,ftxryao,ftchenliqvn,ftyujinghe}@mail.scut.edu.cn{gaoyang2025,zjin}@scut.edu.cn
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE68762
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.05900

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

AICA-Bench: Holistically Examining the Capabilities of VLMs in Affective Image Content Analysis Dong She, Xianrong YaoLiqun ChenJinghe YuYang Gao, Zhanpeng JinSchool of Future Technology,South China University of Technology,Guangzhou {ftdshe,ftxryao,ftchenliqvn,ftyujinghe}@mail.scut.edu.cn{gaoyang2025,zjin}@scut.edu.cn Abstract Vision-Language Models (VLMs) have demonstrated strong capabilities in perception, yet holistic Affective Image Content Analysis (AICA)—which integrates perception, reasoning, and generation into a unified framework—remains underexplored. To address this, we introduce AICA-Bench, a comprehensive benchmark comprising three core tasks: Emotion Understanding (EU), Reasoning (ER), and Generation (EGCG). We evaluate 23 VLMs, revealing critical gaps: models struggle with intensity calibration and suffer from descriptive shallowness in open-ended tasks. To bridge these gaps, we propose Grounded Affective Tree (GAT) Prompting, a training-free framework that integrates visual scaffolding with hierarchical reasoning. Experiments show that GAT effectively corrects intensity errors and significantly enhances descriptive depth, establishing a robust baseline for future affective multimodal research. 1 Introduction Recent years have witnessed rapid advances in Vision-Language Models (VLMs) Liu et al. 2023a ; Liu et al. 2024a ; Liu et al. 2024b ; Wang et al. 2024 ; Bai et al. 2025 ; Chen et al. 2025 ; Zhu et al. 2025a ; Yao et al. 2025 , which integrate visual and textual modalities to perform a wide range of tasks, from image captioning and visual question answering to grounded reasoning. To assess these capabilities, a variety of benchmarks Yue et al. 2025 ; Liu et al. 2024c ; Fu et al. 2024 ; Wang et al. 2023 have been developed, focusing primarily on factual correctness, semantic grounding, visual reasoning, or multi-discipline understanding. Yet emotional intelligence remains an underexplored but essential aspect of evaluating VLMs and multimodal large language models (MLLMs). Figure 1: Illustration of the three affective tasks in the AICA-Bench benchmark. In response, several recent benchmarks have started to explore the emotional capabilities of VLMs and MLLMs Yang et al. 2024 ; Bhattacharyya and Wang 2025 ; Lian et al. 2025 ; Gao et al. 2025 ; Song et al. 2024 , as summarized in Table 1 . Emo-Bench-M Yang et al. 2024 is a benchmark designed to evaluate the emotional intelligence (EI) capabilities of MLLMs. EVE Bhattacharyya and Wang 2025 introduces an image-based benchmark for emotion recognition, evaluating seven open-source VLMs across classification, grounding, and explanation tasks. AffectGPT Lian et al. 2025 proposes MER-UniBench, a benchmark designed for fine-grained multimodal emotion recognition, with a focus on video and dialogue understanding in MLLMs. EEmo-Bench Gao et al. 2025 focuses on image-evoked emotion assessment. MOSABench Song et al. 2024 introduces a benchmark for multi-object sentiment analysis in MLLMs. Benchmark Model Tasks #Datasets #AICA Datasets #Instr. #Models Prompt EVE Bhattacharyya and Wang 2025 VLM EU 5 5 8,009 7 B+CoT AffectGPT Lian et al. 2025 MLLM EU 9 3 - 17 B EEmo-Bench Gao et al. 2025 MLLM EU 1 1 6,773 19 B EmoBench-M Hu et al. 2025 MLLM EU 13 - 6,226 20 B MOSABench Song et al. 2024 MLLM EU 1 1 1,000 8 B AICA-Bench (Ours) VLM EU, ER, EGCG 9 9 18,124 23 B+CoT Table 1: Comparison of recent emotion benchmarks for VLMs and MLLMs. EU : Emotion Understanding, ER : Emotion Reasoning, EGCG : Emotion-guided Content Generation. #Instr. = total evaluation instructions; Prompt = B : Basic prompting, CoT : Chain-of-Thought prompting. Those benchmarks have made meaningful progress in evaluating the emotional capabilities of VLMs and MLLMs. However, they have yet to deeply investigate AICA, focusing on how emotions are interpreted, explained, and generated from visual content. As shown in Table 1 , most benchmarks include only a few image-based emotion datasets and focus primarily on basic emotion understanding tasks, typically framed as multiple-choice emotion classification. Yet prior work in affective computing and emotion psychology Picard 1997 ; Cambria 2016 highlights that emotional intelligence involves not only recognizing affective cues, but also reasoning about emotional causes and producing contextually appropriate affective expressions. The lack of comprehensive benchmarks for holistic A ffective I mage C ontent A nalysis with VLMs —spanning understanding, reasoning, and generation—is a critical bottleneck in advancing affective intelligence. To bridge this gap, we propose AICA-Bench (Sec 3 ), a holistic benchmark evaluating VLMs across three complementary dimensions: Emotion Understanding (EU) , Emotion Reasoning (ER) , and Emotion-Guided Content Generation (EGCG) . AICA-Bench integrates 9 diverse datasets and 18,124 standardized instructions to assess not only cue recognition but also causal explanation and empathetic expression. Using this benchmark, we conduct a comprehensive evaluation of 23 open- and closed-source VLMs under zero-shot settings. Beyond quantitative analysis (Sec 4 ), our detailed diagnostic error analysis uncovers two recurrent failure patterns(Sec 5 ): (1) Emotion Intensity Hallucination , where models frequently confuse high-arousal emotions (e.g., Amusement ) with low-arousal ones (e.g., Contentment ) due to weak visual grounding; and (2) Descriptive Shallowness , where generated responses suffer from generic, template-like content. In response, we introduce Grounded Affective Tree (GAT) Prompting (Sec 5 ). This training-free framework leverages visual scaffolding to steer models toward precise calibration and richer descriptive depth. Across the three tasks, GAT produces consistent improvements, with the EU Task increasing by 6.15 percentage points and the ER and EGCG Tasks increasing by 3.54 and 3.96 percentage points, respectively. 2 Related Work Vision-Language Models (VLMs) have rapidly advanced from early pretraining frameworks such as Flamingo Alayrac et al. 2022 and BLIP Li et al. 2022 to modern instruction-following systems capable of handling diverse multimodal tasks. Recent open-source representatives include the LLaVA family Liu et al. 2023a ; Liu et al. 2024a ; Liu et al. 2024b , the Qwen2-VL and Qwen2.5-VL series Wang et al. 2024 ; Bai et al. 2025 , and InternVL2.5/3 Chen et al. 2025 ; Zhu et al. 2025a , alongside compact models such as MiniCPM-V Yao et al. 2025 . Commercial systems such as GPT-4o and Gemini 2.5 further demonstrate the practical utility of this paradigm. Most existing benchmarks primarily evaluate factual perception, VQA, captioning, and instruction following Schwenk et al. 2022 ; Yin et al. 2024 ; Tu et al. 2024 ; Yue et al. 2025 ; Thapliyal et al. 2022 ; Liu et al. 2023b , leaving the affective dimension—how models perceive, interpret, and generate emotional content—largely understudied. Recent work has begun probing VLMs’ affective abilities Yang et al. 2024 ; Bhattacharyya and Wang 2025 ; Lian et al. 2025 ; Gao et al. 2025 ; Song et al. 2024 , but these studies typically rely on limited datasets and focus narrowly on emotion classification, without addressing affective reasoning or emotionally grounded generation. Motivated by holistic evaluation efforts such as HELM Liang et al. 2023a and VHELM Liang et al. 2023b , we introduce AICA-Bench , a holistic benchmark that systematically evaluates VLMs across the full spectrum of affective understanding, reasoning, and generation. Figure 2: The instruction curation pipeline of the AICA-Bench benchmark. It consists of two stages: (1) image sample selection and filtering from multiple affective datasets, and (2) automatic instruction generation across three tasks—emotion understanding (EU), emotion reasoning (ER), and emotion-guided content generation (EGCG). 3 The AICA-Bench Benchmark 3.1 Evaluation Tasks Design As discussed in the Introduction section, the existing affective benchmarks for VLMs focus primarily on emotional understanding tasks. However, emotional intelligence extends beyond perception, encompassing the ability to reason about

Read the complete text