Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales
Jun Cao, Jiyi Li, Ziwei Yang, Renjie Zhou✉ · 2025
Read Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales by Jun Cao, Jiyi Li, Ziwei Yang, Renjie Zhou✉ (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-14499.txt.
Introduction
Jun Cao, Jiyi Li, Ziwei Yang, and Renjie Zhou wrote this paper, published in two thousand twenty-five, on reading feelings about specific things in posts that mix text and images. A review may praise a phone's camera but pan its battery, and the image matters too.
Small trained models do this job but often miss fine links between words, pictures, and feelings. Large language models read such detail well yet still trail tuned small models on this task. The team's framework, LRSA, feeds explanations written by a large model into a small one as extra hints. A dual cross-attention step then blends the text and image features more deeply. Across three standard benchmarks, the blend beat both baselines. The authors say the pattern should lift most small models used for this task.
Catalog record
| CREATOR | Jun Cao, Jiyi Li, Ziwei Yang, Renjie Zhou✉ |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 34 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-14499
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales Jun Cao, Jiyi Li, Ziwei Yang, Renjie Zhou✉ Abstract There has been growing interest in Multimodal Aspect-Based Sentiment Analysis (MABSA) in recent years. Existing methods predominantly rely on pre-trained small language models (SLMs) to collect information related to aspects and sentiments from both image and text, with an aim to align these two modalities. However, small SLMs possess limited capacity and knowledge, often resulting in inaccurate identification of meaning, aspects, sentiments, and their interconnections in textual and visual data. On the other hand, Large language models (LLMs) have shown exceptional capabilities in various tasks by effectively exploring fine-grained information in multimodal data. However, some studies indicate that LLMs still fall short compared to fine-tuned small models in the field of ABSA. Based on these findings, we propose a novel framework, termed LRSA, which combines the decision-making capabilities of SLMs with additional information provided by LLMs for MABSA. Specifically, we inject explanations generated by LLMs as rationales into SLMs and employ a dual cross-attention mechanism for enhancing feature interaction and fusion, thereby augmenting the SLMs’ ability to identify aspects and sentiments. We evaluated our method using two baseline models, numerous experiments highlight the superiority of our approach on three widely-used benchmarks, indicating its generalizability and applicability to most pre-trained models for MABSA. 1 Introduction There has been growing interest in Multimodal Aspect-Based Sentiment Analysis (MABSA) as a complex fine-grained task within sentiment analysis [ 13 , 7 ] . The MABSA task encompasses three primary objectives: (1) Multimodal Aspect Term Extraction (MATE), which is focused on identifying aspect terms with sentiment polarity from a sentence within an image-text pair; (2) Multimodal Aspect-Based Sentiment Classification (MASC), which focuses on determine the sentiment polarity for each identified aspect; (3) Multimodal Aspect-Based Sentiment Analysis (MABSA), which entails jointly extracting all aspect terms and predicting their sentiment polarity from the image-text pair [ 7 ] . Despite significant advancements, aspect extraction and sentiment polarity prediction in this fine-grained sentiment analysis scenario with multimodal information remain challenging for current models. Firstly, the semantic complexity of sentences poses difficulties in identifying aspects and comprehending their corresponding sentiment. Secondly, images often contain abundant detailed information, and models frequently struggle to focus on all key information in the image or mistakenly pay attention to irrelevant areas, introducing noise. Lastly, it is challenging for the model to capture the overall connection between images and text, particularly when judging the connection between more specific image regions and textual vocabulary. Recent approaches often employ pre-trained small language models (SLMs) to provide representations and understanding of image-text pairs [ 11 , 32 ] . These approaches also incorporate various modules to assist the small models in comprehending information from image-text pairs. Other enhancement methods include contrastive learning and image-text matching to align semantics and mitigate modality gaps [ 27 ] . While SLMs have demonstrated improvements, their knowledge and limitations also influence the further improvement of the models. Incorporating external knowledge is essential to strengthen the SLMs’ understanding of image-text pairs. Some methods utilize image captions generated by ClipCap [ 28 , 15 ] as image prompts to assist image comprehension. However, this external information is often simplistic and crude, as image captions usually lack a relationship with sentiments and provide limited assistance in fine-grained aspect extraction and sentiment judgment. Figure 1: An example of the LLM processing image-text pairs to generate the explanation of images and text. Large Language Models (LLMs) possess powerful abilities to understand, track, and generate complex language [ 16 , 1 ] . They have made significant impressions in various tasks [ 23 ] and are considered promising solutions for general-purpose tasks [ 14 ] . LLMs are typically trained on large-scale text corpora, granting them richer knowledge and the ability to mine fine-grained information, subtle patterns, and relationships from data. Nevertheless, studies have demonstrated that LLMs’ performance on the ABSA task is not as favorable as fine-tuned small models. We believe that LLMs are yet incapable of selecting and integrating received information to draw conclusions and make judgments in this fine-grained task. However, by employing SLMs’ flexible task-specific learning and utilizing LLMs’ generated information as informative rationales, we can combine the strengths of small and large models. LLM leverages its robust knowledge repository, superior information retrieval, as well as reasoning capabilities to generate insightful rationales. These rationales serve as external sources that the small model can refer to for making informed judgments. With these considerations, we propose a novel framework LRSA that combines the decision-making abilities of SLMs with the additional information provided by LLMs for MABSA. Specifically, we use LLMs to generate rationales for understanding image-text pairs and their connection as shown in Figure 1 , and to inject this information in SLMs as the connection of the SLMs and LLMs. Inside SLMs, to further improve the model’s performance, we employ a dual cross-attention mechanism to fuse the rationales with the image-text pair, emphasizing the useful information related to the current task and eliminating unnecessary noise. The image and text features refer to the rationale features in the fusion process to enhance them, making the fused feature representation more comprehensive and accurate, thus improving the understanding ability and predictive performance of the SLM. In the result generation stage, we connect the fused feature of the image-text pairs with the rationale feature to provide more information for the SLMs to make judgments. Our contributions can be summarized as follows: 1) We propose a novel framework LRSA that combines the decision-making capabilities of SLMs with the additional information provided by LLMs for MABSA, by using rationales generated by LLMs for SLMs to reference. We deeply integrate LLMs in the field of MABSA. 2) To better capture fine-grained information and enhance feature representation, we employ a dual cross-attention mechanism to separately fuse image features with the rationales features of LLMs’ understanding of images, and text features with the rationales features of LLMs’ understanding of texts. 3) We evaluated our method using two baseline models, experimental results on two benchmark datasets show that our method surpasses those of the baseline models, indicating its generalizability and applicability to most pre-trained models for MABSA. 2 Related works 2.1 Aspect-based Sentiment Analysis Aspect-based Sentiment Analysis (ABSA) aims to predict the sentiment polarity of aspect terms within sentences as a fine-grained task. Benefiting from the excellent performance of language models like BERT [ 5 ] and RoBERTa [ 12 ] in NLP, most recent works use pre-trained language models to to capture the semantic relationship between a given aspect and its context [ 19 , 10 ] . In addition, constructing aspect-oriented dependency trees can assist in linking aspect words with opinion words and learning the syntactic feature representation of aspects to to enhance the performance of language models further [ 22 , 21 ] . More recently, studies have also adopted end-to-end models for extracting all aspects and sentiment elements in triplets.- [ 17 , 3 ] 2.2 Multimodal Aspect-based Sentiment Analysis As social media rapidly expands, Multimodal Aspect-Based Sentiment Analysis has garnered significant research interest in recent years. Ju et al. [ 7 ] were the first to implement MABSA within a cohesive framework,