Representation Decomposition for Learning Similarity and Contrastness Across Modalities for Affective Computing
Yuanhe Tian, Pengsen Cheng, Guoqing Jin, Lei Zhang, Yan Song · 2025
Read Representation Decomposition for Learning Similarity and Contrastness Across Modalities for Affective Computing by Yuanhe Tian, Pengsen Cheng, Guoqing Jin, Lei Zhang, Yan Song (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2506-07086.txt.
Description
Yuanhe Tian and four co-authors present an AI approach that reads feelings from image and text together, in this 2025 paper. Machines that judge mood from posts must weigh words against pictures, and the two often clash: a cheerful caption can sit on a grim photo. Simple fusion of the two streams misses such conflicts.
Their method first encodes and aligns the two streams with pretrained models. It then splits each stream's code into a shared part, carrying the common feeling, and a private part, carrying cues found in only one stream. An attention step blends the parts into a soft prompt for a large multimodal model. Tests covered aspect-based sentiment, emotion analysis, and hateful meme detection. The method beat strong rivals on all three tasks, which the authors credit to keeping shared and private cues apart.
Catalog record
| CREATOR | Yuanhe Tian, Pengsen Cheng, Guoqing Jin, Lei Zhang, Yan Song |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2506-07086