From Prediction to Justification: Aligning Sentiment Reasoning with Human Rationale via Reinforcement Learning
Shihao Zhang 1 , Ziwei Wang 1 , Jie Zhou 1 , Yulan Wu 1 , Qin Chen 1 , Zhikai Lei 2 , Liyang Yu 1,3∗ , Liang Dou 2 , Liang He 2 1 School of Computer Science and Technology, East China Normal University, 2 Shanghai Qiji Zhifeng Co., Ltd. 3 Ocean University of China · 2026
Read From Prediction to Justification: Aligning Sentiment Reasoning with Human Rationale via Reinforcement Learning by Shihao Zhang 1 , Ziwei Wang 1 , Jie Zhou 1 , Yulan Wu 1 , Qin Chen 1 , Zhikai Lei 2 , Liyang Yu 1,3∗ , Liang Dou 2 , Liang He 2 1 School of Computer Science and Technology, East China Normal University, 2 Shanghai Qiji Zhifeng Co., Ltd. 3 Ocean University of China (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13398.txt.
Introduction
Shihao Zhang and eight colleagues build ABSA-R1, a tool that makes sentiment analysis reason before it predicts, like people who give reasons for a feeling. A reward model that lines up reasoning with the final label keeps the two in step. Rejection sampling picks hard cases where the model's own reasoning wobbles. On four benchmarks, explicit reasoning improves both clarity and scores in sentiment classification and triplet extraction over baselines that do not reason.
Catalog record
| CREATOR | Shihao Zhang 1 , Ziwei Wang 1 , Jie Zhou 1 , Yulan Wu 1 , Qin Chen 1 , Zhikai Lei 2 , Liyang Yu 1,3∗ , Liang Dou 2 , Liang He 2 1 School of Computer Science and Technology, East China Normal University, 2 Shanghai Qiji Zhifeng Co., Ltd. 3 Ocean University of China |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 29976 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.13398
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
From Prediction to Justification: Aligning Sentiment Reasoning with Human Rationale via Reinforcement Learning Shihao Zhang 1 , Ziwei Wang 1 , Jie Zhou 1 , Yulan Wu 1 , Qin Chen 1 , Zhikai Lei 2 , Liyang Yu 1,3∗ , Liang Dou 2 , Liang He 2 1 School of Computer Science and Technology, East China Normal University, 2 Shanghai Qiji Zhifeng Co., Ltd. 3 Ocean University of China Abstract While Aspect-based Sentiment Analysis (ABSA) systems have achieved high accuracy in identifying sentiment polarities, they often operate as "black boxes," lacking the explicit reasoning capabilities characteristic of human affective cognition. Humans do not merely categorize sentiment; they construct causal explanations for their judgments. To bridge this gap, we propose ABSA-R1, a large language model framework designed to mimic this “reason-before-predict" cognitive process. By leveraging reinforcement learning (RL), ABSA-R1 learns to articulate the why behind the what, generating natural language justifications that ground its sentiment predictions. We introduce a Cognition-Aligned Reward Model (formerly sentiment-aware reward model) that enforces consistency between the generated reasoning path and the final emotional label. Furthermore, inspired by metacognitive monitoring, we implement a performance-driven rejection sampling strategy that selectively targets hard cases where the model’s internal reasoning is uncertain or inconsistent. Experimental results on four benchmarks demonstrate that equipping models with this explicit reasoning capability not only enhances interpretability but also yields superior performance in sentiment classification and triplet extraction compared to non-reasoning baselines. Keywords: Sentiment; Aspect-based Sentiment Analysis; Reasoning; Reinforcement Learning Introduction Aspect-Based Sentiment Analysis (ABSA) is a fine-grained sentiment understanding task that aims to identify the sentiment polarity expressed toward specific aspects in a given text ( Zhou et al. 2019a ) . For example, in the sentence “The battery life is great, but the screen is dim," ABSA systems are expected to recognize that “battery life" is associated with positive sentiment, while “screen" elicits negative sentiment. Beyond simple classification, more advanced variants such as Aspect-Opinion-Sentiment Triplet Extraction (AOSTE) ( Xu et al. 2021c ; Chen et al. 2022b ) further require models to jointly extract aspect terms, opinion expressions, and their sentiment polarities, enabling a comprehensive cognitive mapping of subjective language. Figure 1: An example of sentiment thinking. Over the years, significant progress has been made in ABSA through both traditional supervised learning ( Pontiki et al. 2014a ; Mohammad et al. 2013a ) and pre-trained language models such as BERT and its domain-adapted variants ( Devlin et al. 2018a ) . These methods typically treat ABSA as a statistical classification or sequence labeling problem, achieving strong performance in label prediction ( Li et al. 2019a ) . Furthermore, large language models (LLMs) such as T5 ( Chung et al. 2022a ) and LLaMA ( MetaAI 2024a ) have been utilized to generate aspects, opinions, and sentiments via instruction tuning ( Scaria et al. 2024a ; Wang et al. 2024a ) . However, a critical cognitive gap remains: these models often operate as opaque black boxes. Unlike humans, who construct internal justifications before reaching a sentiment judgment, these models output labels without providing human-intelligible rationales. This lack of transparency limits their interpretability, failing to align with the explicability inherent in human affective cognition. Recently, reasoning LLMs equipped with chain-of-thought or reinforcement learning have achieved impressive performance in complex tasks across domains. These models do not merely predict outcomes; they articulate the logical steps leading to them, mirroring the “System 2” thinking process in human cognition. This progress motivates a critical question at the intersection of NLP and cognitive modeling: Can we endow LLMs with the ability to explicitly learn the affective reasoning processes humans use to attribute sentiment? For example, as shown in Fig. 1 , the model should provide a reasoning path to explain why a user feels positively about staff and decor, rather than just identifying what the sentiment is. In this work, we propose ABSA-R1, a novel sentiment reasoning LLM for ABSA. Trained via Reinforcement Learning (RL), our model adopts a “reason before prediction" cognitive paradigm, learning to generate natural language explanations that serve as the causal basis for its sentiment predictions. To guide this process, we design a cognition-aligned reward model (formerly sentiment-aware reward model) that jointly assesses the factual accuracy of the prediction and the logical coherence of the explanation, ensuring alignment between the generated rationale and the emotional label. Furthermore, we introduce a performance-driven rejection sampling strategy that functions as a metacognitive filter. This strategy dynamically identifies “hard” training samples—instances where the model’s internal reasoning is ambiguous or inconsistent—enabling more effective learning from cognitively challenging cases. Our main contributions are threefold: • We propose ABSA-R1, an RL-based LLM framework explicitly designed to mimic human-like sentiment reasoning in ABSA, generating interpretable justifications that ground its predictions. • We introduce a cognition-aligned reward model that ensures the synchronization of sentiment prediction with high-quality reasoning, alongside a performance-driven rejection sampling strategy to enhance robustness on ambiguous samples. • Extensive experiments on benchmark datasets demonstrate that ABSA-R1 achieves SOTA performance in both ABSC and AOSTE, while providing transparent, human-readable reasoning traces that bridge the gap between machine prediction and human understanding. Related Work Aspect-Based Sentiment Analysis Aspect-Based Sentiment Analysis (ABSA), a core task in fine-grained sentiment understanding, focuses on parsing the sentiment orientations towards different aspects in text ( Liu et al. 2024a ) . This task has evolved through several stages: from early rule- and lexicon-based methods with limited generalization ( Hu & Liu 2004a ) , to machine learning approaches leveraging SVMs and CRFs with handcrafted features ( Dong et al. 2014a ; Wang et al. 2016a ) , and more recently, deep learning models employing RNNs, attention mechanisms, and graph neural networks to capture contextual and syntactic dependencies ( Tang et al. 2016a ; Xue & Li 2018a ) . With the rise of pre-trained language models (PLMs), fine-tuning BERT-based architectures has become a dominant paradigm, significantly boosting performance across ABSA subtasks including aspect extraction, aspect sentiment classification (ASC), and aspect-opinion sentiment triplet extraction (ASTE) ( Xu et al. 2021c ; Chen et al. 2022b ; Mao et al. 2021a ) . More recent work further casts ABSA as a sequence-to-sequence generation problem using models like T5 or large language models (LLMs) such as ChatGPT and LLaMA, enabling unified multi-task frameworks ( Shen et al. 2021a ; Scaria et al. 2024a ) . Despite these advances, most existing methods remain focused on prediction accuracy and lack mechanisms for generating interpretable reasoning, offering no insight into why a particular sentiment is assigned, which limits their transparency and trustworthiness in real-world applications. Reasoning LLMs Recent advances in LLMs have demonstrated that explicit reasoning, particularly through Chain-of-Thought prompting, can significantly enhance performance on complex reasoning tasks by encouraging models to generate intermediate justifications before producing final answers ( Wei & Wang 2023a ; Kojima et al. 2023a ) . To train models for reasoning rather than mere pattern matching, reinforcement learning frameworks such as RLHF ( Ouyang & Wu 2022a ) and its variants (e.g., PPO ( Schulman et al. 2017a ) , DPO ( Rafailov & Sharma 2024a ) ) have been widely adopted. These methods optimize models based on learned or engineered reward signals that reflect human preferences or