LLaMA-Based Models for Aspect-Based Sentiment Analysis
Jakub Šmíd, Pavel Přibáň, Pavel Král University of West Bohemia, Faculty of Applied Sciences · 2025
Read LLaMA-Based Models for Aspect-Based Sentiment Analysis by Jakub Šmíd, Pavel Přibáň, Pavel Král University of West Bohemia, Faculty of Applied Sciences (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2508-08649.txt.
Introduction
Jakub Šmíd, Pavel Přibáň, and Pavel Král published this study in 2025. It tests language models on aspect-based sentiment analysis. That task asks not just whether a text is positive, but how it judges each aspect it names. Plain large models lag behind small tuned models on this compound task. The team asked what happens when the large models themselves get fine-tuned for it.
They tuned LLaMA-based models and tested them on four tasks across eight English data sets. One tuned model, based on Orca 2, beat the prior best results on every task. Yet all models did poorly when asked to work cold, with no or few examples. The paper ends with an error study showing where even the tuned models still stumble.
Catalog record
| CREATOR | Jakub Šmíd, Pavel Přibáň, Pavel Král University of West Bohemia, Faculty of Applied Sciences |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2508-08649
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
LLaMA-Based Models for Aspect-Based Sentiment Analysis Jakub Šmíd, Pavel Přibáň, Pavel Král University of West Bohemia, Faculty of Applied Sciences Abstract While large language models (LLMs) show promise for various tasks, their performance in compound aspect-based sentiment analysis (ABSA) tasks lags behind fine-tuned models. However, the potential of LLMs fine-tuned for ABSA remains unexplored. This paper examines the capabilities of open-source LLMs fine-tuned for ABSA, focusing on LLaMA-based models. We evaluate the performance across four tasks and eight English datasets, finding that the fine-tuned Orca 2 model surpasses state-of-the-art results in all tasks. However, all models struggle in zero-shot and few-shot scenarios compared to fully fine-tuned ones. Additionally, we conduct error analysis to identify challenges faced by fine-tuned models. 1 Introduction Aspect-based sentiment analysis (ABSA) aims to extract detailed sentiment information from text ( Zhang et al. 2022 ) . ABSA includes four sentiment elements: aspect term ( a ), aspect category ( c ), opinion term ( o ), and sentiment polarity ( p ). Given the example review “The steak was delicious” , the elements are “steak” , “food quality” , “delicious” and “positive” , respectively. Initially, ABSA research focused on extracting individual sentiment elements, e.g. aspect term extraction or aspect category detection ( Pontiki et al. 2014 ) . Recent research has transitioned towards compound tasks involving multiple sentiment elements, such as aspect sentiment triplet extraction (ASTE) ( Peng et al. 2020 ) , target aspect category detection (TASD) ( Wan et al. 2020 ) , aspect category opinion sentiment (ACOS) ( Cai et al. 2021 ) , and aspect sentiment quad prediction (ASQP) ( Zhang et al. 2021a ) . Table 1 shows the output formats of these ABSA tasks. Task Output Example output ASTE {( a , o , p )} {(“steak”, “delicious”, POS)} TASD {( a , c , p )} {(“steak”, food quality, POS)} ACOS {( a , c , o , p )} {(“steak”, food quality, “delicious”, POS)} ASQP {( a , c , o , p )} {(“steak”, food quality, “delicious”, POS)} Table 1: Output format for selected ABSA tasks for a review: “The steak was delicious” . ACOS focuses on implicit aspect and opinion terms in contrast to ASQP. Modern ABSA research often utilizes pre-trained language models, mainly focusing on sequence-to-sequence models. Compound ABSA tasks are typically formulated as text generation problems ( Zhang et al. 2021b ; Zhang et al. 2021a ; Gao et al. 2022 ; Hu et al. 2022 ; Gou et al. 2023 ) , which allows to solve compound ABSA tasks simultaneously. Lately, large language models (LLMs), such as ChatGPT ( OpenAI 2022 ) , LLaMA 2 ( Touvron et al. 2023b ) and Orca 2 ( Mitra et al. 2023 ) , have made significant progress across various natural language processing tasks. However, more traditional approaches that fine-tune Transformer-based models with sufficient data have shown superior performance over ChatGPT in compound ABSA tasks ( Zhang et al. 2023 ; Gou et al. 2023 ) . Additionally, fine-tuning LLMs on a single GPU is challenging due to their large number of parameters. Techniques like QLoRA ( Dettmers et al. 2023 ) address this challenge using a quantized 4-bit frozen backbone LLM with a small set of learnable LoRA weights ( Hu et al. 2021 ) . However, studies have yet to explore the capabilities of fine-tuned open-source LLMs for ABSA. This paper examines the unexplored potential of LLaMA-based models fine-tuned for English ABSA alongside their performance in zero-shot and few-shot scenarios. Our key contributions include: 1) Introducing the capabilities of fine-tuned LLaMA-based models for ABSA. 2) Conducting a comparative analysis of two LLaMA-based models against state-of-the-art results across four ABSA tasks and eight datasets. 3) Evaluating models’ performance in zero-shot, few-shot, and fine-tuning scenarios, demonstrating the superior performance of the fine-tuned Orca 2 model, surpassing state-of-the-art results across all datasets and tasks. 4) Presenting error analysis of the top-performing model. 1 1 1 Code and datasets are available at https://github.com/biba10/LLaMA-ABSA . 2 Related Work Early ABSA studies focused on predicting one or two sentiment elements ( Liu et al. 2015 ; Zhou et al. 2015 ; He et al. 2019 ; Cai et al. 2020 ) before progressing to more complex tasks involving triplets and quadruplets, such as ASTE ( Peng et al. 2020 ) , TASD ( Wan et al. 2020 ) , ASQP ( Zhang et al. 2021a ) and ACOS ( Cai et al. 2021 ) . Recent ABSA research focuses primarily on text generation initiated by GAS ( Zhang et al. 2021b ) . PARAPHRASE ( Zhang et al. 2021a ) converts labels to natural language. LEGO-ABSA ( Gao et al. 2022 ) explores multi-tasking, DLO ( Hu et al. 2022 ) optimizes element ordering, MVP ( Gou et al. 2023 ) combines differently ordered outputs, and Scaria et al. 2023 adopt instruction tuning. Gou et al. 2023 and Zhang et al. 2023 show that ChatGPT struggles with compound ABSA tasks in zero-shot and few-shot settings. Simmering and Huoviala 2023 report promising results with close-source LLMs for a single simple ABSA task. 3 Experimental Setup We employ the 7B and 13B versions of LLaMA 2 ( Touvron et al. 2023b ) and Orca 2 ( Mitra et al. 2023 ) models from the HuggingFace Transformers library 2 2 2 https://github.com/huggingface/transformers ( Wolf et al. 2020 ) . LLaMA 2 offers models of various sizes tailored for dialogue tasks, building upon the LLaMA framework ( Touvron et al. 2023a ) . Orca 2 extends this collection with enhanced reasoning capabilities. 3.1 Experimental Details For fine-tuning, we follow recommendations from Dettmers et al. 2023 and use QLoRA with the following settings: 4-bit NormalFloat (NF4) with double quantization and bf16 computation datatype, batch size of 16, constant learning rate of 2e-4, AdamW optimizer ( Loshchilov and Hutter 2019 ) , LoRA adapters ( Hu et al. 2021 ) on all linear Transformer block layers, and LoRA r=64 and \alpha=16 . We fine-tune the models for up to 5 epochs and choose the best-performing model based on validation loss. Following Mitra et al. 2023 , we compute loss only on tokens generated by the model, excluding the prompt with instructions. For zero-shot and few-shot experiments, we use 4-bit quantization of the models. Preliminary experiments indicated that 4-bit quantized models performed similarly to 8-bit quantized models and non-quantized models. All experiments, including zero-shot and few-shot scenarios, employ greedy search decoding and are conducted on an NVIDIA A40 with 48 GB GPU memory. 3.2 Evaluation Metrics We use micro F1-score as the primary evaluation metric, chosen based on related work, and report average results from 5 runs with different seeds. We consider a predicted sentiment tuple correct only if all its elements exactly match the gold tuple. 3.3 Tasks & Datasets We evaluate the LLMs on four tasks: two involving quadruplets (ASQP and ACOS) and two involving triplets (TASD and ASTE). We select two datasets for each task and use the same data splits as previous works for a fair comparison. Table 1 displays the output targets for each task. We use Rest15 and Rest16 datasets for ASQP in the restaurant domain, initially introduced in SemEval tasks