Benchmarking Logistic Regression, SVM, and LightGBM Against BiLSTM with Attention for Sentiment Analysis on Indonesian Product Reviews
Razin Hafid Hamdi, Ivana Margareth Hutabarat, Hanna Gresia Sinaga, Luluk Muthoharoh, Ardika Satria, Martin C.T. Manullang · 2026
Read Benchmarking Logistic Regression, SVM, and LightGBM Against BiLSTM with Attention for Sentiment Analysis on Indonesian Product Reviews by Razin Hafid Hamdi, Ivana Margareth Hutabarat, Hanna Gresia Sinaga, Luluk Muthoharoh, Ardika Satria, Martin C.T. Manullang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-25452.txt.
Introduction
Hamdi and five colleagues benchmark classic machine learning against BiLSTM with Attention for Indonesian product review sentiment. The set holds 19,728 reviews, split evenly between positive and negative. For classic learning, Logistic Regression, Support Vector Machine, and LightGBM were tried with 10-fold cross-checks, with Logistic Regression best at 97.26 percent accuracy and the same F1-score. The BiLSTM with Attention model, on 3,946 held-out test reviews, scored a touch lower on both measures. Classic learning thus rivals, and slightly beats, deeper sequential models, while running cheaper.
Catalog record
| CREATOR | Razin Hafid Hamdi, Ivana Margareth Hutabarat, Hanna Gresia Sinaga, Luluk Muthoharoh, Ardika Satria, Martin C.T. Manullang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 16175 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.25452
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Benchmarking Logistic Regression, SVM, and LightGBM Against BiLSTM with Attention for Sentiment Analysis on Indonesian Product Reviews Razin Hafid Hamdi, Ivana Margareth Hutabarat, Hanna Gresia Sinaga, Luluk Muthoharoh, Ardika Satria, Martin C.T. Manullang Abstract Sentiment analysis of product reviews on e-commerce platforms plays a critical role in automatically understanding customer satisfaction and providing actionable insights for sellers seeking to improve product quality. This paper presents a comprehensive benchmarking study comparing a Machine Learning (ML) approach via the PyCaret AutoML framework against a Deep Learning (DL) approach based on a Bidirectional Long Short-Term Memory (BiLSTM) architecture with an Attention mechanism for binary sentiment classification on Indonesian product reviews. The dataset comprises 19,728 samples balanced equally between positive and negative reviews. For the ML approach, three prominent algorithms were evaluated via 10-fold stratified cross-validation: Logistic Regression (LR), Support Vector Machine (SVM) with a linear kernel, and Light Gradient Boosting Machine (LightGBM). Logistic Regression achieved the best ML performance with an accuracy of 97.26% and an F1-score of 97.26%. The BiLSTM with Attention model, evaluated on 3,946 held-out test samples, achieved an accuracy of 97.24% and an F1-score of 97.24%. These comparative results demonstrate that traditional ML algorithms with proper preprocessing and feature extraction can compete closely with, and even marginally outperform, more complex sequential DL architectures on high-dimensional datasets, while simultaneously offering greater computational efficiency. Keywords Sentiment Analysis \cdot PyCaret \cdot BiLSTM \cdot Attention Mechanism \cdot Indonesian Product Reviews \cdot Machine Learning 1 Introduction The rapid growth of the e-commerce industry in Indonesia has generated an enormous volume of textual data in the form of product reviews. These reviews contain customer opinions that are highly valuable for businesses to evaluate their products. However, manually analyzing thousands of reviews is both inefficient and prone to subjective bias. Consequently, the application of Natural Language Processing (NLP) techniques for automated sentiment classification has become a crucial solution. A primary challenge in processing Indonesian product reviews is the prevalent use of colloquial language, slang ( bahasa gaul ), abbreviations, and irregular sentence structures. Various NLP studies have compared traditional Machine Learning (ML) and Deep Learning (DL) approaches to address these challenges. Traditional ML approaches require careful feature engineering, yet the emergence of AutoML frameworks such as PyCaret ( 1 ) has substantially streamlined the ML workflow by automating preprocessing, hyperparameter tuning, and systematic model evaluation at scale. In contrast, Deep Learning models have demonstrated significant advantages across a range of sequence processing tasks. Bidirectional Long Short-Term Memory (BiLSTM) networks can learn text representations from both forward and backward directions to comprehensively retain contextual information. The addition of an Attention mechanism ( 2 ) further enables the model to assign higher weights to specific tokens that most significantly contribute to the sentiment polarity, regardless of their position in the sentence. This study aims to conduct a comprehensive benchmark between leading ML algorithms in a PyCaret AutoML environment—specifically Logistic Regression, SVM, and LightGBM—against a state-of-the-art DL architecture (BiLSTM with Attention) implemented using PyTorch. We compare evaluation metrics including Accuracy, Precision, Recall, and F1-Score to determine which approach is most optimal for handling a 19,728-sample dataset of Indonesian product reviews. The main contributions of this work are: (1) a rigorous empirical comparison of ML and DL methods on an Indonesian-language e-commerce sentiment dataset; (2) a demonstration that classical linear models can remain highly competitive when paired with appropriate feature engineering; and (3) a fully reproducible experimental pipeline deployed on Hugging Face Spaces. 2 Related Work A body of research has examined the effectiveness of Machine Learning and Deep Learning for Indonesian sentiment analysis. In the traditional ML domain, Support Vector Machines (SVM) and Logistic Regression have consistently served as strong baselines for text classification due to their robustness in high-dimensional feature spaces ( 9 ; 11 ) . The use of PyCaret as an AutoML library has gained popularity for its ability to benchmark dozens of models—from LR and Naive Bayes to SVM, Random Forest, and LightGBM—automatically, thereby minimizing experimental cycle errors and accelerating prototyping time ( 1 ; 8 ) . In the Deep Learning domain, RNN-based architectures, particularly LSTM networks introduced by Hochreiter & Schmidhuber ( 4 ) , are widely applied for their capacity to model long-term dependencies in sequential text. The BiLSTM extension has been shown to further improve accuracy by observing sequences from two directions ( 6 ) . Additionally, the integration of the Attention mechanism ( 2 ) ensures the model focuses on the hidden states representing the most sentiment-bearing tokens within a sentence. Comparative studies between ML and DL for Indonesian sentiment analysis on e-commerce data have shown mixed results ( 7 ; 10 ) . LightGBM, introduced by Ke et al. ( 5 ) , has shown competitive performance in tabular and high-dimensional feature settings, making it a natural candidate for TF-IDF-transformed text features. Our work extends this line of inquiry by providing a controlled, head-to-head comparison using a modern AutoML pipeline alongside a custom-built DL model under identical dataset conditions. 3 Dataset The dataset used in this study originates from customer product reviews collected from Indonesian marketplace platforms, publicly available on Hugging Face Datasets ( 3 ) . The total dataset comprises 19,728 rows with a binary target label (Positive/Negative). The data is split using a percentage validation technique, yielding 15,782 samples (80%) for the training set and 3,946 samples (20%) for the test set. The dataset is near-perfectly balanced, with the test split containing exactly 1,973 Negative-class reviews and 1,973 Positive-class reviews, ensuring that accuracy metrics are directly comparable and unbiased by class imbalance. 3.1 Preprocessing The preprocessing pipeline applied to raw text consists of the following sequential steps: 1. Case Folding: All characters are converted to lowercase. 2. Cleansing: URLs (matching patterns http:// , https:// , www ), HTML tags, mentions ( @user ), hashtags ( #topic ), punctuation, and numeric characters are removed using regular expressions. 3. Slang Normalization: Informal Indonesian words are mapped to their standard forms using a curated slang dictionary (e.g., bgt \rightarrow banget ). This step is critical for handling the high prevalence of colloquial language in user-generated reviews. 4. Tokenization: Text is split into tokens based on whitespace boundaries. Tokens with fewer than 2 characters are subsequently filtered out. 5. Whitespace Normalization: Redundant whitespace is removed to produce a clean token sequence. 3.2 Feature Extraction for ML For the Machine Learning pipeline, cleaned text was transformed into numerical features using TF-IDF vectorization with the following configuration: max_features=5000 , ngram_range=(1,2) (unigrams and bigrams), min_df=3 , max_df=0.90 , and sublinear_tf=True . This yielded a sparse feature matrix of dimension N\times 5000 as input to the PyCaret classifiers. 4 Methodology 4.1 Machine Learning Pipeline (PyCaret) The PyCaret classification module was used to orchestrate the ML experimental pipeline. The setup() function was initialized with train_size=0.8 and configured to include automatic missing-value imputation and MinMax normalization. We focused on benchmarking three algorithms: • Logistic Regression (LR): A linear estimator that efficiently maps features to discrete class probabilities via a sigmoid function. • Support Vector Machine (SVM): Configured with a linear kernel, trained to maximize the separating margin between classes in the high-dimensional TF-IDF feature space. • Light Gradient Boosting Machine (LightGBM): A gradient boosting decision tree framework