SENTIMENT ANALYSIS OF INDONESIAN SPOTIFY REVIEWS USING MACHINE LEARNING AND BILSTM
Uliano Wilyam Purba, Andre Hadiman Rotua Parhusip, Sahid Maulana, Luluk Muthoharoh M. Si, Ardika Satria M. Si, Martin C.T. Manullang Ph.D. · 2026
Read SENTIMENT ANALYSIS OF INDONESIAN SPOTIFY REVIEWS USING MACHINE LEARNING AND BILSTM by Uliano Wilyam Purba, Andre Hadiman Rotua Parhusip, Sahid Maulana, Luluk Muthoharoh M. Si, Ardika Satria M. Si, Martin C.T. Manullang Ph.D. (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-03443.txt.
Introduction
Uliano Wilyam Purba, Andre Hadiman Rotua Parhusip, Sahid Maulana, Luluk Muthoharoh, Ardika Satria, and Martin C. T. Manullang benchmark sentiment methods on Indonesian Spotify reviews, a three-class task over one hundred thousand scraped comments. After cleaning, about seventy thousand samples remained for comparing classic models with a two-layer bidirectional LSTM under one shared preprocessing pipeline.
The decision tree led the classic models, while the BiLSTM earned the best weighted F1 overall yet failed on the small neutral class. Oversampling with SMOTE gave the classic route more balanced three-class behavior. The authors conclude each approach wins on a different criterion.
Catalog record
| CREATOR | Uliano Wilyam Purba, Andre Hadiman Rotua Parhusip, Sahid Maulana, Luluk Muthoharoh M. Si, Ardika Satria M. Si, Martin C.T. Manullang Ph.D. |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 20 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.03443
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
SENTIMENT ANALYSIS OF INDONESIAN SPOTIFY REVIEWS USING MACHINE LEARNING AND BILSTM Uliano Wilyam Purba, Andre Hadiman Rotua Parhusip, Sahid Maulana, Luluk Muthoharoh M. Si, Ardika Satria M. Si, Martin C.T. Manullang Ph.D. Abstract This paper compares three Scikit-learn classifiers — Support Vector Machine (SVM), Multinomial Naive Bayes (MNB), and Decision Tree (DT) — with a two-layer Bidirectional Long Short-Term Memory (BiLSTM) model for three-class sentiment classification of Indonesian Spotify reviews. From 100,000 scraped reviews, 70,155 cleaned samples are used for the machine learning track, while the BiLSTM is trained on a stratified 20,000-sample subset because training is performed in a CPU-only environment. Both tracks use the same preprocessing pipeline: slang normalization, stopword removal, and Sastrawi stemming. In 5-fold cross-validation, Decision Tree is the strongest classical model with weighted F1-score 0.7269, ahead of SVM (0.6958) and MNB (0.4619). On its held-out test set, BiLSTM reaches weighted F1-score 0.8069, but fails on the minority Neutral class (F1 = 0.000). These results show that BiLSTM provides stronger overall performance, whereas ML with SMOTE yields more balanced three-class recognition. Keywords Sentiment Analysis \cdot Indonesian NLP \cdot BILSTM \cdot TF-IDF \cdot Decision Tree \cdot Google Play Store Reviews \cdot PyCaret \cdot SMOTE \cdot Class Imbalance \cdot Hugging Face Deployment 1 Introduction Sentiment analysis is valuable for app developers because user reviews reveal satisfaction, complaints, and feature requests at scale. For Indonesian reviews, this task is harder because the text is highly informal, rich in slang, and often inconsistent in spelling. Spotify reviews from the Google Play Store provide a relevant case study for comparing classical machine learning and deep learning on Indonesian text. In this work, SVM, Multinomial Naive Bayes, and Decision Tree with TF-IDF are benchmarked against a two-layer BiLSTM. The comparison is practical rather than perfectly symmetric: classical models use the full 70,155 cleaned samples, while BiLSTM is trained on a 20,000-sample stratified subset due to CPU-only constraints. This study contributes: (1) a shared Indonesian preprocessing pipeline for both tracks; (2) a 5-fold benchmark of three classical classifiers with TF-IDF and SMOTE; (3) evaluation of a compact BiLSTM with 1,096,579 parameters; and (4) a transparent comparison of overall and per-class performance, supported by public deployment of both models. • ML demo: https://huggingface.co/spaces/tubespba-kelompoktuwir/sentimen-spotify-ml • DL demo: https://huggingface.co/spaces/tubespba-kelompoktuwir/sentimen-spotify-dl 2 Related Work 2.1 Sentiment Analysis: Foundational Work Pang and Lee [ 9 ] established the main tasks and evaluation principles of sentiment analysis. Their survey also showed that classical supervised models with bag-of-words or TF-IDF features remain strong baselines for document-level polarity classification. 2.2 Classical Machine Learning for Text Classification TF-IDF combined with SVM, Multinomial Naive Bayes, and tree-based models is widely used in sentiment classification. Prior studies on app reviews and social media show that SVM is often strong, while performance still depends heavily on feature design, label balance, and language characteristics [ 10 , 11 ] . 2.3 Deep Learning for Sentiment Analysis LSTM and BiLSTM models are designed to capture sequential context more effectively than bag-of-words features. Prior work reports that BiLSTM performs well for sentiment tasks and can outperform classical methods on Indonesian text, although the gain usually comes with higher computational cost [ 5 , 13 , 7 ] . 2.4 Indonesian NLP and Sentiment Analysis Indonesian NLP research has improved through resources such as IndoNLU and IndoBERT [ 12 ] , yet sentiment analysis still faces challenges from informal spelling, slang, and morphology. Previous Indonesian studies also report that BiLSTM is competitive, especially when preprocessing is carefully tailored to the language [ 6 , 4 , 1 , 2 ] . 2.5 App Review Sentiment Analysis App-review sentiment analysis is an applied setting where deep models often perform well, but class imbalance remains a persistent issue [ 8 , 3 ] . To the best of our knowledge, a direct benchmark between Scikit-learn classifiers and a PyTorch BiLSTM on large-scale Indonesian Spotify reviews with shared preprocessing has not been previously reported. 3 Dataset 3.1 Data Source and Collection The dataset consists of 100,000 Indonesian-language Spotify reviews scraped from the Google Play Store using the google-play-scraper library. For modeling, only review text and star rating are used. The raw dataset includes reviewer name, review text, rating, date, likes, and application version. Only the Ulasan and Rating columns are retained for sentiment classification. 3.2 Sentiment Label Construction Following common rating-based labeling, ratings 1–2 are mapped to Negatif, rating 3 to Netral, and ratings 4–5 to Positif. This scheme matches the natural interpretation of app-store reviews. 3.3 Dataset Statistics and Class Distribution Figure 1: Sentiment distribution rating label from raw data. The dataset is strongly imbalanced, with positive reviews dominating the corpus. After label mapping, deduplication, and removal of empty reviews, the cleaned dataset contains 70,155 samples: Positif 47,991 (68.4%), Negatif 18,026 (25.7%), and Netral 4,138 (5.9%). Table 1: Dataset Statistics and Class Distribution Split Positive Netral Negative Total Full cleaned dataset 47,991 (68.4%) 4,138 (5.9%) 18,026 (25.7%) 70,155 ML training set 47,991 4,138 18,026 70,155 DL subset (20K stratified) 13,681 1,180 5,139 20,000 DL Train 9,852 851 3,700* 14,219* DL Validation \sim 1,739 \sim 150 \sim 653* 2,510* DL Test \sim 2,090 \sim 179 \sim 786* 2,953 Frequent negative themes include advertisements, premium access issues, and playback or lyric synchronization problems. These topics also appear in the cleaned vocabulary used by both tracks. 3.4 Preprocessing Pipeline A unified preprocessing pipeline is applied to both tracks: 1. Lowercasing: all text is converted to lowercase. 2. Noise removal: URLs, mentions, hashtags, punctuation, symbols, and digits are removed. 3. Slang normalization: 124 informal forms are mapped to standard Indonesian (e.g., gak \rightarrow tidak). 4. Stopword removal: 809 general and domain-specific stopwords are filtered out. 5. Morphological stemming: tokens are reduced to root forms using Sastrawi [11]. After preprocessing, the median review length is 4 tokens and the 95th percentile is 17 tokens. Therefore, the BiLSTM maximum sequence length is set to 17, and reviews that become empty after cleaning are removed. Table 2: Preprocessing Pipeline Summary Step Method Detail Lowercasing Python str.lower() All characters converted to lowercase Noise removal Regex substitution URLs, @mentions, punctuation, digits removed Slang normalization Custom dictionary 124 entries (e.g., gak \rightarrow tidak ) Stopword removal Sastrawi + custom 809 stopwords including domain terms Morphological stemming Sastrawi (ECS) 40+ suffix/prefix rules for Indonesian 4 Methodology This study follows two parallel tracks — machine learning (ML) and deep learning (DL) — with the same preprocessing pipeline. 4.1 Machine Learning Track (Scikit-learn) 4.1.1 Feature Extraction: TF-IDF Preprocessed reviews are converted into TF-IDF vectors with a maximum of 3,000 features [ 9 ] . 4.1.2 Class Balancing: SMOTE Because the dataset is imbalanced, SMOTE [ 3 ] is applied only to the training fold in each cross-validation split. 4.1.3 Classifiers Three classifiers are compared: SVM, Multinomial Naive Bayes, and Decision Tree. All are implemented in Scikit-learn and evaluated with stratified 5-fold cross-validation on the cleaned dataset. 4.2 Deep Learning Track (BiLSTM) 4.2.1 Tokenization and Vocabulary Texts in the DL track are tokenized at word level. The training split produces a 3,285-token vocabulary, and all sequences are padded or truncated to length 17. 4.2.2 Model Architecture The BiLSTM model uses 128-dimensional embeddings, a two-layer bidirectional LSTM with 128 hidden units per direction, dropout, and a fully