A Multi-Model Approach to English-Bangla Sentiment Classification of Government Mobile Banking App Reviews

Md. Naim Molla 1 , Md Muhtasim Munif Fahim 1 , Md. Binyamin 2 , Md Jahid Hasan Imran 1 , Tonmoy Shil 1 , Nura Rayhan 3 , and Md Rezaul Karim, PhD 1 · 2026

Read A Multi-Model Approach to English-Bangla Sentiment Classification of Government Mobile Banking App Reviews by Md. Naim Molla 1 , Md Muhtasim Munif Fahim 1 , Md. Binyamin 2 , Md Jahid Hasan Imran 1 , Tonmoy Shil 1 , Nura Rayhan 3 , and Md Rezaul Karim, PhD 1 (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-13057.txt.

Introduction

Molla and six colleagues classify sentiment in 5,652 English and Bangla reviews of four Bangladeshi government banking apps, filtered from 11,414 raw reviews. Traditional models beat transformer-based ones: Random Forest reaches 0.815 accuracy and Linear SVM 0.804 weighted F1, both above fine-tuned XLM-RoBERTa. Reviewers complain most about slow transactions and poor interface design. eJanata rates worst, and a 16.1-point accuracy gap between Bangla and English text signals a need for low-resource language work.

Catalog record

CREATORMd. Naim Molla 1 , Md Muhtasim Munif Fahim 1 , Md. Binyamin 2 , Md Jahid Hasan Imran 1 , Tonmoy Shil 1 , Nura Rayhan 3 , and Md Rezaul Karim, PhD 1
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE22221
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.13057

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Multi-Model Approach to English-Bangla Sentiment Classification of Government Mobile Banking App Reviews Md. Naim Molla 1 , Md Muhtasim Munif Fahim 1 , Md. Binyamin 2 , Md Jahid Hasan Imran 1 , Tonmoy Shil 1 , Nura Rayhan 3 , and Md Rezaul Karim, PhD 1 Abstract For millions of users in developing economies who depend on mobile banking as their primary gateway to financial services, app quality directly shapes financial access. The study analyzed 5,652 Google Play reviews in English and Bangla (filtered from 11,414 raw reviews) for four Bangladeshi government banking apps. The authors used a hybrid labeling approach that combined use of the reviewer’s star rating for each review along with a separate independent XLM-RoBERTa classifier to produce moderate inter-method agreement ( κ = 0.459 \kappa=0.459 ). Traditional models outperformed transformer-based ones: Random Forest produced the highest accuracy (0.815), while Linear SVM produced the highest weighted F1 score (0.804); both were higher than the performance of fine-tuned XLM-RoBERTa (0.793). McNemar’s test confirmed that all classical models were significantly superior to the off-the-shelf XLM-RoBERTa ( p < 0.05 p<0.05 ), while differences with the fine-tuned variant were not statistically significant. DeBERTa-v3 was applied to analyze the sentiment at the aspect level across the reviews for the four apps; the reviewers expressed their dissatisfaction primarily with the speed of transactions and with the poor design of interfaces; eJanata app received the worst ratings from the reviewers across all apps. Three policy recommendations are made based on these findings—remediation of app quality, trust-centred release management, and Bangla-first NLP adoption—to assist state-owned banks in moving towards improving their digital services through data-driven methods. Notably, a 16.1-percentage-point accuracy gap between Bangla and English text highlights the need for low-resource language model development. I Introduction The use of mobile banking is increasing at an incredibly fast pace in Bangladesh as many customers of the four main state-owned banks (Sonali Bank, Agrani Bank, Janata Bank, Rupali Bank) have downloaded the banking applications to the Google Play Store and now have direct access to their bank accounts through this means [ 1 ] . These applications are particularly important for the large number of government employees, retirees and people living in rural areas who do not have the same level of access to private-sector banking as those in urban areas [ 2 ] . Reviews of users on the Google Play Store are a free source of information that can be used to determine how satisfied users are with the services they receive from the banks; however, they may also identify service failures and other issues [ 3 ] . The process of using automated methods to analyze user reviews of mobile banking applications poses several problems. First, user reviews of mobile banking applications often contain multiple languages including English, Bangla script and Romanized Bangla. As such, the effectiveness of most monolingual pipelines will depend upon the ability to translate the review into a language that can be processed by the pipeline; however, this can introduce additional error or “noise” [ 4 ] . Second, while there have been studies on the use of natural language processing (NLP) to analyze user reviews of mobile banking applications, to date, there has been no study that has analyzed user reviews of all four of the state-owned banking applications using a unified bilingual corpus and comparing the performance of both classical and transformer-based machine learning models to classify the sentiment expressed in the reviews. Much of the existing research in Bangla NLP related to fintech has focused on monolingual non-banking corpora [ 5 , 6 ] . This paper makes the following contributions: 1. A bilingual (English–Bangla) sentiment dataset created specifically for the purpose of evaluating user experience with the mobile banking applications of the four state-owned banks in Bangladesh, which was developed using a hybrid star-rating/model-validated labeling approach. 2. A systematic comparison of four classical machine learning models (Naïve Bayes, Linear SVM, Logistic Regression, Random Forest) against off-the-shelf (OTS) and fine-tuned XLM-RoBERTa [ 4 ] , with McNemar’s test [ 8 ] and 95% bootstrap confidence intervals to evaluate statistical significance. 3. Aspect-level sentiment analysis using DeBERTa-v3 [ 9 ] to evaluate the degree to which users were expressing positive or negative sentiment towards each of six service dimensions. 4. An evaluation of the degree to which there is a difference in performance between the English and Bangla reviews, and an examination of the longitudinal trend in sentiment expressed by users of the mobile banking applications of the four state-owned banks in Bangladesh from 2021–2025. 5. Actionable policy recommendations for state-owned banks based on the sentiment findings, including performance service level agreements (SLAs), trust-based release management, and the implementation of Bangla-first NLP pipelines to address linguistic equity. II Related Work Sentiment and NLP of app reviews. One of the biggest challenges with collecting data for app reviews is the gap between what users rate their apps (e.g., 1 or 5) and the written comments they leave about their experience with an app [ 10 , 11 ] . There has been a number of very successful models using BERT [ 12 ] and RoBERTa [ 13 ] on a wide variety of English review datasets [ 14 ] . However, Bangla, which also has many of these same characteristics (agglutinative morphology, frequent use of code-switching, etc.), presents a much greater challenge. While there are some notable examples of monolingual and multilingual efforts in this space (i.e., SentNoB [ 5 ] , BanglaBook [ 15 ] , and BanglaBERT [ 16 ] ), no team participating in the BLP-2023 shared task (71 total teams) [ 17 ] focused on the banking domain. Multilingual transformers and ABSA. The fact that the multilingual version of RoBERTa (XLM-RoBERTa [ 4 ] ) was trained on over 100 languages makes it easier to leverage knowledge gained from one language to improve model performance on other languages. This includes leveraging the Twitter version of XLM-RoBERTa (XLM-T [ 7 ] ) and additional prompt-tuning variations [ 18 ] to perform zero-shot polarity classification. Although the SemEval-2014 [ 19 ] remains the standard benchmark for ABSA research, recent results show that DeBERTa [ 9 ] ’s attention mechanism (which is specifically designed to be able to disentangle different levels of abstraction) can achieve top results on several ABSA benchmark tasks [ 20 ] , and that additional cross-lingual alignment techniques [ 21 ] and hybrid architectures (such as Instruct-DeBERTa [ 22 ] ) will likely continue to break new ground in the field. Yet, we could find no prior research that performs bilingual financial-service evaluation. Sentiment analysis of mobile banking apps. Prior research into banking app user feedback frequently focuses on three main aspects: (1) how easy it is to use the app, (2) whether the app appears trustworthy to the user, and (3) how reliable the app is to avoid crashing [ 23 ] . Prior research has also demonstrated the ability to identify common latent topics associated with banking app reviews using topic modelling approaches such as BERTopic and LDA [ 24 ] . Additionally, prior work demonstrates that negative spikes in user reviews are most commonly found after software updates

Read the complete text