Bridging Emotions and Architecture: Sentiment Analysis in Modern Distributed Systems
Mahak Shah, Akaash Vishal Hazarika, Meetu Malhotra, Sachin C. Patil, Joshit Mohanty · 2025
Read Bridging Emotions and Architecture: Sentiment Analysis in Modern Distributed Systems by Mahak Shah, Akaash Vishal Hazarika, Meetu Malhotra, Sachin C. Patil, Joshit Mohanty (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-18260.txt.
Introduction
Mahak Shah and four co-authors study sentiment analysis on distributed systems in a 2025 paper. Reading emotion in text serves social media watch, customer feedback, and market research. Data volumes in those fields are huge. Splitting the work across many machines is a natural response.
The paper surveys how the two fields meet, methods first, then hurdles. An experiment trains sentiment models on one machine and on a cluster. Speed and accuracy are compared head to head for both setups. The authors weigh where distribution pays off and where it does not. They close with open problems for future work. The study serves teams that must scale text analysis without losing quality.
Catalog record
| CREATOR | Mahak Shah, Akaash Vishal Hazarika, Meetu Malhotra, Sachin C. Patil, Joshit Mohanty |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 17 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-18260
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Bridging Emotions and Architecture: Sentiment Analysis in Modern Distributed Systems Mahak Shah, Akaash Vishal Hazarika, Meetu Malhotra, Sachin C. Patil, Joshit Mohanty Abstract Sentiment analysis is a field within NLP that has gained importance because it is applied in various areas such as; social media surveillance, customer feedback evaluation and market research. At the same time, distributed systems allow for effective processing of large amounts of data. Therefore, this paper examines how sentiment analysis converges with distributed systems by concentrating on different approaches, challenges and future investigations. Furthermore, we do an extensive experiment where we train sentiment analysis models using both single node configuration and distributed architecture to bring out the benefits and shortcomings of each method in terms of performance and accuracy. I Introduction At present, there is overwhelming textual information available on the internet. Because of such sheer amount of data, we ought to have proper data processing methods. Sentiment analysis refers to the use of scripted information to ascertain a person’s attitude towards an event or phenomenon. Often, the amount and diversity of structured information is greater than what traditional processing technologies are capable of processing, leading to the need for distributed systems. With a common goal being reached through multiple applications running from different computers in the network, there is an opportunity for more experience in data processing. This paper will explain why it is important to consider combining sentiment analysis with some aspects of distributed systems that help to analyze large datasets. The distributed approach will be compared against a single node architecture. The comparison will be done on the basis of performance, accuracy and resource utilization to show the effectiveness of the proposed approach. II Literature Review II-A Sentiment Analysis Techniques There were many methodologies for sentiment analysis developed. In the past, rule-based approaches were used, relying on dictionaries which listed words with their particular sentiment. These maps were not sensitive to the intricacies in the language as it is actually spoken. Pang and Lee [1] conducted an intensive analysis of the initial models of sentiment analysis and the approaches taken to solve the challenges of detecting sentiments on a nuanced level. Progress was achieved by creating algorithms based on the Support Vector Machine approach and the Naive Bayes classifier which analyzed the text using statistical relations and achieved improved accuracy. Wang et al. [2] designed a model that integrates traditional machine learning with rule-based systems in order to enhance categorization performance. At the moment, deep learning techniques have changed the way sentiment analysis [ 3 ] is done. Long Short Term Memory (LSTM) networks and transformers, e.g., BERT, GPT are widely used now since they are extremely efficient at understanding words in their context which has led to major improvements in sentiment classification [3]. II-B Distributed Computing Each node in a distributed architecture pools memory and compute resources. Apache Spark and Hadoop are two popular frameworks to make this ensemble possible. These fast data processing frameworks use in-memory structures and map-reduce implementation. An in-depth comparative analytical study of the performance of Hadoop and Spark Engines was performed by Hazarika et al which has found that for a given data processing need, there are specific scenarios for which respective framework can be considered favorable for execution. [ 5 ] . II-C Distributed Sentiment Analysis Lately, there is some attention paid to sentiment analysis for distributed systems [ 6 , 7 , 8 ] . Evidently, some emotion processing works are easier to perform with distributed systems. For instance, Zhang et al. [ 9 ] were able to utilize Apache Spark to process Twitter data streams with incredible speed. Kumar and Singh [6] found that LSTM is more effective and precise in a distributed design [ 11 , 12 ] to analyze sentiment data [ 13 , 14 ] . III Methodology III-A Experimental Setup We will demonstrate how we incorporate sentiment analysis into a distributed system with the help Sentiment140 dataset. This dataset contains 1.6 million tweets which have been preclassified to positive, negative or neutral sentiments. Hence, this dataset is a useful resource when it comes to building sentiment analysis models. We aimed at comparing performance variation between single-node setup as well as multi-node/cluster setups in terms of data processing time, accuracy, and resource consumption. III-B Tools and Technologies Used In our experiment, we used Apache Spark as a distributed data processing model. BERT was our pre-trained model that computed contextual embeddings of the text data provided. We achieved this by employing python programming language through libraries like PySpark that connect both technologies. III-C Model Training Fig. 1 shows two different setups for the experiment: single node and distributed architecture. Fig. 1: Sentiment Analysis in Distributed and Single-Node Environment III-C 1 Training on a Single Node The dataset required preprocessing to optimise text features. Preprocessing included lowercasing everything, removing special characters and tokenization done through the NLTK library in Python. BERT embeddings were then created using Hugging Face Transformers library after this, tweets were converted into numerical vectors based on semantics. Subsequently, a logistic regression model was developed utilizing these embeddings through a conventional implementation provided by Scikit-learn. The model was trained utilizing a dataset comprising 1.5 million tweets, with a batch size of 32, and implemented a 70-30 partition for training and validation purposes. The training process was executed over a duration of 10 epochs, with the learning rate established at 0.01. While processing the test set that contained many tweets, we achieved almost 85% accuracy with this model, but during processing the resource utilization went haywire which becomes a bottleneck when working on huge datasets. III-C 2 Training in Distributed Architecture In contrast to the centralized way, the distributed architecture took one data set and spread it across four nodes within a separated local cluster. Each node had 4 cpu cores and 8 gb of ram, all these supported distribution of processing. The first task was to distribute the dataset whereby each node received 400000 tweets. Data was efficiently loaded and manipulated using PySpark’s DataFrame API. Additionally, all nodes preprocessed their own data before going directly to generating BERT embeddings, hence allowing for parallelizing of both embedding and BERT processes which led to much faster execution time by considering them together. For conducting the distributed training, we used Spark’s MLib. We followed the same parameters as in single-node setup only that this time around all data had to be communicated between nodes via spark’s distributed data structures thereby joining nodes into one. As such all updates of model weights were done with For model training in the distributed setup, we utilized Spark’s MLlib, enabling us to train the logistic regression model in a distributed fashion. The model was trained using the same parameters as in the single-node setup, but the data was communicated between nodes using Spark’s distributed data structures. This facilitated enhanced model training efficiency as updates to the model weights were synchronized across nodes. The processing times were greatly reduced by using this distributed approach, and we were able to run the model on multiple cores and nodes for faster convergence purposes. It was evident from the overall accuracy of about 88% realized from the test set of the