Performance Analysis of Supervised Machine Learning Algorithms for Text Classification Published in 2016 19th International Conference on Computer and Information Technology (ICCIT)

Sadia Zaman Mishu, S M Rafiuddin · 2025

Read Performance Analysis of Supervised Machine Learning Algorithms for Text Classification Published in 2016 19th International Conference on Computer and Information Technology (ICCIT) by Sadia Zaman Mishu, S M Rafiuddin (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-00983.txt.

Introduction

Sadia Zaman Mishu and S M Rafiuddin analyze supervised learning methods for text classification. Demand for this task grows in web searching and data mining. It also grows in ranking and recommendation systems. Documents can be sorted by various kinds of classifiers. Supervised methods use labeled documents to learn the categories.

The paper shows the process on different datasets. It applies standard supervised techniques to those sets. Accuracy of each classifier is measured for comparison. A neural network with back propagation is used with other models. Together they form a platform for labeled text sorting. A benchmark approach is used to analyze performance. Tests on real data reveal which model works well in accuracy.

Catalog record

CREATORSadia Zaman Mishu, S M Rafiuddin
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE16 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-00983

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Performance Analysis of Supervised Machine Learning Algorithms for Text Classification Published in 2016 19th International Conference on Computer and Information Technology (ICCIT) Sadia Zaman Mishu, S M Rafiuddin Abstract The demand for text classification is growing significantly in web searching, data mining, web ranking, recommendation systems, and so many other fields of information and technology. This paper illustrates the text classification process on different dataset using some standard supervised machine learning techniques. Text documents can be classified through various kinds of classifiers. Labeled text documents are used to classify the text in supervised classifications. This paper applies these classifiers on different kinds of labeled documents and measures the accuracy of the classifiers. An Artificial Neural Network (ANN) model using Back Propagation Network (BPN) is used with several other models to create an independent platform for labeled and supervised text classification process. An existing benchmark approach is used to analysis the performance of classification using labeled documents. Experimental analysis on real data reveals which model works well in terms of classification accuracy. 1 Introduction Text classification of labeled documents is becoming increasingly necessary due to the rapid growth of documents across the World Wide Web (WWW). Text classification is the task of assigning a document to a predefined category [ 1 ] . In this research, several machine learning algorithms are applied to multiple datasets to determine the accuracy of the classifiers. The machine learning algorithms explored include Naïve Bayes techniques such as Multinomial Naïve Bayes and Bernoulli Naïve Bayes; linear classifiers including Logistic Regression and Stochastic Gradient Descent; Support Vector Machine models such as Support Vector Clustering (SVC) and Linear SVC; and Artificial Neural Network models such as the Back Propagation Network. These algorithms are evaluated on experimental datasets such as the Reuters Corpus [ 13 ] , the Brown Corpus [ 14 ] , and the Movie Review Corpus [ 15 ] . Formally, let D=\{d_{1},d_{2},\ldots,d_{n}\} be a set of documents, and let C=\{c_{1},c_{2},\ldots,c_{m}\} be the set of classes. The text classification process aims to assign each document d_{i} to the appropriate class c_{j} [ 1 ] . This research addresses hard categorization of text, where every document must be placed into exactly one of the specific classes. The text classification process generally involves the following steps [ 1 ] : 1. Reading documents 2. Tokenizing texts 3. Stemming 4. Stop word deletion 5. Vector representation of text 6. Feature selection 7. Applying supervised learning algorithms 8. Measuring accuracy Different metrics can be used to measure classification performance; in this research, the F1 score is employed as the primary evaluation metric [ 1 ] . By comparing the classification methods, a voted classifier is ultimately constructed based on the documents and their features. 2 Literature Review There are several works in the recent past on text classification. Li et al. [ 2 ] proposed a text classification technique using positive and unlabeled data. This paper defined two classes: one labeled as positive and another as unlabeled. The unlabeled data contained both positive and negative examples. The task was to identify the labeled data within the unlabeled class. The key finding was that only one class needed explicit labels, while the other did not. Aggarwal et al. [ 3 ] surveyed various approaches to text categorization. This work discussed multiple aspects of automatic text categorization and compared techniques along with their benefits and shortcomings for different applications. Tong et al. [ 4 ] applied support vector machines (SVMs) to text categorization. Their study demonstrated that many relevant features could be discovered when applying SVMs. In contrast, our work employs an extensive package of SVM classification techniques optimized for text classification, with a voting process to ensure the best results. Liu et al. [ 5 ] discussed classifying text by labeling words instead of documents. While this method can be effective, it requires documents to be enriched with many relevant words for each class. In our work, we instead use Named Entity Recognition (NER) in Natural Language Processing (NLP) to simplify identifying relevant features for document labeling. Schütze [ 6 ] estimated the conditional probability of a word given a class as its relative frequency in documents belonging to that class. However, Bernoulli Naïve Bayes can be inefficient for long documents, as it does not account for multiple occurrences of words. Tang et al. [ 7 ] employed a Bayesian approach for automated text categorization, using information gain (IG) and maximum discrimination (MD) for feature selection. In our work, we use the document frequency metric for feature selection and apply Naïve Bayes to compare results with previous works [ 1 ] . 3 Methodologies Document frequency (DF) is used for feature selection in text classification [ 1 ] . It is defined as DF(t_{k})=P(t_{k}), (1) where DF(t_{k}) is the document frequency of term t_{k} and P(t_{k}) is its probability of occurrence. The classifier models used for text classification are described below. 3.1 Naïve Bayes Model The Naïve Bayes model includes Multinomial Naïve Bayes and Bernoulli Naïve Bayes [ 8 ] . 3.1.1 Multinomial Naïve Bayes Multinomial Naïve Bayes models the count of terms in documents [ 8 ] . This variation accounts for the number of occurrences of a term t in training documents from class C , including multiple occurrences. Algorithm 1 Multinomial Naïve Bayes Training 1: V\leftarrow\textsc{ExtractVocabulary}(D) 2: N\leftarrow\textsc{CountDocs}(D) 3: for each c\in C do 4: N_{c}\leftarrow\textsc{CountDocsInClass}(D,c) 5: \textit{prior}[c]\leftarrow N_{c}/N 6: \textit{text}_{c}\leftarrow\textsc{ConcatTextOfAllDocsInClass}(D,c) 7: for each t\in V do 8: T_{ct}\leftarrow\textsc{CountTokensOfTerm}(\textit{text}_{c},t) 9: for each t\in V do 10: \textit{condprob}[t][c]\leftarrow\dfrac{T_{ct}+1}{\sum_{t^{\prime}}(T_{ct^{\prime}}+1)} 11: return V,\ \textit{prior},\ \textit{condprob} Algorithm 2 Multinomial Naïve Bayes Application 1: W\leftarrow\textsc{ExtractTokensFromDoc}(V,d) 2: for each c\in C do 3: \textit{score}[c]\leftarrow\log\textit{prior}[c] 4: for each t\in W do 5: \textit{score}[c]\leftarrow\textit{score}[c]+\log\textit{condprob}[t][c] 6: return \operatorname*{arg\,max}\limits_{c\in C}\ \textit{score}[c] 3.1.2 Bernoulli Naïve Bayes This variant generates each token from the vocabulary and assigns 1 if the token appears in the document and 0 otherwise [ 8 ] . Algorithm 3 Bernoulli Naïve Bayes Training 1: V\leftarrow\textsc{ExtractVocabulary}(D) 2: N\leftarrow\textsc{CountDocs}(D) 3: for each c\in C do 4: N_{c}\leftarrow\textsc{CountDocsInClass}(D,c) 5: \textit{prior}[c]\leftarrow N_{c}/N 6: for each t\in V do 7: N_{ct}\leftarrow\textsc{CountDocsInClassContainingTerm}(D,c,t) 8: \textit{condprob}[t][c]\leftarrow\dfrac{N_{ct}+1}{N_{c}+2} 9: return V,\ \textit{prior},\ \textit{condprob} Algorithm 4 Bernoulli Naïve Bayes Application 1: V_{d}\leftarrow\textsc{ExtractTermsFromDoc}(V,d) 2: for each c\in C do 3: \textit{score}[c]\leftarrow\log\textit{prior}[c] 4: for each t\in V do 5: if t\in V_{d} then 6: \textit{score}[c]\leftarrow\textit{score}[c]+\log\textit{condprob}[t][c] 7: else 8: \textit{score}[c]\leftarrow\textit{score}[c]+\log\!\big(1-\textit{condprob}[t][c]\big) 9: return \operatorname*{arg\,max}\limits_{c\in C}\ \textit{score}[c] 3.2 Linear Classifier Model The linear classifier model is based on gradient-descent methods, including Logistic Regression and Stochastic Gradient Descent (SGD). 3.2.1 Logistic Regression The probability of assigning document d^{(i)} to class c=j is [ 9 ] P\!\left(c^{(i)}=j\mid d^{(i)};\theta\right)=\frac{\exp(\theta_{j}^{\top}d^{(i)})}{\sum_{l=1}^{k}\exp(\theta_{l}^{\top}d^{(i)})}, (2) where \theta_{1},\ldots,\theta_{k}\in\mathbb{R}^{n+1} are the parameters. The hypothesis vector is h_{\theta}(d^{(i)})=\begin{bmatrix}P(c^{(i)}{=}1\mid d^{(i)};\theta)\\ \vdots\\ P(c^{(i)}{=}k\mid d^{(i)};\theta)\end{bmatrix}=\frac{1}{\sum_{j=1}^{k}\exp(\theta_{j}^{\top}d^{(i)})}\begin{bmatrix}\exp(\theta_{1}^{\top}d^{(i)})\\ \vdots\\ \exp(\theta_{k}^{\top}d^{(i)})\end{bmatrix}. (3) The (multiclass) cross-entropy cost is J(\theta)=-\frac{1}{m}\sum_{i=1}^{m}\sum_{j=1}^{k}\mathbf{1}\{c^{(i)}=j\}\,\log\frac{\exp(\theta_{j}^{\top}d^{(i)})}{\sum_{l=1}^{k}\exp(\theta_{l}^{\top}d^{(i)})}, (4) with gradient \nabla_{\theta_{j}}J(\theta)=-\frac{1}{m}\sum_{i=1}^{m}\big[\;d^{(i)}\big(\mathbf{1}\{c^{(i)}=j\}-P(c^{(i)}=j\mid d^{(i)};\theta)\big)\big]. (5) 3.2.2 Stochastic Gradient Descent For a training pair (d^{(i)},c^{(i)}) , a simple squared-error cost is [ 10 ] \mathrm{cost}\!\left(\theta;(d^{(i)},c^{(i)})\right)=\frac{1}{2}\big(h_{\theta}(d^{(i)})-c^{(i)}\big)^{2}, (6) and the empirical objective is J_{\text{train}}(\theta)=\frac{1}{2m}\sum_{i=1}^{m}\mathrm{cost}\!\left(\theta;(d^{(i)},c^{(i)})\right). (7) Algorithm 5 Stochastic Gradient Descent (one epoch) 1: Randomly shuffle the dataset 2: for i=1 to m do \triangleright training examples 3: for j=0 to n do \triangleright features 4: \theta_{j}\leftarrow\theta_{j}-\alpha\,\big(h_{\theta}(d^{(i)})-c^{(i)}\big)\,d^{(i)}_{j} Here, \alpha is the learning rate (often decayed over iterations) to promote convergence of \theta . 3.3

Read the complete text