Bernoulli Naive Bayes

Dataset 

Link to the dataset - https://raw.githubusercontent.com/amankharwal/SMS-Spam-Detection/master/spam.csv

 

The dataset we're using will be helpful in SMS Message Spam Detection. It consists of 4 features and one target variable. 

 

(i) message - text message, categorical feature. 

 

(ii) Unnamed:2 - unknown feature, we will be dropping this. 

 

(iii) Unnamed:3 - unknown feature, we will be dropping this. 

 

(iv) Unnamed:4 - unknown feature, we will be dropping this. 

 

(v) class - target variable, binary feature - ‘spam’ or ‘ham’ . 

 

Implementation 

For self-implementation, we will have to create three functions, one for estimating prior probability, one for estimating conditional probability, and one for prediction. The working of the same has been discussed in the previous section.

 

For simplicity, we would be using the already existing sklearn library for Bernoulli Naive Bayes implementation. 

 

Importing Necessary Libraries

Firstly, we will load some basic libraries:-

(i) Numpy - for linear algebra. 

(ii) Pandas - for data analysis. 

(iii) Seaborn - for data visualization.

(iv) Matplotlib - for data visualisation. 

(v) BernoulliNB - for Bernoulli Naive Bayes implementation. 

(vi) CountVectorizer - for sparse matrix representation. 

import numpy as np  import pandas as pd  import seaborn as sns from matplotlib import pyplot as plt from sklearn.naive_bayes import BernoulliNB  from sklearn.feature_extraction.text import CountVectorizer

Loading Data

#loading dataset df = pd.read_csv('spam.csv', encoding= 'latin-1')

 

Visualization

We visualize the dataset by printing the first ten rows of the data frame. We use the head() function for the same. 

#visualizing dataset df.head(n=10)

 

Output

 

Above, we observe all the features and the target variable 'class.' Also, we notice that three columns 'Unnamed:2', 'Unnamed:3' and 'Unnamed:4' contain many NaN or missing values. We will be handling the same in the next section. 

 

Now, we use the shape function to get an idea about the dimensions of the dataset. 

df.shape

 

Output

 

From the above, we observe there are 5572 examples and five columns.  

 

Preprocessing

1. Data imputation

We drop 'Unnamed:2', 'Unnamed:3' and 'Unnamed:4' as they contain too many missing values. Also, these features are unknown, so there is no point in retaining them. 

#dropping columns with too many NaN values df= df.drop(['Unnamed: 2', 'Unnamed: 3', 'Unnamed: 4'], axis=1)

 

df.shape

 

Output

 

We notice that three features have been dropped, and now our data contains just two columns, one representing the 'message' feature and one representing the 'class' target variable.

 

2. Binarization

To test Bernoulli Naive Bayes on the dataset, we need to ensure that all values are binary. 

 

So, firstly, we check if our target variable values are binary or not. 

#checking if target variable is binary or not  np.unique(df['class']) #2 unique values, hence it is binary

 

Output

We notice that our target variable has binary values, 'ham' or 'spam.' 

 

Secondly, we check if our ‘message’ feature values are binary or not. 

#checking if 'message' feature is binary or not  np.unique(df['message']) # > 2 unique values , hence it is not binary

 

Output

 

We notice that our ‘message’ feature is not binary. So we will use CountVectorizer to fix this. 

 

3. Vectorization

Now, we will use CountVectorizer() to fix our 'message' feature by creating a sparse matrix.  

#creating sparse matrix using CountVectorizer #converting df columns to individual array  x =df["message"].values  y = df["class"].values # creating count vectorizer object  cv = CountVectorizer() #tranforming values  x = cv.fit_transform(x) v= x.toarray() #printing sparse matrix  print(v)

Output

 

4. Data arrangement

Now, we will just arrange our dataset such that our target variable is the last column. This will make training easier. 

#shifting target column to the end  first_col = df.pop('message') df.insert(0, 'message', first_col) df

 

Output

5. Train-Test Split

Now, we will divide our data into training data and testing data. We will have a 3:1 train test split. This would imply that our training data will have 4179 examples, whereas our testing data will have 1393 examples. 

#train test split = 3:1  train_x = x[:4179] train_y = y[:4179] test_x = x[4179:] test_y = y[4179:]

Training 

We will build our Bernoulli Naive Bayes model using the sklearn library and then train it. 

bnb = BernoulliNB(binarize=0.0) model = bnb.fit(train_x, train_y) y_pred_train= bnb.predict(train_x) y_pred_test = bnb.predict(test_x)

We have passed 'binarize' as a parameter for binarizing the values of the dataset. We have also generated the prediction, and now we will move on to the results.  

 

Results

Now, we analyze our model and generate the results. 

print(bnb.score(train_x, train_y)*100) print(bnb.score(test_x, test_y)*100)

 

 

We notice that we get good results on both training and testing sets. The training set gives us a score of 98.73, whereas the testing set gives us a score of 98.20. 

 

Now, we will also generate classification reports for training and testing sets. 

 

For training set:-

#for training set  from sklearn.metrics import classification_report print(classification_report(train_y, y_pred_train))

 

Output

For testing set:-

#for testing set  from sklearn.metrics import classification_report print(classification_report(test_y, y_pred_test))

 

 

As visible from the above, we have been able to get good results. Finally, we are done studying Bernoulli Naive Bayes. 

 

Frequently Asked Questions

1. How many types of Naive Bayes Classifiers are there?
   Naive Bayes can be classified into three types:-
   (i)  Multinomial Naive Bayes- suitable for discrete features.  
   (ii) Bernoulli Naive Bayes - suitable for binary features. 
   (iii) Gaussian Naive Bayes - suitable for continuous features. 
    
2. What is the limitation of the Naive Bayes Classifier?
   The main limitation of the Naive Bayes classifier is its assumption of conditional independence - all features are independent of each other. In reality, this is highly improbable. 
    
3. What is the advantage of the Naive Bayes Classifier?
   The main advantage of the Naive Bayes classifier is that it is really fast in the case of multi-class predictions. When it comes to classification, it performs better than other models such as Logistic Regression. 

Key Takeaways

Congratulations on making it this far. This blog discussed a fundamental overview of the Bernoulli Naive Bayes Classifier!!

 

We learned about Data Loading, Data Visualisation, Data Preprocessing, and Training. We learned how to visualize data then, based on this EDA, took significant decisions concerning preprocessing, made our model training ready, and finally generated the results.

 

If you are preparing for the upcoming Campus Placements, don’t worry. Coding Ninjas has your back. Visit this link for a carefully crafted and designed course on-campus placements and interview preparation.