spam_predict_1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Text SMS - Spam Classification Model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The base requirement of this project is to analyse the SMS dataset and come up with a machine learning models to predict or claissify the sms text. For getting my latest code and datasets please do visit my **[github.com](https://github.com/vijayanandrp/ML-003-SMS-Spam-Detector)** account.\n",
    "\n",
    "##### The following are the list of actions that we gonna do to solve this problem approach\n",
    "\n",
    "1. Reading a text-based dataset into pandas\n",
    "2. Vectorizing our dataset\n",
    "3. Building and evaluating a model\n",
    "4. Comparing models\n",
    "5. Examining a model for further insight\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Reading a text-based dataset into pandas\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have very good well-defined [datasets](https://raw.githubusercontent.com/vijayanandrp/ML-003-SMS-Spam-Detector/master/data/sms.csv) already, then this would be the very first section in data science job. The main job of this section is to loading/reading the datasets into a **pandas** object for data analysis without any errors. You can download the dataset [here](https://raw.githubusercontent.com/vijayanandrp/ML-003-SMS-Spam-Detector/master/data/sms.csv). This dataset is really cool to analyse. Please do read the output carefully. It helps you a lot in future."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import os\n",
    "\n",
    "spam_file = 'data/sms.csv'\n",
    "\n",
    "if not os.path.isfile(spam_file):\n",
    "    print(spam_file, ' is missing.')\n",
    "    exit()\n",
    "\n",
    "# 1. Loading dataset\n",
    "sms_df = pd.read_csv(spam_file, sep='\\t')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5574, 2)"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# getting the shape of dataset \n",
    "# means getting number of rows * columns values\n",
    "sms_df.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['label', 'message'], dtype='object')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# getting the column names of dataset\n",
    "sms_df.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>message</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>43</th>\n",
       "      <td>ham</td>\n",
       "      <td>WHO ARE YOU SEEING?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1536</th>\n",
       "      <td>spam</td>\n",
       "      <td>You have won a Nokia 7250i. This is what you g...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5231</th>\n",
       "      <td>ham</td>\n",
       "      <td>It means u could not keep ur words.</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>655</th>\n",
       "      <td>ham</td>\n",
       "      <td>Did u got that persons story</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3461</th>\n",
       "      <td>ham</td>\n",
       "      <td>I am back. Bit long cos of accident on a30. Ha...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                            message\n",
       "43     ham                                WHO ARE YOU SEEING?\n",
       "1536  spam  You have won a Nokia 7250i. This is what you g...\n",
       "5231   ham                It means u could not keep ur words.\n",
       "655    ham                       Did u got that persons story\n",
       "3461   ham  I am back. Bit long cos of accident on a30. Ha..."
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# getting the random sample values from the dataset\n",
    "sms_df.sample(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>message</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  label                                            message\n",
       "0   ham  Go until jurong point, crazy.. Available only ...\n",
       "1   ham                      Ok lar... Joking wif u oni...\n",
       "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
       "3   ham  U dun say so early hor... U c already then say...\n",
       "4   ham  Nah I don't think he goes to usf, he lives aro..."
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# reading the top / head values from the dataset\n",
    "sms_df.head(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>message</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>5569</th>\n",
       "      <td>spam</td>\n",
       "      <td>This is the 2nd time we have tried 2 contact u...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5570</th>\n",
       "      <td>ham</td>\n",
       "      <td>Will ü b going to esplanade fr home?</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5571</th>\n",
       "      <td>ham</td>\n",
       "      <td>Pity, * was in mood for that. So...any other s...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5572</th>\n",
       "      <td>ham</td>\n",
       "      <td>The guy did some bitching but I acted like i'd...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5573</th>\n",
       "      <td>ham</td>\n",
       "      <td>Rofl. Its true to its name</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     label                                            message\n",
       "5569  spam  This is the 2nd time we have tried 2 contact u...\n",
       "5570   ham               Will ü b going to esplanade fr home?\n",
       "5571   ham  Pity, * was in mood for that. So...any other s...\n",
       "5572   ham  The guy did some bitching but I acted like i'd...\n",
       "5573   ham                         Rofl. Its true to its name"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# gettting the last / tail values from the dataset\n",
    "sms_df.tail(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['ham', 'spam'], dtype=object)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let us see what the unique values in the label column\n",
    "sms_df.label.unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     4827\n",
       "spam     747\n",
       "Name: label, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let us see how many spam values and ham values are in the dataset.\n",
    "sms_df.label.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# converting label to a numerical value \n",
    "sms_df['label_num'] = sms_df['label'].map({'ham': 0, 'spam': 1})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 2. Feature matrix (X), response vector (y) and train_test_split\n",
    "X = sms_df['message']\n",
    "y = sms_df['label_num']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5574,)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Let's check the shape of X and y\n",
    "X.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5574,)"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "# split X and y into train and test sets\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(4180,)\n",
      "(1394,)\n",
      "(4180,)\n",
      "(1394,)\n"
     ]
    }
   ],
   "source": [
    "for df in [X_train, X_test, y_train, y_test]:\n",
    "    print(df.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Vectorizing our dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# instantiate the vectorizer\n",
    "vector = CountVectorizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# learn training data vocabulary, then use it to create a document-term matrix\n",
    "vector.fit(X_train)\n",
    "X_train_dtm = vector.transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# equivalent step for previous cell\n",
    "X_train_dtm = vector.fit_transform(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['〨ud',\n",
       " 'èn',\n",
       " 'zouk',\n",
       " 'zogtorius',\n",
       " 'zoe',\n",
       " 'zeros',\n",
       " 'zed',\n",
       " 'zealand',\n",
       " 'zac',\n",
       " 'yupz']"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the fitted vocabulary \n",
    "# getting last 10 features\n",
    "vector.get_feature_names()[:-11:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7465"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# total features/tokens/columns in the matrix\n",
    "len(vector.get_feature_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<4180x7465 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 54983 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the document matrix\n",
    "X_train_dtm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# transform testing dataset (using fitted vocabulary) in to a document-term matrix\n",
    "X_test_dtm = vector.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<1394x7465 sparse matrix of type '<class 'numpy.int64'>'\n",
       "\twith 17831 stored elements in Compressed Sparse Row format>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the document matrix \n",
    "X_test_dtm"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3. Building and evaluating a model\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use [multinomial Naive Bayes](http://scikit-learn.org/stable/modules/generated/sklearn.naive_bayes.MultinomialNB.html):\n",
    "\n",
    "> The multinomial Naive Bayes classifier is suitable for classification with **discrete features**  (e.g., word counts for text classification). The multinomial distribution normally requires integer feature counts. However, in practice, fractional counts such as **tf-idf** may also work.\n",
    "\n",
    "Meaning of *discrete* - :\tseparate, distinct, individual, detached, unattached, disconnected, discontinuous, disjunct, disjoined"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import and instantiate a Multinomial Naive Bayes model\n",
    "from sklearn.naive_bayes import MultinomialNB\n",
    "nb = MultinomialNB()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 12 ms, sys: 0 ns, total: 12 ms\n",
      "Wall time: 14 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True)"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# train the  model using X_train_dtm and target values y_train (timing it with an IPython \"magic command\")\n",
    "%time nb.fit(X_train_dtm, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# make the class predictions for X_test_dtm\n",
    "y_predict = nb.predict(X_test_dtm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.98995695839311337"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate accuracy of class predictions\n",
    "from sklearn import metrics\n",
    "metrics.accuracy_score(y_test, y_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1193,    2],\n",
       "       [  12,  187]])"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# print the confusion matrix\n",
    "metrics.confusion_matrix(y_test, y_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1290    Hey...Great deal...Farm tour 9am to 5pm $95/pa...\n",
       "5046    We have sent JD for Customer Service cum Accou...\n",
       "Name: message, dtype: object"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# FALSE POSITIVES (ham incorrectly classified as spam)\n",
    "X_test[(y_predict==1) & (y_test==0)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2663    Hello darling how are you today? I would love ...\n",
       "1940    More people are dogging in your area now. Call...\n",
       "5429    Santa Calling! Would your little ones like a c...\n",
       "2402    Babe: U want me dont u baby! Im nasty and have...\n",
       "869     Hello. We need some posh birds and chaps to us...\n",
       "3064    Hi babe its Jordan, how r u? Im home from abro...\n",
       "3530    Xmas & New Years Eve tickets are now on sale f...\n",
       "1663    Hi if ur lookin 4 saucy daytime fun wiv busty ...\n",
       "2430    Guess who am I?This is the first time I create...\n",
       "1469    Hi its LUCY Hubby at meetins all day Fri & I w...\n",
       "4676    Hi babe its Chloe, how r u? I was smashed on s...\n",
       "1458    CLAIRE here am havin borin time & am now alone...\n",
       "Name: message, dtype: object"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# FALSE NEGATIVES (spam incorrectly classified as ham)\n",
    "X_test[(y_predict==0) & (y_test==1)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'CLAIRE here am havin borin time & am now alone U wanna cum over 2nite? Chat now 09099725823 hope 2 C U Luv CLAIRE xx Calls£1/minmoremobsEMSPOBox45PO139WA'"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_test[1458]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  1.30006229e-02,   9.95618101e-04,   6.15130955e-04, ...,\n",
       "         9.77996362e-01,   1.41464847e-04,   1.32646379e-01])"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate predicted probabilities for X_test_dtm\n",
    "y_pred_prob =  nb.predict_proba(X_test_dtm)[:, 1]\n",
    "y_pred_prob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.96901242614747374"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate AUC\n",
    "metrics.roc_auc_score(y_test, y_predict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4. Comparing models\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "We will compare multinomial Naive Bayes with [logistic regression](http://scikit-learn.org/stable/modules/linear_model.html#logistic-regression):\n",
    "\n",
    "> Logistic regression, despite its name, is a **linear model for classification** rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "# import and instantiate the logistic regression model\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "logreg = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 48 ms, sys: 0 ns, total: 48 ms\n",
      "Wall time: 47.5 ms\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# train and model using X_train_dtm (well calibarated)\n",
    "%time logreg.fit(X_train_dtm, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "# to make the class predictions of X_test_dtm\n",
    "y_predict = logreg.predict(X_test_dtm)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.98206599713055953"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to calculate the accuracy\n",
    "metrics.accuracy_score(y_test, y_predict)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# to calculate the predicted probabilities for X_test_dtm (well calibarated)\n",
    "y_pred_prob = logreg.predict_proba(X_test_dtm)[:, 1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.98607682765290894"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# to calculate AUC\n",
    "metrics.roc_auc_score(y_test, y_pred_prob)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 5. Examining a model for further insight\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will examine the our trained **Naive Nayes Model** to calculate the approximate **\"spamminess\"** of each token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# store the vocabulary/tokens/feature_names/column values of X_train\n",
    "X_train_tokens = vector.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "7465"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# total tokens or feature extracted from CountVectorizer \n",
    "len(X_train_tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['00', '000', '000pes', '008704050406', '0089', '0121', '01223585236', '01223585334', '0125698789', '02', '0207', '02073162414', '02085076972', '021', '03', '04', '0430', '05', '050703', '0578']\n"
     ]
    }
   ],
   "source": [
    "# first 20 tokens \n",
    "print(X_train_tokens[:20])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['〨ud', 'èn', 'zouk', 'zogtorius', 'zoe', 'zeros', 'zed', 'zealand', 'zac', 'yupz', 'yup', 'yuou', 'yuo', 'yunny', 'yun', 'yummy', 'yummmm', 'ything', 'ystrday', 'yrs']\n"
     ]
    }
   ],
   "source": [
    "# last 20 tokens\n",
    "print(X_train_tokens[:-21:-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.,   0.,   1., ...,   0.,   1.,   1.],\n",
       "       [  6.,  21.,   0., ...,   1.,   0.,   0.]])"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Naive Bayes counts the number of times each token appears in each class\n",
    "nb.feature_count_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 7465)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# row represent  classes (rows), columns represents tokens\n",
    "nb.feature_count_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  0.,  1., ...,  0.,  1.,  1.])"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# number of times each token appears across all HAM messages \n",
    "ham_token_count = nb.feature_count_[0, :]\n",
    "ham_token_count "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([  6.,  21.,   0., ...,   1.,   0.,   0.])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# number of times each token appears across all SPAM messages \n",
    "spam_token_count = nb.feature_count_[1, :]\n",
    "spam_token_count "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>00</th>\n",
       "      <td>0</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000</th>\n",
       "      <td>0</td>\n",
       "      <td>21</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000pes</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>008704050406</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0089</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              ham  spam\n",
       "token                  \n",
       "00              0     6\n",
       "000             0    21\n",
       "000pes          1     0\n",
       "008704050406    0     1\n",
       "0089            0     1"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a DataFrame of tokens with their separate ham and spam counts \n",
    "import pandas as pd\n",
    "tokens = pd.DataFrame({'token': X_train_tokens, 'ham': ham_token_count, 'spam': spam_token_count}).set_index('token')\n",
    "tokens.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>0</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tantrum</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>little</th>\n",
       "      <td>23</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>known</th>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>massive</th>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ham  spam\n",
       "token             \n",
       "872        0     2\n",
       "tantrum    1     0\n",
       "little    23     1\n",
       "known      1     0\n",
       "massive    2     0"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Examine random Dataframe rows\n",
    "tokens.sample(5, random_state=6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3632.,   548.])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Number of samples encountered for each class during fitting. \n",
    "# This value is weighted by the sample weight when provided.\n",
    "# Naive Bayes counts the number of observations in each class\n",
    "nb.class_count_"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before we can calculate the \"spamminess\" of each token, we need to avoid **dividing by zero** and account for the **class imbalance**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tantrum</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>little</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>known</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>massive</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ham  spam\n",
       "token             \n",
       "872        1     3\n",
       "tantrum    2     1\n",
       "little    24     2\n",
       "known      2     1\n",
       "massive    3     1"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add 1 to ham and spam counts to avoid dividing by 0\n",
    "tokens.ham += 1\n",
    "tokens.spam += 1\n",
    "tokens.sample(5, random_state=6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "      <th>ham_freq</th>\n",
       "      <th>spam_freq</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.005474</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tantrum</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000551</td>\n",
       "      <td>0.001825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>little</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>0.006608</td>\n",
       "      <td>0.003650</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>known</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000551</td>\n",
       "      <td>0.001825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>massive</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000826</td>\n",
       "      <td>0.001825</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ham  spam  ham_freq  spam_freq\n",
       "token                                  \n",
       "872        1     3  0.000275   0.005474\n",
       "tantrum    2     1  0.000551   0.001825\n",
       "little    24     2  0.006608   0.003650\n",
       "known      2     1  0.000551   0.001825\n",
       "massive    3     1  0.000826   0.001825"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert spam and ham tokens into frequencies\n",
    "tokens['ham_freq'] = tokens.ham / nb.class_count_[0]\n",
    "tokens['spam_freq'] = tokens.spam / nb.class_count_[1]\n",
    "tokens.sample(5, random_state=6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "      <th>ham_freq</th>\n",
       "      <th>spam_freq</th>\n",
       "      <th>spam_ratio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>872</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.005474</td>\n",
       "      <td>19.883212</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tantrum</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000551</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>3.313869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>little</th>\n",
       "      <td>24</td>\n",
       "      <td>2</td>\n",
       "      <td>0.006608</td>\n",
       "      <td>0.003650</td>\n",
       "      <td>0.552311</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>known</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000551</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>3.313869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>massive</th>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>0.000826</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>2.209246</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         ham  spam  ham_freq  spam_freq  spam_ratio\n",
       "token                                              \n",
       "872        1     3  0.000275   0.005474   19.883212\n",
       "tantrum    2     1  0.000551   0.001825    3.313869\n",
       "little    24     2  0.006608   0.003650    0.552311\n",
       "known      2     1  0.000551   0.001825    3.313869\n",
       "massive    3     1  0.000826   0.001825    2.209246"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens['spam_ratio'] = tokens.spam_freq / tokens.ham_freq\n",
    "tokens.sample(5, random_state=6)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "      <th>ham_freq</th>\n",
       "      <th>spam_freq</th>\n",
       "      <th>spam_ratio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>claim</th>\n",
       "      <td>1</td>\n",
       "      <td>82</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.149635</td>\n",
       "      <td>543.474453</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>prize</th>\n",
       "      <td>1</td>\n",
       "      <td>71</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.129562</td>\n",
       "      <td>470.569343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>uk</th>\n",
       "      <td>1</td>\n",
       "      <td>61</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.111314</td>\n",
       "      <td>404.291971</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>150p</th>\n",
       "      <td>1</td>\n",
       "      <td>53</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.096715</td>\n",
       "      <td>351.270073</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tone</th>\n",
       "      <td>1</td>\n",
       "      <td>45</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.082117</td>\n",
       "      <td>298.248175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "      <td>40</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.072993</td>\n",
       "      <td>265.109489</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1</td>\n",
       "      <td>37</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.067518</td>\n",
       "      <td>245.226277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>guaranteed</th>\n",
       "      <td>1</td>\n",
       "      <td>37</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.067518</td>\n",
       "      <td>245.226277</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1000</th>\n",
       "      <td>1</td>\n",
       "      <td>34</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.062044</td>\n",
       "      <td>225.343066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>500</th>\n",
       "      <td>1</td>\n",
       "      <td>32</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.058394</td>\n",
       "      <td>212.087591</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.052920</td>\n",
       "      <td>192.204380</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cs</th>\n",
       "      <td>1</td>\n",
       "      <td>27</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.049270</td>\n",
       "      <td>178.948905</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ringtone</th>\n",
       "      <td>1</td>\n",
       "      <td>26</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.047445</td>\n",
       "      <td>172.321168</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10p</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.043796</td>\n",
       "      <td>159.065693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>awarded</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.043796</td>\n",
       "      <td>159.065693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>www</th>\n",
       "      <td>3</td>\n",
       "      <td>69</td>\n",
       "      <td>0.000826</td>\n",
       "      <td>0.125912</td>\n",
       "      <td>152.437956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>000</th>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.040146</td>\n",
       "      <td>145.810219</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5000</th>\n",
       "      <td>1</td>\n",
       "      <td>21</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.038321</td>\n",
       "      <td>139.182482</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>weekly</th>\n",
       "      <td>1</td>\n",
       "      <td>21</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.038321</td>\n",
       "      <td>139.182482</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mob</th>\n",
       "      <td>1</td>\n",
       "      <td>21</td>\n",
       "      <td>0.000275</td>\n",
       "      <td>0.038321</td>\n",
       "      <td>139.182482</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            ham  spam  ham_freq  spam_freq  spam_ratio\n",
       "token                                                 \n",
       "claim         1    82  0.000275   0.149635  543.474453\n",
       "prize         1    71  0.000275   0.129562  470.569343\n",
       "uk            1    61  0.000275   0.111314  404.291971\n",
       "150p          1    53  0.000275   0.096715  351.270073\n",
       "tone          1    45  0.000275   0.082117  298.248175\n",
       "16            1    40  0.000275   0.072993  265.109489\n",
       "18            1    37  0.000275   0.067518  245.226277\n",
       "guaranteed    1    37  0.000275   0.067518  245.226277\n",
       "1000          1    34  0.000275   0.062044  225.343066\n",
       "500           1    32  0.000275   0.058394  212.087591\n",
       "100           1    29  0.000275   0.052920  192.204380\n",
       "cs            1    27  0.000275   0.049270  178.948905\n",
       "ringtone      1    26  0.000275   0.047445  172.321168\n",
       "10p           1    24  0.000275   0.043796  159.065693\n",
       "awarded       1    24  0.000275   0.043796  159.065693\n",
       "www           3    69  0.000826   0.125912  152.437956\n",
       "000           1    22  0.000275   0.040146  145.810219\n",
       "5000          1    21  0.000275   0.038321  139.182482\n",
       "weekly        1    21  0.000275   0.038321  139.182482\n",
       "mob           1    21  0.000275   0.038321  139.182482"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the data values by spam ratio\n",
    "tokens.sort_values('spam_ratio', ascending=False)[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "      <th>ham_freq</th>\n",
       "      <th>spam_freq</th>\n",
       "      <th>spam_ratio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>gt</th>\n",
       "      <td>240</td>\n",
       "      <td>1</td>\n",
       "      <td>0.066079</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.027616</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lt</th>\n",
       "      <td>237</td>\n",
       "      <td>1</td>\n",
       "      <td>0.065253</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.027965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>he</th>\n",
       "      <td>176</td>\n",
       "      <td>1</td>\n",
       "      <td>0.048458</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.037658</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lor</th>\n",
       "      <td>123</td>\n",
       "      <td>1</td>\n",
       "      <td>0.033866</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.053884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>she</th>\n",
       "      <td>118</td>\n",
       "      <td>1</td>\n",
       "      <td>0.032489</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.056167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>later</th>\n",
       "      <td>116</td>\n",
       "      <td>1</td>\n",
       "      <td>0.031938</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.057136</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>da</th>\n",
       "      <td>111</td>\n",
       "      <td>1</td>\n",
       "      <td>0.030562</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.059709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ask</th>\n",
       "      <td>67</td>\n",
       "      <td>1</td>\n",
       "      <td>0.018447</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.098921</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>but</th>\n",
       "      <td>332</td>\n",
       "      <td>5</td>\n",
       "      <td>0.091410</td>\n",
       "      <td>0.009124</td>\n",
       "      <td>0.099815</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>amp</th>\n",
       "      <td>66</td>\n",
       "      <td>1</td>\n",
       "      <td>0.018172</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.100420</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>said</th>\n",
       "      <td>65</td>\n",
       "      <td>1</td>\n",
       "      <td>0.017896</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.101965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>doing</th>\n",
       "      <td>65</td>\n",
       "      <td>1</td>\n",
       "      <td>0.017896</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.101965</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>home</th>\n",
       "      <td>129</td>\n",
       "      <td>2</td>\n",
       "      <td>0.035518</td>\n",
       "      <td>0.003650</td>\n",
       "      <td>0.102756</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>really</th>\n",
       "      <td>63</td>\n",
       "      <td>1</td>\n",
       "      <td>0.017346</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.105202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>morning</th>\n",
       "      <td>60</td>\n",
       "      <td>1</td>\n",
       "      <td>0.016520</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.110462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>come</th>\n",
       "      <td>175</td>\n",
       "      <td>3</td>\n",
       "      <td>0.048183</td>\n",
       "      <td>0.005474</td>\n",
       "      <td>0.113618</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lol</th>\n",
       "      <td>57</td>\n",
       "      <td>1</td>\n",
       "      <td>0.015694</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.116276</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>its</th>\n",
       "      <td>170</td>\n",
       "      <td>3</td>\n",
       "      <td>0.046806</td>\n",
       "      <td>0.005474</td>\n",
       "      <td>0.116960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>anything</th>\n",
       "      <td>55</td>\n",
       "      <td>1</td>\n",
       "      <td>0.015143</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.120504</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cos</th>\n",
       "      <td>55</td>\n",
       "      <td>1</td>\n",
       "      <td>0.015143</td>\n",
       "      <td>0.001825</td>\n",
       "      <td>0.120504</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          ham  spam  ham_freq  spam_freq  spam_ratio\n",
       "token                                               \n",
       "gt        240     1  0.066079   0.001825    0.027616\n",
       "lt        237     1  0.065253   0.001825    0.027965\n",
       "he        176     1  0.048458   0.001825    0.037658\n",
       "lor       123     1  0.033866   0.001825    0.053884\n",
       "she       118     1  0.032489   0.001825    0.056167\n",
       "later     116     1  0.031938   0.001825    0.057136\n",
       "da        111     1  0.030562   0.001825    0.059709\n",
       "ask        67     1  0.018447   0.001825    0.098921\n",
       "but       332     5  0.091410   0.009124    0.099815\n",
       "amp        66     1  0.018172   0.001825    0.100420\n",
       "said       65     1  0.017896   0.001825    0.101965\n",
       "doing      65     1  0.017896   0.001825    0.101965\n",
       "home      129     2  0.035518   0.003650    0.102756\n",
       "really     63     1  0.017346   0.001825    0.105202\n",
       "morning    60     1  0.016520   0.001825    0.110462\n",
       "come      175     3  0.048183   0.005474    0.113618\n",
       "lol        57     1  0.015694   0.001825    0.116276\n",
       "its       170     3  0.046806   0.005474    0.116960\n",
       "anything   55     1  0.015143   0.001825    0.120504\n",
       "cos        55     1  0.015143   0.001825    0.120504"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the data values by spam ratio\n",
    "tokens.sort_values('spam_ratio', ascending=True)[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Enter any single text words - money\n",
      "Spam Ratio of money is 0.6762997169670787\n",
      "Enter any single text words - vijay\n",
      "Spam Ratio of vijay is 1.3255474452554743\n",
      "Enter any single text words - anand\n",
      "Spam Ratio of anand is 3.313868613138686\n",
      "Enter any single text words - cool\n",
      "Spam Ratio of cool is 0.7101147028154328\n",
      "Enter any single text words - honey\n",
      "Spam Ratio of honey is 1.1046228710462287\n",
      "Enter any single text words - click\n",
      "Spam Ratio of click is 13.255474452554745\n",
      "Enter any single text words - won\n",
      "Spam Ratio of won is 36.15129396151294\n",
      "Enter any single text words - q\n"
     ]
    }
   ],
   "source": [
    "test_value = input('Enter any single text words - ').lower()\n",
    "while test_value.strip().lower() not in ['q', 'end', 'quit']:\n",
    "    if test_value:\n",
    "        try:\n",
    "            print('Spam Ratio of {} is {}'.format(test_value, tokens.loc[test_value, 'spam_ratio']))\n",
    "        except:\n",
    "            print('Try again! The word {} is not found in the training dictionary.'.format(test_value)) \n",
    "    test_value = input('Enter any single text words - ')\n",
    "    test_value = test_value.lower().strip()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In my repo I have grid search for the model. Please visit my github - [SMS Spam predictor](https://github.com/vijayanandrp/ML-003-SMS-Spam-Detector) for more infos."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}