How to fit Tfidf, train lda topic model in gensim and visualize topic model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1> Motivation: </h1>\n",
    "<p>This notebook is designed to serve as supplementary material for my post on how to visuallize text data which can be read <span style=\"color:blue\">[__HERE__.](https://www.sammywealth.com/blog/visualizing-text-data) </span>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the neccessary libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from matplotlib import pyplot as plt\n",
    "import seaborn as sns\n",
    "plt.style.use('ggplot')\n",
    "%matplotlib inline\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Document</th>\n",
       "      <th>Class</th>\n",
       "      <th>Source</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>50</th>\n",
       "      <td>Historically, the political spectrum was...</td>\n",
       "      <td>PoliticalScience</td>\n",
       "      <td>https://learn.saylor.org/course/polsc108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>51</th>\n",
       "      <td>this unit, we will look at the state, a ...</td>\n",
       "      <td>PoliticalScience</td>\n",
       "      <td>https://learn.saylor.org/course/polsc109</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>52</th>\n",
       "      <td>This unit looks at the various forms of gove...</td>\n",
       "      <td>PoliticalScience</td>\n",
       "      <td>https://learn.saylor.org/course/polsc110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>53</th>\n",
       "      <td>The Max Planck Manual has a global perspecti...</td>\n",
       "      <td>PoliticalScience</td>\n",
       "      <td>https://learn.saylor.org/course/polsc111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>54</th>\n",
       "      <td>This unit traces the emergence of a world sy...</td>\n",
       "      <td>PoliticalScience</td>\n",
       "      <td>https://learn.saylor.org/course/polsc112</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                             Document             Class  \\\n",
       "50        Historically, the political spectrum was...  PoliticalScience   \n",
       "51        this unit, we will look at the state, a ...  PoliticalScience   \n",
       "52    This unit looks at the various forms of gove...  PoliticalScience   \n",
       "53    The Max Planck Manual has a global perspecti...  PoliticalScience   \n",
       "54    This unit traces the emergence of a world sy...  PoliticalScience   \n",
       "\n",
       "                                      Source  \n",
       "50  https://learn.saylor.org/course/polsc108  \n",
       "51  https://learn.saylor.org/course/polsc109  \n",
       "52  https://learn.saylor.org/course/polsc110  \n",
       "53  https://learn.saylor.org/course/polsc111  \n",
       "54  https://learn.saylor.org/course/polsc112  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_excel('MyArticle.xlsx')\n",
    "df.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocess the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will now tokenize the documents. There are many libraries out there that we can use for this purpose. We could use SpaCy, nltk, TextBlob or even create our own function using regular expressions. However, would use `textblob` to tokenize these documents. I will also remove English stopwords and words made of less than 3 characters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk.corpus import stopwords\n",
    "my_stopwords = stopwords.words('english')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from textblob import TextBlob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'How cool Ame reca came 10/15/2018'"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "' '.join([word for word in TextBlob('How cool is Ame? reca came.\\t on 10/15/2018!').words if (not word in my_stopwords and len(word) > 2)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['CleanDoc'] = df['Document'].apply(lambda x: ' '.join([word.lower() for word in TextBlob(x).words if (not word.lower() in my_stopwords and len(word) > 2)]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Let's compare one cleaned document with its raw version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The novel opens with Mrs. Bennet trying to persuade Mr. Bennet to visit Mr. Bingley, a rich and eligible bachelor who has arrived in the neighbourhood. After some verbal sparring with Mr. Bennet baiting his wife, it transpires that this visit has already taken place at Netherfield, Mr. Bingley's rented house. The visit is followed by an invitation to a ball at the local assembly rooms that the whole neighbourhood will attend.\n"
     ]
    }
   ],
   "source": [
    "print(df.iloc[0,0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "novel opens mrs bennet trying persuade bennet visit bingley rich eligible bachelor arrived neighbourhood verbal sparring bennet baiting wife transpires visit already taken place netherfield bingley rented house visit followed invitation ball local assembly rooms whole neighbourhood attend\n"
     ]
    }
   ],
   "source": [
    "print(df.iloc[0,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Great! Let's use the TfidfVectorizer of sklearn to transform our cleaned documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "tv = TfidfVectorizer(min_df= 0, max_df = 0.04)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(55, 1096)\n",
      "Length of Vocabulary: 1096\n"
     ]
    }
   ],
   "source": [
    "X = tv.fit_transform(df['CleanDoc'])\n",
    "print(X.shape)\n",
    "vocab = tv.get_feature_names()\n",
    "print('Length of Vocabulary: {}'.format(len(vocab)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert the transformed documents to a matrix and export it to excel for use in Gephi"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "matrix = pd.DataFrame(X.toarray(), index = df['Class'], columns=vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>100</th>\n",
       "      <th>15</th>\n",
       "      <th>1565</th>\n",
       "      <th>1648</th>\n",
       "      <th>1830s</th>\n",
       "      <th>300</th>\n",
       "      <th>40</th>\n",
       "      <th>able</th>\n",
       "      <th>absorb</th>\n",
       "      <th>accept</th>\n",
       "      <th>...</th>\n",
       "      <th>work</th>\n",
       "      <th>working</th>\n",
       "      <th>works</th>\n",
       "      <th>worth</th>\n",
       "      <th>writes</th>\n",
       "      <th>written</th>\n",
       "      <th>york</th>\n",
       "      <th>young</th>\n",
       "      <th>younger</th>\n",
       "      <th>zones</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Class</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>PrideAndPrejudice</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PrideAndPrejudice</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PrideAndPrejudice</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PrideAndPrejudice</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>PrideAndPrejudice</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 1096 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                   100   15  1565  1648  1830s  300   40  able  absorb  \\\n",
       "Class                                                                    \n",
       "PrideAndPrejudice  0.0  0.0   0.0   0.0    0.0  0.0  0.0   0.0     0.0   \n",
       "PrideAndPrejudice  0.0  0.0   0.0   0.0    0.0  0.0  0.0   0.0     0.0   \n",
       "PrideAndPrejudice  0.0  0.0   0.0   0.0    0.0  0.0  0.0   0.0     0.0   \n",
       "PrideAndPrejudice  0.0  0.0   0.0   0.0    0.0  0.0  0.0   0.0     0.0   \n",
       "PrideAndPrejudice  0.0  0.0   0.0   0.0    0.0  0.0  0.0   0.0     0.0   \n",
       "\n",
       "                   accept  ...    work  working  works  worth  writes  \\\n",
       "Class                      ...                                          \n",
       "PrideAndPrejudice     0.0  ...     0.0      0.0    0.0    0.0     0.0   \n",
       "PrideAndPrejudice     0.0  ...     0.0      0.0    0.0    0.0     0.0   \n",
       "PrideAndPrejudice     0.0  ...     0.0      0.0    0.0    0.0     0.0   \n",
       "PrideAndPrejudice     0.0  ...     0.0      0.0    0.0    0.0     0.0   \n",
       "PrideAndPrejudice     0.0  ...     0.0      0.0    0.0    0.0     0.0   \n",
       "\n",
       "                   written  york  young  younger  zones  \n",
       "Class                                                    \n",
       "PrideAndPrejudice      0.0   0.0    0.0      0.0    0.0  \n",
       "PrideAndPrejudice      0.0   0.0    0.0      0.0    0.0  \n",
       "PrideAndPrejudice      0.0   0.0    0.0      0.0    0.0  \n",
       "PrideAndPrejudice      0.0   0.0    0.0      0.0    0.0  \n",
       "PrideAndPrejudice      0.0   0.0    0.0      0.0    0.0  \n",
       "\n",
       "[5 rows x 1096 columns]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matrix.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "matrix.to_excel('topics matrix.xlsx', index = True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(55, 1096)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "matrix.shape\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h2>If you are interested in the Gephi tutorial part only, <span style=\"color:red\"> STOP HERE AND GO TO GEPHI.</span> otherwise <span style=\"color:BLUE\"> KEEP READING.</span></h2>\n",
    "<br></br>\n",
    "<p></p>\n",
    "<h2> The following section is about <span style=\"color:GREEN\"> HOW TO VISUALIZE LDA TOPIC MODELS.</span></h2>\n",
    "<p> There are many algorithms for doing topic modelling but <span style=\"color:blue\"> gensim</span> is one of the most common ones. <p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\gensim\\utils.py:1212: UserWarning: detected Windows; aliasing chunkize to chunkize_serial\n",
      "  warnings.warn(\"detected Windows; aliasing chunkize to chunkize_serial\")\n"
     ]
    }
   ],
   "source": [
    "import gensim"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Since we had already preprocessed out text above, all we need to do now is [tokenize](https://nlp.stanford.edu/IR-book/html/htmledition/tokenization-1.html) the cleaned documents."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Document</th>\n",
       "      <th>Class</th>\n",
       "      <th>Source</th>\n",
       "      <th>CleanDoc</th>\n",
       "      <th>Tokenized</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The novel opens with Mrs. Bennet trying to per...</td>\n",
       "      <td>PrideAndPrejudice</td>\n",
       "      <td>https://en.wikipedia.org/wiki/Pride_and_Prejudice</td>\n",
       "      <td>novel opens mrs bennet trying persuade bennet ...</td>\n",
       "      <td>[novel, opens, mrs, bennet, trying, persuade, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>At the ball, Mr. Bingley is open and cheerful,...</td>\n",
       "      <td>PrideAndPrejudice</td>\n",
       "      <td>https://en.wikipedia.org/wiki/Pride_and_Prejudice</td>\n",
       "      <td>ball bingley open cheerful popular guests appe...</td>\n",
       "      <td>[ball, bingley, open, cheerful, popular, guest...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>When Jane visits Miss Bingley, she is caught i...</td>\n",
       "      <td>PrideAndPrejudice</td>\n",
       "      <td>https://en.wikipedia.org/wiki/Pride_and_Prejudice</td>\n",
       "      <td>jane visits miss bingley caught rain shower wa...</td>\n",
       "      <td>[jane, visits, miss, bingley, caught, rain, sh...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Mr. Collins, a cousin of Mr. Bennet and heir t...</td>\n",
       "      <td>PrideAndPrejudice</td>\n",
       "      <td>https://en.wikipedia.org/wiki/Pride_and_Prejudice</td>\n",
       "      <td>collins cousin bennet heir longbourn estate vi...</td>\n",
       "      <td>[collins, cousin, bennet, heir, longbourn, est...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Elizabeth and her family meet the dashing and ...</td>\n",
       "      <td>PrideAndPrejudice</td>\n",
       "      <td>https://en.wikipedia.org/wiki/Pride_and_Prejudice</td>\n",
       "      <td>elizabeth family meet dashing charming george ...</td>\n",
       "      <td>[elizabeth, family, meet, dashing, charming, g...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Document              Class  \\\n",
       "0  The novel opens with Mrs. Bennet trying to per...  PrideAndPrejudice   \n",
       "1  At the ball, Mr. Bingley is open and cheerful,...  PrideAndPrejudice   \n",
       "2  When Jane visits Miss Bingley, she is caught i...  PrideAndPrejudice   \n",
       "3  Mr. Collins, a cousin of Mr. Bennet and heir t...  PrideAndPrejudice   \n",
       "4  Elizabeth and her family meet the dashing and ...  PrideAndPrejudice   \n",
       "\n",
       "                                              Source  \\\n",
       "0  https://en.wikipedia.org/wiki/Pride_and_Prejudice   \n",
       "1  https://en.wikipedia.org/wiki/Pride_and_Prejudice   \n",
       "2  https://en.wikipedia.org/wiki/Pride_and_Prejudice   \n",
       "3  https://en.wikipedia.org/wiki/Pride_and_Prejudice   \n",
       "4  https://en.wikipedia.org/wiki/Pride_and_Prejudice   \n",
       "\n",
       "                                            CleanDoc  \\\n",
       "0  novel opens mrs bennet trying persuade bennet ...   \n",
       "1  ball bingley open cheerful popular guests appe...   \n",
       "2  jane visits miss bingley caught rain shower wa...   \n",
       "3  collins cousin bennet heir longbourn estate vi...   \n",
       "4  elizabeth family meet dashing charming george ...   \n",
       "\n",
       "                                           Tokenized  \n",
       "0  [novel, opens, mrs, bennet, trying, persuade, ...  \n",
       "1  [ball, bingley, open, cheerful, popular, guest...  \n",
       "2  [jane, visits, miss, bingley, caught, rain, sh...  \n",
       "3  [collins, cousin, bennet, heir, longbourn, est...  \n",
       "4  [elizabeth, family, meet, dashing, charming, g...  "
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Tokenized'] = df['CleanDoc'].apply(lambda x: x.split())\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h3>They are 3 main steps in creating a gensim corpus.</h3>\n",
    "\n",
    "<ul>\n",
    "    <li>Create Corpus: This is a list of list of tokenized documents. This can easily become too large.</li>\n",
    "    <li>Create token dictionary: This dictionary assignes an index to each token in the entire corpus of documents..</li>\n",
    "    <li>Term Document Frequency: stores the index wordcount pair of each token in the token dictionary.</li>\n",
    "</ul>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[(0, 1), (1, 1), (2, 1), (3, 1), (4, 1), (5, 1), (6, 1), (7, 3), (8, 2), (9, 1), (10, 1), (11, 1), (12, 1), (13, 1), (14, 1), (15, 2), (16, 1), (17, 1), (18, 1), (19, 1), (20, 1), (21, 1), (22, 1), (23, 1), (24, 1), (25, 1), (26, 1), (27, 1), (28, 1), (29, 3), (30, 1), (31, 1)]]\n"
     ]
    }
   ],
   "source": [
    "import gensim.corpora as corpora\n",
    "\n",
    "# Create Corpus\n",
    "texts = df['Tokenized']\n",
    "\n",
    "# Create Dictionary\n",
    "id2word = corpora.Dictionary(df['Tokenized'])\n",
    "\n",
    "# Term Document Frequency\n",
    "corpus = [id2word.doc2bow(text) for text in texts]\n",
    "\n",
    "# View\n",
    "print(corpus[:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train an LDA model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Build LDA model\n",
    "lda_model = gensim.models.ldamodel.LdaModel(corpus=corpus,\n",
    "                                           id2word=id2word,\n",
    "                                           num_topics=4, \n",
    "                                           random_state=100,\n",
    "                                           update_every=1,\n",
    "                                           chunksize=10,\n",
    "                                           passes=10,\n",
    "                                           alpha='auto',\n",
    "                                           per_word_topics=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Get the most dominant topic for each document."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 18), (1, 13), (2, 6), (3, 18)]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "my_topics = [sorted(lda_model[bow][0], key = lambda x: x[1], reverse = True)[0][0] for bow in corpus]\n",
    "[(i, my_topics.count(i)) for i in set(my_topics)]\n",
    "df['PredictedTopic'] = ['Topic {}'.format(topicNumber + 1) for topicNumber in my_topics]\n",
    "mat =pd.DataFrame(X.toarray(), index=my_topics, columns = vocab)\n",
    "mat.to_excel('mat5.xlsx')\n",
    "[(i, my_topics.count(i)) for i in set(my_topics)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Checkout the distribution of topics in our various classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>Document</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Class</th>\n",
       "      <th>PredictedTopic</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">PoliticalScience</th>\n",
       "      <th>Topic 1</th>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 3</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 4</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"4\" valign=\"top\">PrideAndPrejudice</th>\n",
       "      <th>Topic 1</th>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 2</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 3</th>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 4</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">StockTrading</th>\n",
       "      <th>Topic 3</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 4</th>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">Weather</th>\n",
       "      <th>Topic 1</th>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Topic 2</th>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                  Document\n",
       "Class             PredictedTopic          \n",
       "PoliticalScience  Topic 1                9\n",
       "                  Topic 3                1\n",
       "                  Topic 4                2\n",
       "PrideAndPrejudice Topic 1                6\n",
       "                  Topic 2                3\n",
       "                  Topic 3                4\n",
       "                  Topic 4                3\n",
       "StockTrading      Topic 3                1\n",
       "                  Topic 4               13\n",
       "Weather           Topic 1                3\n",
       "                  Topic 2               10"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(by = ['Class', 'PredictedTopic']).agg({'Document': 'count'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It appears that political science is distributed between topics 1, 3 and 4 while PrideAndPrejudice is in all topics. Stock trading and weather seem to be unique to topics 4 and 2.\n",
    "<p>Any one who has read [Pride And Prejudice by Jane Austen](https://en.wikipedia.org/wiki/Pride_and_Prejudice) will agree that it is a mixture of politics and weather and wealth.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Let's retire to seeing the topics. First I get the top 40 words that define each topic and then I use the amzing `pyLDAvis` library to plot the topics."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Topic 1: \n",
      "political | states | unit | state | government | course | perspective | elizabeth | politics | united | darcy | issues | look | forms | contemporary | world | others | ideologies | end | general | across | one | questions | jane | principles | study | science | various | institutions | role | global | century | twentieth | seen | creation | learn | first | governance | among | populations\n",
      "------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "Topic 2: \n",
      "weather | earth | temperature | surface | different | air | lower | differences | system | due | one | pressure | solar | systems | occur | atmosphere | scale | changes | large | sun | climate | atmospheric | phenomena | effect | causes | affect | years | known | higher | part | spot | moisture | sunlight | cell | jet | stream | altitudes | cause | however | caused\n",
      "------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "Topic 3: \n",
      "make | sudan | written | would | people | local | future | lead | relationships | stability | anyway | unitary | prime | president | laws | contemplate | planck | hope | minister | chose | max | manual | legislature | federal | parliament | difference | types | global | darcy | explore | central | governments | democracy | elizabeth | questions | small | whole | still | perspective | time\n",
      "------------------------------------------------------------------------------------------------------------------------------------------------------------\n",
      "Topic 4: \n",
      "stock | economic | shares | european | led | issues | investors | stocks | company | price | another | see | colonies | powers | raw | excess | interests | materials | control | social | spectrum | buy | continue | many | may | sell | exchange | fall | traders | also | day | market | volume | money | extract | take | position | understand | trying | n't\n",
      "------------------------------------------------------------------------------------------------------------------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "for topic in lda_model.show_topics(formatted=False, num_words=40):\n",
    "    print('Topic {}'.format(str(topic[0]+1))+ ': ' + '\\n'+' | '.join([word[0] for word in topic[1]]) + '\\n'+ '--'*78)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(0, [('political', 0.030811898), ('states', 0.020873385), ('unit', 0.020143062), ('state', 0.017383704), ('government', 0.0130092), ('course', 0.010391863), ('perspective', 0.009877052), ('elizabeth', 0.009265046), ('politics', 0.00878077), ('united', 0.008147428), ('darcy', 0.007319779), ('issues', 0.0064352266), ('look', 0.006275257), ('forms', 0.006274406), ('contemporary', 0.005577479), ('world', 0.005574635), ('others', 0.00495894), ('ideologies', 0.004958465), ('end', 0.0046034767), ('general', 0.00459834), ('across', 0.004597168), ('one', 0.004315701), ('questions', 0.0042195744), ('jane', 0.0040715877), ('principles', 0.0040077767), ('study', 0.004007536), ('science', 0.003983518), ('various', 0.0037095603), ('institutions', 0.0037095598), ('role', 0.003708071), ('global', 0.0036281978), ('century', 0.003562005), ('twentieth', 0.003562005), ('seen', 0.0035601829), ('creation', 0.0035514492), ('learn', 0.0035409431), ('first', 0.0035409431), ('governance', 0.0035404793), ('among', 0.0035404793), ('populations', 0.0035387424)])\n",
      "(1, [('weather', 0.034833614), ('earth', 0.011024385), ('temperature', 0.01070653), ('surface', 0.01038692), ('different', 0.009368431), ('air', 0.009121847), ('lower', 0.00898547), ('differences', 0.008805601), ('system', 0.00880422), ('due', 0.0072967047), ('one', 0.0071475594), ('pressure', 0.006905108), ('solar', 0.005955083), ('systems', 0.005955083), ('occur', 0.005955083), ('atmosphere', 0.0059542432), ('scale', 0.0053102607), ('changes', 0.0053093005), ('large', 0.0052211285), ('sun', 0.0050047566), ('climate', 0.005004756), ('atmospheric', 0.0050037275), ('phenomena', 0.0047277096), ('effect', 0.004367877), ('causes', 0.004350041), ('affect', 0.004350039), ('years', 0.0043358887), ('known', 0.0043358887), ('higher', 0.0042639785), ('part', 0.0042102416), ('spot', 0.0040544295), ('moisture', 0.0040544295), ('sunlight', 0.0040544295), ('cell', 0.0040544295), ('jet', 0.0040544295), ('stream', 0.0040544295), ('altitudes', 0.004053296), ('cause', 0.0040531023), ('however', 0.0037591858), ('caused', 0.003671521)])\n",
      "(2, [('make', 0.0141209895), ('sudan', 0.012626654), ('written', 0.0074716196), ('would', 0.0074623525), ('people', 0.007292913), ('local', 0.007232683), ('future', 0.007111937), ('lead', 0.006535789), ('relationships', 0.006535786), ('stability', 0.006535786), ('anyway', 0.006535786), ('unitary', 0.006535786), ('prime', 0.006535786), ('president', 0.006535786), ('laws', 0.006535786), ('contemplate', 0.006535786), ('planck', 0.006535786), ('hope', 0.006535786), ('minister', 0.006535786), ('chose', 0.006535786), ('max', 0.006535786), ('manual', 0.006535786), ('legislature', 0.006535786), ('federal', 0.006535786), ('parliament', 0.006535786), ('difference', 0.006372695), ('types', 0.006372695), ('global', 0.0063096588), ('darcy', 0.0060581868), ('explore', 0.006043082), ('central', 0.0060377233), ('governments', 0.0059918254), ('democracy', 0.0059620147), ('elizabeth', 0.0058634453), ('questions', 0.005328921), ('small', 0.0053025465), ('whole', 0.004926262), ('still', 0.0041881315), ('perspective', 0.0039404277), ('time', 0.003440027)])\n",
      "(3, [('stock', 0.013698467), ('economic', 0.013323671), ('shares', 0.011018119), ('european', 0.010049599), ('led', 0.010045596), ('issues', 0.009523734), ('investors', 0.009493206), ('stocks', 0.008819934), ('company', 0.008808304), ('price', 0.008662621), ('another', 0.007958415), ('see', 0.0072807116), ('colonies', 0.006774896), ('powers', 0.006774896), ('raw', 0.006774896), ('excess', 0.006774896), ('interests', 0.006774896), ('materials', 0.006774896), ('control', 0.006774886), ('social', 0.0067737303), ('spectrum', 0.0067737303), ('buy', 0.006468037), ('continue', 0.0063189813), ('many', 0.0063102716), ('may', 0.0062174103), ('sell', 0.005967885), ('exchange', 0.0056937668), ('fall', 0.005530845), ('traders', 0.0051081832), ('also', 0.005077921), ('day', 0.004860569), ('market', 0.004769013), ('volume', 0.004697988), ('money', 0.00449798), ('extract', 0.004183751), ('take', 0.004084929), ('position', 0.00401899), ('understand', 0.0040065595), ('trying', 0.0038738088), (\"n't\", 0.0037614563)])\n"
     ]
    }
   ],
   "source": [
    "for topic in lda_model.show_topics(formatted=False, num_words=40):\n",
    "    print(topic)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Perplexity:  -7.281403321042627\n",
      "\n",
      "Coherence Score:  0.6346605815698426\n"
     ]
    }
   ],
   "source": [
    "from gensim.models import CoherenceModel\n",
    "# Compute Perplexity\n",
    "print('\\nPerplexity: ', lda_model.log_perplexity(corpus))  # a measure of how good the model is. lower the better.\n",
    "\n",
    "# Compute Coherence Score\n",
    "coherence_model_lda = CoherenceModel(model=lda_model, texts=texts, dictionary=id2word, coherence='c_v')\n",
    "coherence_lda = coherence_model_lda.get_coherence()\n",
    "print('\\nCoherence Score: ', coherence_lda)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\decomposition\\nmf.py:972: DeprecationWarning: invalid escape sequence \\s\n",
      "  \"\"\"\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\decomposition\\nmf.py:1186: DeprecationWarning: invalid escape sequence \\s\n",
      "  \"\"\"\n",
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\sklearn\\manifold\\t_sne.py:420: DeprecationWarning: invalid escape sequence \\s\n",
      "  \"\"\"\n"
     ]
    }
   ],
   "source": [
    "# Plotting tools\n",
    "import pyLDAvis\n",
    "import pyLDAvis.gensim  # don't skip this"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\ProgramData\\Anaconda3\\lib\\site-packages\\pyLDAvis\\_prepare.py:257: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=True'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass sort=False\n",
      "\n",
      "  return pd.concat([default_term_info] + list(topic_dfs))\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<link rel=\"stylesheet\" type=\"text/css\" href=\"https://cdn.rawgit.com/bmabey/pyLDAvis/files/ldavis.v1.0.0.css\">\n",
       "\n",
       "\n",
       "<div id=\"ldavis_el1175226587542275609743629204\"></div>\n",
       "<script type=\"text/javascript\">\n",
       "\n",
       "var ldavis_el1175226587542275609743629204_data = {\"mdsDat\": {\"x\": [0.17547489853339843, -0.07013690865707707, -0.1258965420923948, 0.020558552216073417], \"y\": [0.03531187231995868, -0.16062856334486533, 0.13682465548914574, -0.011507964464239058], \"topics\": [1, 2, 3, 4], \"cluster\": [1, 1, 1, 1], \"Freq\": [34.990943908691406, 29.249521255493164, 25.894948959350586, 9.864580154418945]}, \"tinfo\": {\"Category\": [\"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\"], \"Freq\": [25.0, 27.0, 18.0, 18.0, 9.0, 17.0, 9.0, 4.0, 12.0, 8.0, 7.0, 8.0, 8.0, 3.0, 9.0, 6.0, 9.0, 6.0, 7.0, 10.0, 8.0, 7.0, 6.0, 6.0, 6.0, 8.0, 7.0, 5.0, 7.0, 12.0, 27.00734519958496, 18.29600715637207, 17.65586280822754, 9.108709335327148, 7.696548938751221, 7.141409873962402, 5.50040864944458, 5.49966287612915, 11.40286636352539, 4.888790130615234, 4.886297225952148, 4.346626281738281, 4.34621000289917, 4.0350542068481445, 4.030551910400391, 4.029524803161621, 15.237220764160156, 3.512909412384033, 3.5126984119415283, 3.5688412189483643, 3.2515153884887695, 3.2502105236053467, 8.657465934753418, 2.676146984100342, 2.676146984100342, 2.6758530139923096, 2.673131227493286, 2.6705591678619385, 2.6670286655426025, 3.122180223464966, 3.122180223464966, 8.121027946472168, 3.1205830574035645, 6.415956497192383, 3.1037189960479736, 3.1037189960479736, 3.2515158653259277, 5.640625953674316, 3.4916462898254395, 3.6985549926757812, 3.7828121185302734, 25.52260971069336, 8.077573776245117, 7.844681262969971, 7.610502243041992, 6.6835832595825195, 5.34630012512207, 5.059377193450928, 4.363292694091797, 4.363292694091797, 4.363292694091797, 4.362677574157715, 3.890831232070923, 3.666987895965576, 3.6669883728027344, 3.666234254837036, 6.583659648895264, 3.200345993041992, 3.1872777938842773, 3.1872763633728027, 3.176908254623413, 3.176908254623413, 2.9706830978393555, 2.9706830978393555, 2.9706830978393555, 2.9706830978393555, 2.9706830978393555, 2.9706830978393555, 2.9698524475097656, 2.9697105884552, 2.7543578147888184, 6.451869487762451, 6.864254951477051, 3.8901278972625732, 3.825524091720581, 6.450857162475586, 3.4639956951141357, 5.237021446228027, 3.1242196559906006, 3.0848467350006104, 8.885764122009277, 8.642645835876465, 7.147106647491455, 6.518858432769775, 6.1579437255859375, 5.713669300079346, 5.721213340759277, 5.619169235229492, 4.3946614265441895, 4.3946614265441895, 4.3946614265441895, 4.3946614265441895, 4.3946614265441895, 4.3946614265441895, 4.3939056396484375, 4.3939056396484375, 4.195611953735352, 6.516262054443359, 3.8711788654327393, 3.6933672428131104, 3.5876848697662354, 3.3135175704956055, 3.152898073196411, 3.09350848197937, 2.713867664337158, 2.598928928375244, 4.098924160003662, 2.4399383068084717, 3.0474367141723633, 2.335761308670044, 4.394655227661133, 5.162373065948486, 4.033038139343262, 4.722768306732178, 6.177746772766113, 4.0932745933532715, 3.2938873767852783, 3.1201438903808594, 1.8462951183319092, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 1.6150434017181396, 3.489405870437622, 1.0349197387695312, 1.8440051078796387, 1.7872519493103027, 0.7409418821334839, 0.7409418225288391, 0.724125862121582, 0.724125862121582, 1.615044116973877, 0.7101906538009644, 0.7315031886100769, 1.4932923316955566, 1.4919681549072266, 1.4806264638900757, 1.4732599258422852, 1.310300350189209, 1.2173174619674683, 1.8021352291107178, 1.757414698600769, 1.5747421979904175, 1.5747421979904175, 1.5591654777526855, 1.4970248937606812], \"Term\": [\"weather\", \"political\", \"states\", \"unit\", \"stock\", \"state\", \"economic\", \"make\", \"government\", \"earth\", \"shares\", \"temperature\", \"surface\", \"sudan\", \"elizabeth\", \"european\", \"course\", \"investors\", \"led\", \"perspective\", \"darcy\", \"air\", \"company\", \"stocks\", \"price\", \"politics\", \"lower\", \"people\", \"united\", \"issues\", \"political\", \"states\", \"unit\", \"course\", \"politics\", \"united\", \"look\", \"forms\", \"government\", \"contemporary\", \"world\", \"others\", \"ideologies\", \"end\", \"general\", \"across\", \"state\", \"principles\", \"study\", \"jane\", \"institutions\", \"role\", \"perspective\", \"define\", \"police\", \"discipline\", \"citizens\", \"even\", \"something\", \"twentieth\", \"century\", \"elizabeth\", \"seen\", \"darcy\", \"learn\", \"first\", \"various\", \"issues\", \"science\", \"questions\", \"one\", \"weather\", \"earth\", \"temperature\", \"surface\", \"air\", \"due\", \"pressure\", \"systems\", \"occur\", \"solar\", \"atmosphere\", \"scale\", \"climate\", \"sun\", \"atmospheric\", \"lower\", \"effect\", \"causes\", \"affect\", \"known\", \"years\", \"moisture\", \"cell\", \"spot\", \"stream\", \"jet\", \"sunlight\", \"altitudes\", \"cause\", \"however\", \"differences\", \"different\", \"changes\", \"large\", \"system\", \"phenomena\", \"one\", \"higher\", \"part\", \"stock\", \"economic\", \"shares\", \"european\", \"investors\", \"company\", \"stocks\", \"price\", \"materials\", \"colonies\", \"excess\", \"raw\", \"powers\", \"interests\", \"spectrum\", \"social\", \"buy\", \"led\", \"sell\", \"exchange\", \"fall\", \"traders\", \"day\", \"market\", \"extract\", \"understand\", \"continue\", \"n't\", \"volume\", \"dividend\", \"control\", \"another\", \"may\", \"see\", \"issues\", \"many\", \"also\", \"sudan\", \"written\", \"laws\", \"contemplate\", \"stability\", \"relationships\", \"prime\", \"president\", \"planck\", \"parliament\", \"minister\", \"max\", \"manual\", \"legislature\", \"hope\", \"federal\", \"unitary\", \"chose\", \"anyway\", \"make\", \"still\", \"would\", \"local\", \"lady\", \"catherine\", \"firms\", \"makers\", \"lead\", \"tend\", \"difficult\", \"explore\", \"central\", \"governments\", \"democracy\", \"small\", \"whole\", \"people\", \"future\", \"types\", \"difference\", \"global\", \"darcy\"], \"Total\": [25.0, 27.0, 18.0, 18.0, 9.0, 17.0, 9.0, 4.0, 12.0, 8.0, 7.0, 8.0, 8.0, 3.0, 9.0, 6.0, 9.0, 6.0, 7.0, 10.0, 8.0, 7.0, 6.0, 6.0, 6.0, 8.0, 7.0, 5.0, 7.0, 12.0, 27.498302459716797, 18.864774703979492, 18.242441177368164, 9.561957359313965, 8.149828910827637, 7.625474452972412, 5.95349645614624, 5.953455448150635, 12.431463241577148, 5.376111030578613, 5.376398086547852, 4.8029465675354, 4.803168773651123, 4.538589954376221, 4.538168430328369, 4.538282871246338, 17.185029983520508, 3.9659972190856934, 3.9660215377807617, 4.046419620513916, 3.7046031951904297, 3.7047576904296875, 10.022128105163574, 3.129234790802002, 3.129234790802002, 3.129269599914551, 3.1292788982391357, 3.1289234161376953, 3.130312204360962, 3.6975772380828857, 3.6975772380828857, 9.920286178588867, 3.697840929031372, 8.260088920593262, 3.6963775157928467, 3.6963775157928467, 4.0238165855407715, 12.114831924438477, 4.663939952850342, 5.3574652671813965, 10.720159530639648, 25.96737289428711, 8.516884803771973, 8.27824592590332, 8.058865547180176, 7.124207496643066, 5.780028820037842, 5.492992877960205, 4.796776294708252, 4.796776294708252, 4.796776294708252, 4.796529293060303, 4.338191509246826, 4.100471496582031, 4.1004719734191895, 4.100168704986572, 7.383032321929932, 3.6425023078918457, 3.641120433807373, 3.641120672225952, 3.6400249004364014, 3.6400249004364014, 3.4041666984558105, 3.4041666984558105, 3.4041666984558105, 3.4041666984558105, 3.4041666984558105, 3.4041666984558105, 3.4038333892822266, 3.403776168823242, 3.229236602783203, 7.722317695617676, 8.446676254272461, 4.663608074188232, 4.63181209564209, 9.009511947631836, 4.293073654174805, 10.720159530639648, 4.302906036376953, 4.723475933074951, 9.350393295288086, 9.114376068115234, 7.61046838760376, 6.990653991699219, 6.622706413269043, 6.17598819732666, 6.192493438720703, 6.083710670471191, 4.866954803466797, 4.866954803466797, 4.866954803466797, 4.866954803466797, 4.866954803466797, 4.866954803466797, 4.866999626159668, 4.866999626159668, 4.659100532531738, 7.277034282684326, 4.333734035491943, 4.1574931144714355, 4.054762363433838, 3.781747341156006, 3.618089199066162, 3.559009075164795, 3.180543899536133, 3.068352222442627, 4.853392601013184, 2.907147169113159, 3.632059335708618, 2.804462432861328, 5.562330722808838, 7.02252721786499, 5.534778594970703, 7.281424045562744, 12.114831924438477, 6.506971836090088, 6.284633636474609, 3.629328489303589, 2.3549845218658447, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 2.1243984699249268, 4.841279983520508, 1.551980972290039, 3.0522639751434326, 2.9922704696655273, 1.2538490295410156, 1.2538492679595947, 1.2421290874481201, 1.2421290874481201, 2.82070255279541, 1.2478580474853516, 1.415459156036377, 3.0213828086853027, 3.022505760192871, 3.027602195739746, 3.0310311317443848, 2.6590566635131836, 2.506350517272949, 5.006261825561523, 4.970484733581543, 4.39321231842041, 4.39321231842041, 7.258820056915283, 8.260088920593262], \"loglift\": [30.0, 29.0, 28.0, 27.0, 26.0, 25.0, 24.0, 23.0, 22.0, 21.0, 20.0, 19.0, 18.0, 17.0, 16.0, 15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0, 1.032099962234497, 1.0195000171661377, 1.0174000263214111, 1.0015000104904175, 0.992900013923645, 0.984499990940094, 0.9708999991416931, 0.97079998254776, 0.963699996471405, 0.9550999999046326, 0.9545000195503235, 0.9502999782562256, 0.9501000046730042, 0.9325000047683716, 0.9315000176429749, 0.9312000274658203, 0.9297999739646912, 0.9287999868392944, 0.9286999702453613, 0.9244999885559082, 0.9196000099182129, 0.9192000031471252, 0.9036999940872192, 0.8937000036239624, 0.8937000036239624, 0.8934999704360962, 0.8924999833106995, 0.891700029373169, 0.8899000287055969, 0.8809000253677368, 0.8809000253677368, 0.8500000238418579, 0.8804000020027161, 0.7973999977111816, 0.8752999901771545, 0.8752999901771545, 0.8370000123977661, 0.2856000065803528, 0.7605999708175659, 0.6794999837875366, 0.00839999970048666, 1.2120000123977661, 1.176300048828125, 1.1755000352859497, 1.1720999479293823, 1.1655000448226929, 1.1512999534606934, 1.1470999717712402, 1.134600043296814, 1.134600043296814, 1.134600043296814, 1.1345000267028809, 1.1204999685287476, 1.1175999641418457, 1.1175999641418457, 1.117400050163269, 1.1146999597549438, 1.0999000072479248, 1.0961999893188477, 1.0961999893188477, 1.0931999683380127, 1.0931999683380127, 1.0930999517440796, 1.0930999517440796, 1.0930999517440796, 1.0930999517440796, 1.0930999517440796, 1.0930999517440796, 1.092900037765503, 1.092900037765503, 1.070199966430664, 1.0496000051498413, 1.0219000577926636, 1.0479999780654907, 1.038100004196167, 0.8952000141143799, 1.0147000551223755, 0.5128999948501587, 0.9092000126838684, 0.8033000230789185, 1.3001999855041504, 1.2979999780654907, 1.2883000373840332, 1.2812000513076782, 1.27839994430542, 1.273300051689148, 1.2719999551773071, 1.2717000246047974, 1.2489999532699585, 1.2489999532699585, 1.2489999532699585, 1.2489999532699585, 1.2489999532699585, 1.2489999532699585, 1.248900055885315, 1.248900055885315, 1.2462999820709229, 1.2407000064849854, 1.2382999658584595, 1.232699990272522, 1.228700041770935, 1.2188999652862549, 1.2135000228881836, 1.2108999490737915, 1.1923999786376953, 1.1850999593734741, 1.1821999549865723, 1.1758999824523926, 1.1756000518798828, 1.1682000160217285, 1.1154999732971191, 1.0434000492095947, 1.034600019454956, 0.9182000160217285, 0.6776000261306763, 0.8876000046730042, 0.7050999999046326, 2.16510009765625, 2.0729000568389893, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 2.042099952697754, 1.988800048828125, 1.9110000133514404, 1.8122999668121338, 1.8008999824523926, 1.7901999950408936, 1.7901999950408936, 1.7766000032424927, 1.7766000032424927, 1.7585999965667725, 1.7525999546051025, 1.6561000347137451, 1.6115000247955322, 1.610200047492981, 1.6009000539779663, 1.5947999954223633, 1.6085000038146973, 1.593999981880188, 1.2944999933242798, 1.2764999866485596, 1.2903000116348267, 1.2903000116348267, 0.7781999707221985, 0.608299970626831], \"logprob\": [30.0, 29.0, 28.0, 27.0, 26.0, 25.0, 24.0, 23.0, 22.0, 21.0, 20.0, 19.0, 18.0, 17.0, 16.0, 15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0, -3.4798998832702637, -3.86929988861084, -3.904900074005127, -4.566699981689453, -4.735199928283691, -4.810100078582764, -5.071100234985352, -5.071300029754639, -4.342100143432617, -5.189000129699707, -5.189499855041504, -5.306600093841553, -5.306700229644775, -5.380899906158447, -5.3821001052856445, -5.382299900054932, -4.052199840545654, -5.519499778747559, -5.519599914550781, -5.503699779510498, -5.596799850463867, -5.5971999168396, -4.617499828338623, -5.791600227355957, -5.791600227355957, -5.7916998863220215, -5.792699813842773, -5.793700218200684, -5.795000076293945, -5.637400150299072, -5.637400150299072, -4.68149995803833, -5.637899875640869, -4.917200088500977, -5.643400192260742, -5.643400192260742, -5.596799850463867, -5.046000003814697, -5.525599956512451, -5.4679999351501465, -5.445499897003174, -3.3571999073028564, -4.507599830627441, -4.536900043487549, -4.567200183868408, -4.6971001625061035, -4.920300006866455, -4.975500106811523, -5.123499870300293, -5.123499870300293, -5.123499870300293, -5.123700141906738, -5.238100051879883, -5.297399997711182, -5.297399997711182, -5.297599792480469, -4.712100028991699, -5.433499813079834, -5.437600135803223, -5.437600135803223, -5.440800189971924, -5.440800189971924, -5.507900238037109, -5.507900238037109, -5.507900238037109, -5.507900238037109, -5.507900238037109, -5.507900238037109, -5.508200168609619, -5.508299827575684, -5.583600044250488, -4.732399940490723, -4.670400142669678, -5.23829984664917, -5.255000114440918, -4.732500076293945, -5.354300022125244, -4.940999984741211, -5.457600116729736, -5.470200061798096, -4.290500164031982, -4.31820011138916, -4.508200168609619, -4.600200176239014, -4.657199859619141, -4.732100009918213, -4.7307000160217285, -4.748700141906738, -4.994500160217285, -4.994500160217285, -4.994500160217285, -4.994500160217285, -4.994500160217285, -4.994500160217285, -4.994699954986572, -4.994699954986572, -5.040900230407715, -4.600599765777588, -5.121399879455566, -5.168399810791016, -5.197400093078613, -5.276899814605713, -5.326600074768066, -5.345600128173828, -5.476500034332275, -5.519800186157227, -5.064199924468994, -5.582900047302246, -5.360599994659424, -5.6265997886657715, -4.994500160217285, -4.833499908447266, -5.080399990081787, -4.922500133514404, -4.6539998054504395, -5.0655999183654785, -5.282899856567383, -4.3719000816345215, -4.896599769592285, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -5.0304999351501465, -4.2600998878479, -5.475500106811523, -4.897900104522705, -4.929100036621094, -5.809700012207031, -5.809700012207031, -5.832600116729736, -5.832600116729736, -5.0304999351501465, -5.8520002365112305, -5.822500228881836, -5.108799934387207, -5.1097002029418945, -5.117400169372559, -5.122300148010254, -5.23960018157959, -5.313199996948242, -4.920899868011475, -4.946000099182129, -5.055699825286865, -5.055699825286865, -5.065700054168701, -5.106299877166748]}, \"token.table\": {\"Topic\": [1, 2, 2, 1, 2, 3, 2, 2, 3, 4, 2, 2, 3, 4, 2, 2, 2, 1, 4, 1, 2, 4, 1, 2, 3, 3, 4, 1, 3, 2, 3, 1, 1, 4, 3, 1, 1, 4, 1, 4, 1, 2, 2, 3, 4, 1, 3, 2, 2, 3, 2, 1, 4, 1, 3, 1, 3, 3, 1, 4, 3, 3, 4, 4, 1, 1, 1, 2, 4, 1, 1, 2, 3, 4, 1, 4, 1, 4, 2, 3, 4, 2, 1, 1, 3, 3, 1, 3, 1, 2, 2, 4, 2, 4, 2, 4, 1, 3, 4, 2, 4, 1, 2, 3, 4, 4, 4, 1, 3, 3, 3, 4, 1, 3, 4, 2, 3, 2, 1, 2, 3, 4, 1, 4, 1, 2, 1, 4, 1, 4, 1, 2, 4, 1, 1, 1, 3, 4, 2, 3, 4, 1, 1, 4, 3, 4, 1, 2, 1, 2, 2, 3, 1, 3, 3, 3, 4, 3, 2, 1, 3, 2, 4, 1, 2, 1, 4, 3, 3, 2, 1, 4, 2, 2, 2, 2, 3, 2, 2, 4, 3, 1, 1, 4, 3, 1, 4, 1, 1, 3, 2, 3, 4, 1, 2, 4, 4, 2], \"Freq\": [0.8813906311988831, 0.8239221572875977, 0.9825654029846191, 0.3182365298271179, 0.15911826491355896, 0.4773547947406769, 0.8813592195510864, 0.2847977578639984, 0.7119944095611572, 0.9414429664611816, 0.833936333656311, 0.9755696058273315, 0.8585348129272461, 0.797544002532959, 0.8813740611076355, 0.8239222168922424, 0.8812729120254517, 0.33085131645202637, 0.33085131645202637, 0.8113420605659485, 0.8577049970626831, 0.9414429664611816, 0.9586873054504395, 0.9754976034164429, 0.8218691349029541, 0.9715044498443604, 0.9414429664611816, 0.9300402998924255, 0.8241657614707947, 0.1797807514667511, 0.7191230058670044, 0.9412298798561096, 0.7263844609260559, 0.12106407433748245, 0.8291669487953186, 0.9587008357048035, 0.3299207389354706, 0.3299207389354706, 0.4552477300167084, 0.4552477300167084, 0.1294948011636734, 0.7769688367843628, 0.8287283182144165, 0.11838976293802261, 0.7064845561981201, 0.9586901664733887, 0.7131491303443909, 0.8650475740432739, 0.9393105506896973, 0.9874510169029236, 0.8236096501350403, 0.8064283728599548, 0.10080354660749435, 0.8813309669494629, 1.001336932182312, 0.9587962627410889, 0.8218691349029541, 0.9621182680130005, 0.330974280834198, 0.330974280834198, 0.943234920501709, 0.9864943027496338, 0.9414429664611816, 0.8050693273544312, 0.8116053938865662, 0.8398483991622925, 0.4023752510547638, 0.2011876255273819, 0.4023752510547638, 0.8814128637313843, 0.4132903218269348, 0.13776344060897827, 0.27552688121795654, 0.27552688121795654, 0.8848515748977661, 0.08044105023145676, 0.3302943706512451, 0.3302943706512451, 0.6972032189369202, 0.2324010729789734, 0.9414429664611816, 0.92901211977005, 0.832783579826355, 0.809803307056427, 0.8218691349029541, 0.9059740304946899, 0.49526068568229675, 0.49526068568229675, 0.9885282516479492, 0.8812729120254517, 0.8241701722145081, 0.7975441813468933, 0.8635928630828857, 0.9414429664611816, 0.35452160239219666, 0.7090432047843933, 0.8116053938865662, 0.9619303345680237, 0.9414429664611816, 0.3341943919658661, 0.6683887839317322, 1.007811188697815, 0.9481199383735657, 0.20655694603919983, 0.6196708083152771, 0.8050693273544312, 0.9414429664611816, 0.3073626458644867, 0.6147252917289734, 0.8429312705993652, 0.8218691349029541, 0.9414429664611816, 0.1806757003068924, 0.7227028012275696, 0.9414429664611816, 0.8812729120254517, 0.6879596710205078, 0.833893358707428, 0.3731287717819214, 0.46641096472740173, 0.09328219294548035, 0.09328219294548035, 0.8328220844268799, 0.9414429664611816, 0.21170850098133087, 0.6351255178451538, 0.599249541759491, 0.39949968457221985, 0.8980128765106201, 0.09977921098470688, 0.23293334245681763, 0.6988000273704529, 0.9414429664611816, 0.9587008357048035, 0.9818788170814514, 0.9816157221794128, 0.8218691349029541, 0.9414429664611816, 0.9102506041526794, 0.9862402081489563, 0.9414429664611816, 1.0085735321044922, 0.7466217279434204, 0.1866554319858551, 0.8218691349029541, 0.9414429664611816, 0.8097695708274841, 0.9220432043075562, 0.6432330012321472, 0.21441099047660828, 0.2746715545654297, 0.6866788864135742, 0.8112842440605164, 0.9229915738105774, 0.9197856783866882, 0.3760732114315033, 0.3760732114315033, 0.8218615651130676, 0.833893358707428, 0.9583708643913269, 0.8218615651130676, 0.8812729120254517, 0.9414429664611816, 0.8728526830673218, 0.11638036370277405, 0.9541592597961426, 0.6443377733230591, 0.9625263810157776, 0.9689150452613831, 0.8812729120254517, 1.0085673332214355, 0.8265991806983948, 0.9754974842071533, 0.8812729120254517, 0.9926955699920654, 0.665962815284729, 0.22198760509490967, 0.833893358707428, 0.9663882851600647, 0.8013731837272644, 0.7932841181755066, 0.8113420605659485, 0.4552477300167084, 0.4552477300167084, 0.9777234792709351, 0.9867100715637207, 0.9414429664611816, 0.917975664138794, 0.7455608248710632, 0.8259776830673218, 1.0012564659118652, 0.39898648858070374, 0.39898648858070374, 0.9299906492233276, 0.32762566208839417, 0.6552513241767883, 0.8492624759674072, 0.8241701722145081], \"Term\": [\"across\", \"affect\", \"air\", \"also\", \"also\", \"also\", \"altitudes\", \"another\", \"another\", \"anyway\", \"atmosphere\", \"atmospheric\", \"buy\", \"catherine\", \"cause\", \"causes\", \"cell\", \"central\", \"central\", \"century\", \"changes\", \"chose\", \"citizens\", \"climate\", \"colonies\", \"company\", \"contemplate\", \"contemporary\", \"continue\", \"control\", \"control\", \"course\", \"darcy\", \"darcy\", \"day\", \"define\", \"democracy\", \"democracy\", \"difference\", \"difference\", \"differences\", \"differences\", \"different\", \"different\", \"difficult\", \"discipline\", \"dividend\", \"due\", \"earth\", \"economic\", \"effect\", \"elizabeth\", \"elizabeth\", \"end\", \"european\", \"even\", \"excess\", \"exchange\", \"explore\", \"explore\", \"extract\", \"fall\", \"federal\", \"firms\", \"first\", \"forms\", \"future\", \"future\", \"future\", \"general\", \"global\", \"global\", \"global\", \"global\", \"government\", \"government\", \"governments\", \"governments\", \"higher\", \"higher\", \"hope\", \"however\", \"ideologies\", \"institutions\", \"interests\", \"investors\", \"issues\", \"issues\", \"jane\", \"jet\", \"known\", \"lady\", \"large\", \"laws\", \"lead\", \"lead\", \"learn\", \"led\", \"legislature\", \"local\", \"local\", \"look\", \"lower\", \"make\", \"make\", \"makers\", \"manual\", \"many\", \"many\", \"market\", \"materials\", \"max\", \"may\", \"may\", \"minister\", \"moisture\", \"n't\", \"occur\", \"one\", \"one\", \"one\", \"one\", \"others\", \"parliament\", \"part\", \"part\", \"people\", \"people\", \"perspective\", \"perspective\", \"phenomena\", \"phenomena\", \"planck\", \"police\", \"political\", \"politics\", \"powers\", \"president\", \"pressure\", \"price\", \"prime\", \"principles\", \"questions\", \"questions\", \"raw\", \"relationships\", \"role\", \"scale\", \"science\", \"science\", \"see\", \"see\", \"seen\", \"sell\", \"shares\", \"small\", \"small\", \"social\", \"solar\", \"something\", \"spectrum\", \"spot\", \"stability\", \"state\", \"state\", \"states\", \"still\", \"stock\", \"stocks\", \"stream\", \"study\", \"sudan\", \"sun\", \"sunlight\", \"surface\", \"system\", \"system\", \"systems\", \"temperature\", \"tend\", \"traders\", \"twentieth\", \"types\", \"types\", \"understand\", \"unit\", \"unitary\", \"united\", \"various\", \"volume\", \"weather\", \"whole\", \"whole\", \"world\", \"would\", \"would\", \"written\", \"years\"]}, \"R\": 30, \"lambda.step\": 0.01, \"plot.opts\": {\"xlab\": \"PC1\", \"ylab\": \"PC2\"}, \"topic.order\": [1, 2, 4, 3]};\n",
       "\n",
       "function LDAvis_load_lib(url, callback){\n",
       "  var s = document.createElement('script');\n",
       "  s.src = url;\n",
       "  s.async = true;\n",
       "  s.onreadystatechange = s.onload = callback;\n",
       "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
       "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
       "}\n",
       "\n",
       "if(typeof(LDAvis) !== \"undefined\"){\n",
       "   // already loaded: just create the visualization\n",
       "   !function(LDAvis){\n",
       "       new LDAvis(\"#\" + \"ldavis_el1175226587542275609743629204\", ldavis_el1175226587542275609743629204_data);\n",
       "   }(LDAvis);\n",
       "}else if(typeof define === \"function\" && define.amd){\n",
       "   // require.js is available: use it to load d3/LDAvis\n",
       "   require.config({paths: {d3: \"https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min\"}});\n",
       "   require([\"d3\"], function(d3){\n",
       "      window.d3 = d3;\n",
       "      LDAvis_load_lib(\"https://cdn.rawgit.com/bmabey/pyLDAvis/files/ldavis.v1.0.0.js\", function(){\n",
       "        new LDAvis(\"#\" + \"ldavis_el1175226587542275609743629204\", ldavis_el1175226587542275609743629204_data);\n",
       "      });\n",
       "    });\n",
       "}else{\n",
       "    // require.js not available: dynamically load d3 & LDAvis\n",
       "    LDAvis_load_lib(\"https://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min.js\", function(){\n",
       "         LDAvis_load_lib(\"https://cdn.rawgit.com/bmabey/pyLDAvis/files/ldavis.v1.0.0.js\", function(){\n",
       "                 new LDAvis(\"#\" + \"ldavis_el1175226587542275609743629204\", ldavis_el1175226587542275609743629204_data);\n",
       "            })\n",
       "         });\n",
       "}\n",
       "</script>"
      ],
      "text/plain": [
       "PreparedData(topic_coordinates=              x         y  topics  cluster       Freq\n",
       "topic                                                \n",
       "0      0.175475  0.035312       1        1  34.990944\n",
       "1     -0.070137 -0.160629       2        1  29.249521\n",
       "3     -0.125897  0.136825       3        1  25.894949\n",
       "2      0.020559 -0.011508       4        1   9.864580, topic_info=     Category       Freq         Term      Total  loglift  logprob\n",
       "term                                                              \n",
       "639   Default  25.000000      weather  25.000000  30.0000  30.0000\n",
       "1027  Default  27.000000    political  27.000000  29.0000  29.0000\n",
       "1006  Default  18.000000       states  18.000000  28.0000  28.0000\n",
       "1107  Default  18.000000         unit  18.000000  27.0000  27.0000\n",
       "379   Default   9.000000        stock   9.000000  26.0000  26.0000\n",
       "631   Default  17.000000        state  17.000000  25.0000  25.0000\n",
       "1193  Default   9.000000     economic   9.000000  24.0000  24.0000\n",
       "428   Default   4.000000         make   4.000000  23.0000  23.0000\n",
       "1149  Default  12.000000   government  12.000000  22.0000  22.0000\n",
       "618   Default   8.000000        earth   8.000000  21.0000  21.0000\n",
       "413   Default   7.000000       shares   7.000000  20.0000  20.0000\n",
       "634   Default   8.000000  temperature   8.000000  19.0000  19.0000\n",
       "686   Default   8.000000      surface   8.000000  18.0000  18.0000\n",
       "1237  Default   3.000000        sudan   3.000000  17.0000  17.0000\n",
       "42    Default   9.000000    elizabeth   9.000000  16.0000  16.0000\n",
       "1255  Default   6.000000     european   6.000000  15.0000  15.0000\n",
       "1016  Default   9.000000       course   9.000000  14.0000  14.0000\n",
       "367   Default   6.000000    investors   6.000000  13.0000  13.0000\n",
       "99    Default   7.000000          led   7.000000  12.0000  12.0000\n",
       "948   Default  10.000000  perspective  10.000000  11.0000  11.0000\n",
       "40    Default   8.000000        darcy   8.000000  10.0000  10.0000\n",
       "644   Default   7.000000          air   7.000000   9.0000   9.0000\n",
       "393   Default   6.000000      company   6.000000   8.0000   8.0000\n",
       "414   Default   6.000000       stocks   6.000000   7.0000   7.0000\n",
       "410   Default   6.000000        price   6.000000   6.0000   6.0000\n",
       "1028  Default   8.000000     politics   8.000000   5.0000   5.0000\n",
       "519   Default   7.000000        lower   7.000000   4.0000   4.0000\n",
       "580   Default   5.000000       people   5.000000   3.0000   3.0000\n",
       "1007  Default   7.000000       united   7.000000   2.0000   2.0000\n",
       "402   Default  12.000000       issues  12.000000   1.0000   1.0000\n",
       "...       ...        ...          ...        ...      ...      ...\n",
       "1228   Topic4   1.615043       manual   2.124398   2.0421  -5.0305\n",
       "1227   Topic4   1.615043  legislature   2.124398   2.0421  -5.0305\n",
       "1225   Topic4   1.615043         hope   2.124398   2.0421  -5.0305\n",
       "1224   Topic4   1.615043      federal   2.124398   2.0421  -5.0305\n",
       "1238   Topic4   1.615043      unitary   2.124398   2.0421  -5.0305\n",
       "1222   Topic4   1.615043        chose   2.124398   2.0421  -5.0305\n",
       "1221   Topic4   1.615043       anyway   2.124398   2.0421  -5.0305\n",
       "428    Topic4   3.489406         make   4.841280   1.9888  -4.2601\n",
       "587    Topic4   1.034920        still   1.551981   1.9110  -5.4755\n",
       "356    Topic4   1.844005        would   3.052264   1.8123  -4.8979\n",
       "13     Topic4   1.787252        local   2.992270   1.8009  -4.9291\n",
       "209    Topic4   0.740942         lady   1.253849   1.7902  -5.8097\n",
       "200    Topic4   0.740942    catherine   1.253849   1.7902  -5.8097\n",
       "569    Topic4   0.724126        firms   1.242129   1.7766  -5.8326\n",
       "575    Topic4   0.724126       makers   1.242129   1.7766  -5.8326\n",
       "825    Topic4   1.615044         lead   2.820703   1.7586  -5.0305\n",
       "448    Topic4   0.710191         tend   1.247858   1.7526  -5.8520\n",
       "437    Topic4   0.731503    difficult   1.415459   1.6561  -5.8225\n",
       "1019   Topic4   1.493292      explore   3.021383   1.6115  -5.1088\n",
       "962    Topic4   1.491968      central   3.022506   1.6102  -5.1097\n",
       "1124   Topic4   1.480626  governments   3.027602   1.6009  -5.1174\n",
       "1141   Topic4   1.473260    democracy   3.031031   1.5948  -5.1223\n",
       "484    Topic4   1.310300        small   2.659057   1.6085  -5.2396\n",
       "30     Topic4   1.217317        whole   2.506351   1.5940  -5.3132\n",
       "580    Topic4   1.802135       people   5.006262   1.2945  -4.9209\n",
       "710    Topic4   1.757415       future   4.970485   1.2765  -4.9460\n",
       "1211   Topic4   1.574742        types   4.393212   1.2903  -5.0557\n",
       "1206   Topic4   1.574742   difference   4.393212   1.2903  -5.0557\n",
       "662    Topic4   1.559165       global   7.258820   0.7782  -5.0657\n",
       "40     Topic4   1.497025        darcy   8.260089   0.6083  -5.1063\n",
       "\n",
       "[189 rows x 6 columns], token_table=      Topic      Freq          Term\n",
       "term                               \n",
       "957       1  0.881391        across\n",
       "643       2  0.823922        affect\n",
       "644       2  0.982565           air\n",
       "197       1  0.318237          also\n",
       "197       2  0.159118          also\n",
       "197       3  0.477355          also\n",
       "698       2  0.881359     altitudes\n",
       "318       2  0.284798       another\n",
       "318       3  0.711994       another\n",
       "1221      4  0.941443        anyway\n",
       "607       2  0.833936    atmosphere\n",
       "608       2  0.975570   atmospheric\n",
       "390       3  0.858535           buy\n",
       "200       4  0.797544     catherine\n",
       "701       2  0.881374         cause\n",
       "775       2  0.823922        causes\n",
       "650       2  0.881273          cell\n",
       "962       1  0.330851       central\n",
       "962       4  0.330851       central\n",
       "1173      1  0.811342       century\n",
       "425       2  0.857705       changes\n",
       "1222      4  0.941443         chose\n",
       "1115      1  0.958687      citizens\n",
       "612       2  0.975498       climate\n",
       "1247      3  0.821869      colonies\n",
       "393       3  0.971504       company\n",
       "1223      4  0.941443   contemplate\n",
       "1046      1  0.930040  contemporary\n",
       "158       3  0.824166      continue\n",
       "705       2  0.179781       control\n",
       "...     ...       ...           ...\n",
       "414       3  0.968915        stocks\n",
       "682       2  0.881273        stream\n",
       "1032      1  1.008567         study\n",
       "1237      4  0.826599         sudan\n",
       "684       2  0.975497           sun\n",
       "685       2  0.881273      sunlight\n",
       "686       2  0.992696       surface\n",
       "723       2  0.665963        system\n",
       "723       3  0.221988        system\n",
       "687       2  0.833893       systems\n",
       "634       2  0.966388   temperature\n",
       "448       4  0.801373          tend\n",
       "420       3  0.793284       traders\n",
       "1186      1  0.811342     twentieth\n",
       "1211      1  0.455248         types\n",
       "1211      4  0.455248         types\n",
       "353       3  0.977723    understand\n",
       "1107      1  0.986710          unit\n",
       "1238      4  0.941443       unitary\n",
       "1007      1  0.917976        united\n",
       "450       1  0.745561       various\n",
       "557       3  0.825978        volume\n",
       "639       2  1.001256       weather\n",
       "30        3  0.398986         whole\n",
       "30        4  0.398986         whole\n",
       "955       1  0.929991         world\n",
       "356       2  0.327626         would\n",
       "356       4  0.655251         would\n",
       "289       4  0.849262       written\n",
       "693       2  0.824170         years\n",
       "\n",
       "[198 rows x 3 columns], R=30, lambda_step=0.01, plot_opts={'xlab': 'PC1', 'ylab': 'PC2'}, topic_order=[1, 2, 4, 3])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Visualize the topics\n",
    "pyLDAvis.enable_notebook()\n",
    "vis = pyLDAvis.gensim.prepare(lda_model, corpus, id2word)\n",
    "vis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> <span style=\"color:blue\"> [Stay Updated! Always checkout my blog for more posts!](https://www.sammywealth.com/blog/visualizing-text-data) </span> </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}