FastAI NLP Course - Topic Modeling with NMF and SVD.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "52836d03",
   "metadata": {},
   "source": [
    "# Topic Modeling with SVD and NMF\n",
    "\n",
    "I just follow along the [FastAI NLP](https://www.fast.ai/2019/07/08/fastai-nlp/) course on youtube. The repo with the original notebooks is [here](https://github.com/fastai/course-nlp). This notebook is based on the first three videos of the [youtube playlist](https://www.youtube.com/playlist?list=PLtmWHNX-gukKocXQOkQjuVxglSDYWsSh9)."
   ]
  },
  {
   "cell_type": "raw",
   "id": "87f2f8cd",
   "metadata": {},
   "source": [
    "import numpy as np\n",
    "from sklearn.datasets import fetch_20newsgroups\n",
    "from sklearn import decomposition\n",
    "from scipy import linalg\n",
    "from matplotlib import pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "np.set_printoptions(suppress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b9aa82d",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "In this notebook we use the newsgroups dataset that is available as part of the sklearn package. There are quite a few very useful utility methods that ship with the dataset allowing us to generate train and validation sets. We really won't need the training and testing datasets, but we'll generate them for illustration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "157ef8d1",
   "metadata": {},
   "outputs": [],
   "source": [
    "categories = ['comp.sys.mac.hardware', 'rec.autos', 'sci.electronics', 'misc.forsale']\n",
    "remove = ('headers', 'footers', 'quotes')\n",
    "newsgroups_train = fetch_20newsgroups(subset='train', categories=categories, remove=remove)\n",
    "newsgroups_test = fetch_20newsgroups(subset='test', categories=categories, remove=remove)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2850bd6e",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "One way ticket (return leg of roundtrip ticket) for female traveler\n",
      "\n",
      "\n",
      "\n",
      "San Francisco ==> St. Louis ==> Philadelphia\n",
      "\n",
      "\n",
      "May 21, 1993 (Friday)  leaves SFO     10:25 am\n",
      "                       arrives Phila.  8:43 pm\n",
      "\n",
      "\n",
      "\n",
      ".............$150   or best offer\n",
      "\n",
      "\n",
      "\n",
      "G'day people,\n",
      "\t\n",
      "\tAre there any MR2 owners or motor-head gurus out there, that know why\n",
      "my MR2's engine sounds noisy? The MR2's engine is noisy at the best of times, \n",
      "but not even a nice nose - it's one of those very ugly noises. \n",
      "\tI do an oil change every 2-3 months, and for about 2 months the engine\n",
      "noise sounds relatively quiet during driving and idling. At around the 3 month\n",
      "mark, after an oil change (I've been tracking this very thoroughly for months\n",
      "now) it starts to get that very disgusting noise, not so much during driving,\n",
      "but more so during idling. \n",
      "\tWhat's its problem? \n",
      "\tAlso.. I don't know if it's just me, but if noticed a little\n",
      "performance drop. It just hasn't got the acceleration it used to. \n",
      "\n",
      "\tAny help/tips would be appreciated!!\n",
      "\n",
      "Looking for a VIDEO in and OUT Video card for the IBM.  One that will\n",
      "allow you to watch TV (coax) or video IN, and will do Video out,\n",
      "digitize pictures.  and if I am in Windows, and would like to be able to\n",
      "look the RCA out for the card to my TV and have it display on there, as\n",
      "well as DOS apps.\n",
      "\n",
      "I heard of these SNES and Genesis copiers, that will copy any games, are\n",
      "those for real?\n",
      "                                                                                                                            \n"
     ]
    }
   ],
   "source": [
    "print(\"\\n\".join(newsgroups_train.data[:3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "8a501711",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 2, 1])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "newsgroups_train.target[:3]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c15a3707",
   "metadata": {},
   "source": [
    "## Stemming and Lemmatization\n",
    "\n",
    "Stemming and Lemmatization are methods adopted to reduce the complexity of the model. They achieve this by reducing the total number of words to consider by trying to map all the words in the dataset to its root words. The algorithms are quite flawed and approximate (as we will see in the examples). Nevertheless, there are many advantages to using these methods. Lemmatisation is the fancier algorithm. Stemming can be referred to as a poor man's lemmatisation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "694be427",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package wordnet to /home/abishek/nltk_data...\n",
      "[nltk_data]   Package wordnet is already up-to-date!\n",
      "[nltk_data] Downloading package omw-1.4 to /home/abishek/nltk_data...\n",
      "[nltk_data]   Unzipping corpora/omw-1.4.zip.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "nltk.download('wordnet')\n",
    "nltk.download('omw-1.4')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "6dd96c89",
   "metadata": {},
   "outputs": [],
   "source": [
    "from nltk import stem\n",
    "\n",
    "wnl = stem.WordNetLemmatizer()\n",
    "porter = stem.porter.PorterStemmer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "14f17fe4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['foot', 'foot', 'foot', 'footing']"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "words = ['feet', 'foot', 'foots', 'footing']\n",
    "\n",
    "[wnl.lemmatize(word) for word in words]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "fa20a80e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['feet', 'foot', 'foot', 'foot']"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "[porter.stem(word) for word in words]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "d30c794f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[['fly', 'fly', 'flying'], ['organize', 'organizes', 'organizing'], ['universe', 'university']]\n",
      "[['fli', 'fli', 'fli'], ['organ', 'organ', 'organ'], ['univers', 'univers']]\n"
     ]
    }
   ],
   "source": [
    "word_lists = [\n",
    "    ['fly', 'flies', 'flying'],\n",
    "    ['organize', 'organizes', 'organizing'],\n",
    "    ['universe', 'university']\n",
    "]\n",
    "\n",
    "print([[wnl.lemmatize(word) for word in wordlist] for wordlist in word_lists])\n",
    "print([[porter.stem(word) for word in wordlist] for wordlist in word_lists])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "4bf56455",
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "nlp = spacy.load(\"en_core_web_sm\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "d6bf069b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "lookup\n"
     ]
    },
    {
     "ename": "AttributeError",
     "evalue": "'EnglishLemmatizer' object has no attribute 'lookup'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_3989/393849268.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mlemmatizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnlp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_pipe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"lemmatizer\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlemmatizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmode\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;34m[\u001b[0m\u001b[0mlemmatizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlookup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mword\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mword\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mwords\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/tmp/ipykernel_3989/393849268.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m      3\u001b[0m \u001b[0mlemmatizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnlp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget_pipe\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"lemmatizer\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mlemmatizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmode\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 5\u001b[0;31m \u001b[0;34m[\u001b[0m\u001b[0mlemmatizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlookup\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mword\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mword\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mwords\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m: 'EnglishLemmatizer' object has no attribute 'lookup'"
     ]
    }
   ],
   "source": [
    "#FIXME: this doesn't work. \n",
    "nlp.remove_pipe(\"lemmatizer\")\n",
    "nlp.add_pipe(\"lemmatizer\", config={\"mode\": \"lookup\"})\n",
    "lemmatizer = nlp.get_pipe(\"lemmatizer\")\n",
    "[lemmatizer.lookup(word) for word in words]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ae01533c",
   "metadata": {},
   "source": [
    "## Stop Words\n",
    "\n",
    "Stop words is another traditional NLP technique to reduce model complexity. In most modern environment, stop word removal is discouraged because we actually lose data when we do this. But in resource constrained environment, this is still a useful technique. There is no standard universal stop word list and is very very implementation specific too. We'll this with both spacy and sklearn's stop word list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "d775ad6d",
   "metadata": {},
   "outputs": [],
   "source": [
    "spacy_stop_words = nlp.Defaults.stop_words"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "1d99d6bc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "frozenset({'a',\n",
       "           'about',\n",
       "           'above',\n",
       "           'across',\n",
       "           'after',\n",
       "           'afterwards',\n",
       "           'again',\n",
       "           'against',\n",
       "           'all',\n",
       "           'almost',\n",
       "           'alone',\n",
       "           'along',\n",
       "           'already',\n",
       "           'also',\n",
       "           'although',\n",
       "           'always',\n",
       "           'am',\n",
       "           'among',\n",
       "           'amongst',\n",
       "           'amoungst',\n",
       "           'amount',\n",
       "           'an',\n",
       "           'and',\n",
       "           'another',\n",
       "           'any',\n",
       "           'anyhow',\n",
       "           'anyone',\n",
       "           'anything',\n",
       "           'anyway',\n",
       "           'anywhere',\n",
       "           'are',\n",
       "           'around',\n",
       "           'as',\n",
       "           'at',\n",
       "           'back',\n",
       "           'be',\n",
       "           'became',\n",
       "           'because',\n",
       "           'become',\n",
       "           'becomes',\n",
       "           'becoming',\n",
       "           'been',\n",
       "           'before',\n",
       "           'beforehand',\n",
       "           'behind',\n",
       "           'being',\n",
       "           'below',\n",
       "           'beside',\n",
       "           'besides',\n",
       "           'between',\n",
       "           'beyond',\n",
       "           'bill',\n",
       "           'both',\n",
       "           'bottom',\n",
       "           'but',\n",
       "           'by',\n",
       "           'call',\n",
       "           'can',\n",
       "           'cannot',\n",
       "           'cant',\n",
       "           'co',\n",
       "           'con',\n",
       "           'could',\n",
       "           'couldnt',\n",
       "           'cry',\n",
       "           'de',\n",
       "           'describe',\n",
       "           'detail',\n",
       "           'do',\n",
       "           'done',\n",
       "           'down',\n",
       "           'due',\n",
       "           'during',\n",
       "           'each',\n",
       "           'eg',\n",
       "           'eight',\n",
       "           'either',\n",
       "           'eleven',\n",
       "           'else',\n",
       "           'elsewhere',\n",
       "           'empty',\n",
       "           'enough',\n",
       "           'etc',\n",
       "           'even',\n",
       "           'ever',\n",
       "           'every',\n",
       "           'everyone',\n",
       "           'everything',\n",
       "           'everywhere',\n",
       "           'except',\n",
       "           'few',\n",
       "           'fifteen',\n",
       "           'fifty',\n",
       "           'fill',\n",
       "           'find',\n",
       "           'fire',\n",
       "           'first',\n",
       "           'five',\n",
       "           'for',\n",
       "           'former',\n",
       "           'formerly',\n",
       "           'forty',\n",
       "           'found',\n",
       "           'four',\n",
       "           'from',\n",
       "           'front',\n",
       "           'full',\n",
       "           'further',\n",
       "           'get',\n",
       "           'give',\n",
       "           'go',\n",
       "           'had',\n",
       "           'has',\n",
       "           'hasnt',\n",
       "           'have',\n",
       "           'he',\n",
       "           'hence',\n",
       "           'her',\n",
       "           'here',\n",
       "           'hereafter',\n",
       "           'hereby',\n",
       "           'herein',\n",
       "           'hereupon',\n",
       "           'hers',\n",
       "           'herself',\n",
       "           'him',\n",
       "           'himself',\n",
       "           'his',\n",
       "           'how',\n",
       "           'however',\n",
       "           'hundred',\n",
       "           'i',\n",
       "           'ie',\n",
       "           'if',\n",
       "           'in',\n",
       "           'inc',\n",
       "           'indeed',\n",
       "           'interest',\n",
       "           'into',\n",
       "           'is',\n",
       "           'it',\n",
       "           'its',\n",
       "           'itself',\n",
       "           'keep',\n",
       "           'last',\n",
       "           'latter',\n",
       "           'latterly',\n",
       "           'least',\n",
       "           'less',\n",
       "           'ltd',\n",
       "           'made',\n",
       "           'many',\n",
       "           'may',\n",
       "           'me',\n",
       "           'meanwhile',\n",
       "           'might',\n",
       "           'mill',\n",
       "           'mine',\n",
       "           'more',\n",
       "           'moreover',\n",
       "           'most',\n",
       "           'mostly',\n",
       "           'move',\n",
       "           'much',\n",
       "           'must',\n",
       "           'my',\n",
       "           'myself',\n",
       "           'name',\n",
       "           'namely',\n",
       "           'neither',\n",
       "           'never',\n",
       "           'nevertheless',\n",
       "           'next',\n",
       "           'nine',\n",
       "           'no',\n",
       "           'nobody',\n",
       "           'none',\n",
       "           'noone',\n",
       "           'nor',\n",
       "           'not',\n",
       "           'nothing',\n",
       "           'now',\n",
       "           'nowhere',\n",
       "           'of',\n",
       "           'off',\n",
       "           'often',\n",
       "           'on',\n",
       "           'once',\n",
       "           'one',\n",
       "           'only',\n",
       "           'onto',\n",
       "           'or',\n",
       "           'other',\n",
       "           'others',\n",
       "           'otherwise',\n",
       "           'our',\n",
       "           'ours',\n",
       "           'ourselves',\n",
       "           'out',\n",
       "           'over',\n",
       "           'own',\n",
       "           'part',\n",
       "           'per',\n",
       "           'perhaps',\n",
       "           'please',\n",
       "           'put',\n",
       "           'rather',\n",
       "           're',\n",
       "           'same',\n",
       "           'see',\n",
       "           'seem',\n",
       "           'seemed',\n",
       "           'seeming',\n",
       "           'seems',\n",
       "           'serious',\n",
       "           'several',\n",
       "           'she',\n",
       "           'should',\n",
       "           'show',\n",
       "           'side',\n",
       "           'since',\n",
       "           'sincere',\n",
       "           'six',\n",
       "           'sixty',\n",
       "           'so',\n",
       "           'some',\n",
       "           'somehow',\n",
       "           'someone',\n",
       "           'something',\n",
       "           'sometime',\n",
       "           'sometimes',\n",
       "           'somewhere',\n",
       "           'still',\n",
       "           'such',\n",
       "           'system',\n",
       "           'take',\n",
       "           'ten',\n",
       "           'than',\n",
       "           'that',\n",
       "           'the',\n",
       "           'their',\n",
       "           'them',\n",
       "           'themselves',\n",
       "           'then',\n",
       "           'thence',\n",
       "           'there',\n",
       "           'thereafter',\n",
       "           'thereby',\n",
       "           'therefore',\n",
       "           'therein',\n",
       "           'thereupon',\n",
       "           'these',\n",
       "           'they',\n",
       "           'thick',\n",
       "           'thin',\n",
       "           'third',\n",
       "           'this',\n",
       "           'those',\n",
       "           'though',\n",
       "           'three',\n",
       "           'through',\n",
       "           'throughout',\n",
       "           'thru',\n",
       "           'thus',\n",
       "           'to',\n",
       "           'together',\n",
       "           'too',\n",
       "           'top',\n",
       "           'toward',\n",
       "           'towards',\n",
       "           'twelve',\n",
       "           'twenty',\n",
       "           'two',\n",
       "           'un',\n",
       "           'under',\n",
       "           'until',\n",
       "           'up',\n",
       "           'upon',\n",
       "           'us',\n",
       "           'very',\n",
       "           'via',\n",
       "           'was',\n",
       "           'we',\n",
       "           'well',\n",
       "           'were',\n",
       "           'what',\n",
       "           'whatever',\n",
       "           'when',\n",
       "           'whence',\n",
       "           'whenever',\n",
       "           'where',\n",
       "           'whereafter',\n",
       "           'whereas',\n",
       "           'whereby',\n",
       "           'wherein',\n",
       "           'whereupon',\n",
       "           'wherever',\n",
       "           'whether',\n",
       "           'which',\n",
       "           'while',\n",
       "           'whither',\n",
       "           'who',\n",
       "           'whoever',\n",
       "           'whole',\n",
       "           'whom',\n",
       "           'whose',\n",
       "           'why',\n",
       "           'will',\n",
       "           'with',\n",
       "           'within',\n",
       "           'without',\n",
       "           'would',\n",
       "           'yet',\n",
       "           'you',\n",
       "           'your',\n",
       "           'yours',\n",
       "           'yourself',\n",
       "           'yourselves'})"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.feature_extraction import _stop_words as sklearn_stop_words\n",
    "\n",
    "sklearn_stop_words.ENGLISH_STOP_WORDS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "id": "b4eed19f",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\"'d\",\n",
       " \"'ll\",\n",
       " \"'m\",\n",
       " \"'re\",\n",
       " \"'s\",\n",
       " \"'ve\",\n",
       " 'ca',\n",
       " 'did',\n",
       " 'does',\n",
       " 'doing',\n",
       " 'just',\n",
       " 'make',\n",
       " \"n't\",\n",
       " 'n‘t',\n",
       " 'n’t',\n",
       " 'quite',\n",
       " 'really',\n",
       " 'regarding',\n",
       " 'say',\n",
       " 'unless',\n",
       " 'used',\n",
       " 'using',\n",
       " 'various',\n",
       " '‘d',\n",
       " '‘ll',\n",
       " '‘m',\n",
       " '‘re',\n",
       " '‘s',\n",
       " '‘ve',\n",
       " '’d',\n",
       " '’ll',\n",
       " '’m',\n",
       " '’re',\n",
       " '’s',\n",
       " '’ve'}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spacy_stop_words - sklearn_stop_words.ENGLISH_STOP_WORDS"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "fa2e94d8",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "frozenset({'amoungst',\n",
       "           'bill',\n",
       "           'cant',\n",
       "           'co',\n",
       "           'con',\n",
       "           'couldnt',\n",
       "           'cry',\n",
       "           'de',\n",
       "           'describe',\n",
       "           'detail',\n",
       "           'eg',\n",
       "           'etc',\n",
       "           'fill',\n",
       "           'find',\n",
       "           'fire',\n",
       "           'found',\n",
       "           'hasnt',\n",
       "           'ie',\n",
       "           'inc',\n",
       "           'interest',\n",
       "           'ltd',\n",
       "           'mill',\n",
       "           'sincere',\n",
       "           'system',\n",
       "           'thick',\n",
       "           'thin',\n",
       "           'un'})"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sklearn_stop_words.ENGLISH_STOP_WORDS - spacy_stop_words"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f894873a",
   "metadata": {},
   "source": [
    "## Topic Modeling\n",
    "\n",
    "Topic Modeling is an unsupervised learning problem, actually, where we try to cluster the dataset to see what topics emerge. Here we are using the topics we chose to filter the dataset and then run SVD style decomposition to understand if the realised components actually fall under the topics we originally chose. That was a complicated sentence, but the overall thing is far simpler than what I wrote up.\n",
    "\n",
    "To do this, though, we need the data to be vectorized into numbers. sklearn offers two good options to this: `count vectorizer` and `tfidf vectorizer`. We implement the algorithm with both these options. There is no accuracy estimate at this time because it is unsupervised. We just eyeball to check that the outputs are sensible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "d4f8db68",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "\n",
    "vectorizer = CountVectorizer(stop_words='english')\n",
    "vectors = vectorizer.fit_transform(newsgroups_train.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "fceafb00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2348, 20490)"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vectors.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "45b690b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2348\n"
     ]
    }
   ],
   "source": [
    "print(len(newsgroups_train.data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "6a7f387a",
   "metadata": {},
   "outputs": [],
   "source": [
    "vocab = vectorizer.get_feature_names_out()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "216666af",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(20490,)"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "0e4cc794",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['existence', 'existing', 'exists', 'exit', 'exits', 'exitserver',\n",
       "       'exotic', 'exp', 'expand', 'expandable', 'expanded', 'expander',\n",
       "       'expandibility', 'expanding', 'expands', 'expansion', 'expasion',\n",
       "       'expecially', 'expect', 'expected'], dtype=object)"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocab[8000:8020]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "575cf6fc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 2min 11s, sys: 13.1 s, total: 2min 24s\n",
      "Wall time: 39.1 s\n"
     ]
    }
   ],
   "source": [
    "%time U,s,Vh = linalg.svd(vectors.todense(), full_matrices=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "401b2faa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((2348, 2348), (2348,), (2348, 20490))"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "U.shape, s.shape, Vh.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "2c0202f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "temp1 = U @ np.diag(s) @ Vh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "c6ddfa25",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Verifies that temp1 and vectors are the same.\n",
    "np.allclose(temp1, vectors.todense())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "846dda7e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "# verify U and Vh are orthonormal\n",
    "unit_U = np.eye(U.shape[0])\n",
    "unit_Vh = np.eye(Vh.shape[0])\n",
    "print(np.allclose(np.dot(U, np.transpose(U)), unit_U))\n",
    "print(np.allclose(np.dot(Vh, np.transpose(Vh)), unit_Vh))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "d4695ab6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fe23a11ed30>]"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZK0lEQVR4nO3dfYwc9Z3n8fenunvGHtvgp8ExtomBcJslt4lBE47T5nLZjZIAF62JdMeR0218WU7eP4iUSHt/kN0/NiddpOxpQ3TZB3SwoDhRLhy6JIJbcRtYghQluUCGLBgM62DAHDZ+GPyMH2amu7/3R1XPVPe0mcemp3o+L6lVVb+qrvpVe/yp6l//qkoRgZmZ9Zak2xUwM7OF53A3M+tBDnczsx7kcDcz60EOdzOzHlTudgUA1q9fH1u3bu12NczMCuWZZ555KyIG281bFOG+detWhoeHu10NM7NCkfT6xea5WcbMrAc53M3MepDD3cysBznczcx6kMPdzKwHOdzNzHqQw93MrAcVOtz3Hj7D1x/by1tvj3a7KmZmi0qhw/3lo2f4ix/v4/jZsW5XxcxsUSl0uCcSAH7eiJlZs0KHu7Jh3eluZtZk2nCXtEzS05Kek7RH0n/Oyq+U9JSkfZL+p6S+rLw/m96Xzd/aqcpnJ+4+czczazGTM/dR4Hcj4kPANuAmSTcCfwZ8IyLeB5wA7siWvwM4kZV/I1uuI9RolsHpbmaWN224R+rtbLKSvQL4XeB/ZeW7gFuz8e3ZNNn8j6uRwgussVKfuZuZNZtRm7ukkqRngaPA48ArwMmIqGaLHAA2ZeObgDcAsvmngHVt1rlT0rCk4ZGRkTlVXv5B1cysrRmFe0TUImIbsBm4AXj/fDccEfdGxFBEDA0Otr3X/LQmztzdLGNm1mRWvWUi4iTwJPDPgdWSGg/72AwczMYPAlsAsvmXAscWorKtkqRRr06s3cysuGbSW2ZQ0upsfDnwCeAl0pD/19liO4CHs/FHsmmy+T+O6Ez8Kjt3d1dIM7NmM3nM3kZgl6QS6cHgoYj4W0kvAg9K+i/APwD3Z8vfD3xH0j7gOHB7B+qdanSF7NgGzMyKadpwj4jdwHVtyl8lbX9vLb8A/JsFqd00fIWqmVl7PXGFaodafczMCqvY4e5mGTOztood7rhZxsysnUKHezJxbxmnu5lZXqHDvdHoXne2m5k1KXS4TzTLuNXdzKxJocM9mbz/gJmZ5RQ63Bs3DnOzjJlZs4KHezp0s4yZWbNih3s2dGcZM7NmxQ73iScxmZlZXsHDPR36rpBmZs2KHe6NEWe7mVmTYoe7H5BtZtZWocN98vYD3a2HmdliU+hwn3wSU5crYma2yBQ73H3jMDOztnoj3LtbDTOzRafY4T5xP3fHu5lZXrHD3T+ompm11Rvh3t1qmJktOoUO90R+zJ6ZWTuFDvfGFaq+/YCZWbNpw13SFklPSnpR0h5JX8zKvyLpoKRns9ctufd8WdI+SXslfapTlXezjJlZe+UZLFMF/igifiVpFfCMpMezed+IiD/PLyzpWuB24APA5cDfS/onEVFbyIpn2wLcW8bMrNW0Z+4RcSgifpWNnwFeAja9w1u2Aw9GxGhEvAbsA25YiMq28v3czczam1Wbu6StwHXAU1nRFyTtlvSApDVZ2SbgjdzbDtDmYCBpp6RhScMjIyOzrzm+cZiZ2cXMONwlrQS+D3wpIk4D9wBXA9uAQ8DXZ7PhiLg3IoYiYmhwcHA2b52s08S65vR2M7OeNaNwl1QhDfbvRsQPACLiSETUIqIO3Mdk08tBYEvu7ZuzsgXnrpBmZu3NpLeMgPuBlyLi7lz5xtxinwFeyMYfAW6X1C/pSuAa4OmFq3K+bunQXSHNzJrNpLfMbwO/Dzwv6dms7I+Bz0raRtoTcT/whwARsUfSQ8CLpD1t7uxET5k8R7uZWbNpwz0ifkruiXY5j77De74KfHUe9ZqRJHFHdzOzdnyFqplZDyp2uPvE3cysrWKHO+4tY2bWTqHDfbLJ3eluZpZX6HBnoitkd6thZrbYFDrcG80ybpcxM2tW6HB3T0gzs/YKHe6NG4fV3S5jZtak2OGeDR3tZmbNih3ubnI3M2ur4OHeuJ+7mZnlFTzc06Efs2dm1qzY4Z4Nne1mZs0KHe6JH7NnZtZWocNdvkLVzKytYoe7bxxmZtZWscPdNw4zM2urN8Ld2W5m1qTY4T7RLON0NzPLK3a4+8zdzKytQod74itUzczaKnS4+wHZZmbtFTvc3SxjZtbWtOEuaYukJyW9KGmPpC9m5WslPS7p5Wy4JiuXpG9K2idpt6TrO1V53zjMzKy9mZy5V4E/iohrgRuBOyVdC9wFPBER1wBPZNMANwPXZK+dwD0LXuscCZ+6m5m1mDbcI+JQRPwqGz8DvARsArYDu7LFdgG3ZuPbgW9H6hfAakkbF7riDcK3HzAzazWrNndJW4HrgKeADRFxKJt1GNiQjW8C3si97UBW1rqunZKGJQ2PjIzMtt759fgKVTOzFjMOd0krge8DX4qI0/l5kV5FNKuEjYh7I2IoIoYGBwdn89bmeuFWGTOzVjMKd0kV0mD/bkT8ICs+0mhuyYZHs/KDwJbc2zdnZR2RSD5vNzNrMZPeMgLuB16KiLtzsx4BdmTjO4CHc+Wfy3rN3AicyjXfLDy5n7uZWavyDJb5beD3geclPZuV/THwNeAhSXcArwO3ZfMeBW4B9gHngM8vZIVbCdwX0sysxbThHhE/ZfJi0FYfb7N8AHfOs14z5mYZM7OpCn2FKqT93OvuC2lm1qT44Y5bZczMWhU/3CV3hTQza9ED4e7H7JmZtSp+uOOLmMzMWhU/3CU/Zs/MrEXhwz2Rf1A1M2tV+HCX5CtUzcxaFD/ccZu7mVmr4oe7m2XMzKbogXB3P3czs1bFD3dwbxkzsxbFD3e5zd3MrFXhwz3xY/bMzKYofLj7AdlmZlMVP9z9g6qZ2RSFD3fwjcPMzFoVPtyTBHd0NzNrUfhwF779gJlZq+KHu69QNTObovDhnvgHVTOzKQof7mlXSKe7mVle4cMdN8uYmU0xbbhLekDSUUkv5Mq+IumgpGez1y25eV+WtE/SXkmf6lTFJ7YHTnczsxYzOXP/FnBTm/JvRMS27PUogKRrgduBD2Tv+WtJpYWqbDu+/YCZ2VTThntE/AQ4PsP1bQcejIjRiHgN2AfcMI/6TUuCer2TWzAzK575tLl/QdLurNlmTVa2CXgjt8yBrGwKSTslDUsaHhkZmXMlhM/czcxazTXc7wGuBrYBh4Cvz3YFEXFvRAxFxNDg4OAcq+Fb/pqZtTOncI+IIxFRi4g6cB+TTS8HgS25RTdnZR2TPiC7k1swMyueOYW7pI25yc8AjZ40jwC3S+qXdCVwDfD0/Ko4TV0Ad5cxM2tWnm4BSd8DPgasl3QA+FPgY5K2kabqfuAPASJij6SHgBeBKnBnRNQ6UvOJ+rlZxsys1bThHhGfbVN8/zss/1Xgq/Op1GykXSHNzCyv8FeoSr79gJlZq+KHO26WMTNrVfhwx80yZmZTFD7cE0H41N3MrEnhw93NMmZmUxU/3H3jMDOzKQof7on7uZuZTVH4cPcDss3Mpip8uOMzdzOzKQof7sJ3ljEza1X4cE/kh6iambUqfLj79gNmZlP1RLg72s3MmhU+3BPJV6iambUofLgDfhKTmVmLwoe7fOMwM7Mpih/u4I7uZmYtCh/uiX9QNTObovDhLvn2A2ZmrYof7rhVxsysVfHDXXK4m5m16IFw9xWqZmatCh/uvp+7mdlU04a7pAckHZX0Qq5sraTHJb2cDddk5ZL0TUn7JO2WdH0nKw9QSkTN6W5m1mQmZ+7fAm5qKbsLeCIirgGeyKYBbgauyV47gXsWppoXl0jUfYmqmVmTacM9In4CHG8p3g7sysZ3Abfmyr8dqV8AqyVtXKC6tuUzdzOzqeba5r4hIg5l44eBDdn4JuCN3HIHsrIpJO2UNCxpeGRkZI7VgJJEzWfuZmZN5v2DaqS3ZJx1ukbEvRExFBFDg4ODc95+krhZxsys1VzD/UijuSUbHs3KDwJbcsttzso6piQ3y5iZtZpruD8C7MjGdwAP58o/l/WauRE4lWu+6YgkEbV6J7dgZlY85ekWkPQ94GPAekkHgD8FvgY8JOkO4HXgtmzxR4FbgH3AOeDzHahzk1Lii5jMzFpNG+4R8dmLzPp4m2UDuHO+lZqNkm8cZmY2ReGvUJV7y5iZTVH4cC+5t4yZ2RQ9Ee7uLWNm1qzw4Z7efqDbtTAzW1wKH+6lBJ+5m5m1KH64+wdVM7MpCh/uSSIA/6hqZpZT+HAvKQ13N82YmU0qfLg3ztzdNGNmNqnw4V5qNMv4zN3MbELxw10+czcza1X4cM+yHWe7mdmkwod7pZTuQtX3/TUzm1D4cO8vp7swWnW4m5k1FD7cl1VKgMPdzCyv8OHeOHO/MF7rck3MzBaPwof7mhV9ABw5faHLNTEzWzwKH+7vXTcAwKFTDnczs4bCh/uagfTM/fjZsS7XxMxs8Sh8uC+rlFjRV+LY2w53M7OGwoc7wNqVfRw/O9rtapiZLRq9Ee4r+jnmZhkzswk9Ee7rVvS5zd3MLGde4S5pv6TnJT0raTgrWyvpcUkvZ8M1C1PVi1vrcDcza7IQZ+6/ExHbImIom74LeCIirgGeyKY7qhHu4dv+mpkBnWmW2Q7sysZ3Abd2YBtN1q7oY7Ra59yYr1I1M4P5h3sAj0l6RtLOrGxDRBzKxg8DG9q9UdJOScOShkdGRuZVibUr3NfdzCxvvuH+kYi4HrgZuFPSR/MzI20nadtWEhH3RsRQRAwNDg7OqxLrsnB3jxkzs9S8wj0iDmbDo8APgRuAI5I2AmTDo/Ot5HQmz9zd193MDOYR7pJWSFrVGAc+CbwAPALsyBbbATw830pOZ92KfgBfpWpmlinP470bgB8qfc5dGfgfEfF3kn4JPCTpDuB14Lb5V/OdrVlRAdzmbmbWMOdwj4hXgQ+1KT8GfHw+lZqtlf1lLlvVz3MHTr6bmzUzW7R64gpVSXzkfet5+rUT7utuZkaPhDvAjVet4623R3nm9RPdroqZWdf1TLh/+kMbWVZJePT5w92uiplZ1/VMuA/0lbl6cCVP7u14z0szs0WvZ8Id4Dfes4r9x84ycsb93c1saeupcN/50auIgEefPzT9wmZmPaynwv03Nqxi25bV/Plje3ll5O1uV8fMrGt6Ktwl8RefvY5yIv793zzl5hkzW7J6KtwBtqwd4IH/8GFOnhvntv/+f3n+wKluV8nM7F3Xc+EOcN0Va/jOHTdwbqzKrX/9M+5+bC9j1Xq3q2Vm9q7pyXAHGNq6lse+9C/Zvu1yvvnjffzeX/6U3b49gZktET0b7gCXDlS4+7Zt3Pe5IY6dHeP3/vJn/MG3fsnPX3mLet23KTCz3jWfu0IWxieu3cANV67l2z/fzwM/e41/d99RLr90Gf/qgxv59Acv54ObLyW7u6WZWU/QYrjR1tDQUAwPD78r2zo/VuPv9hzib587xE9eHmG8FlyxdoBPf3AjN/3T9/CByy+llDjozWzxk/RMRAy1nbfUwj3v1LlxfrTnMP9795v8/JVj1OrB8kqJbVtW8+Gta/itzav5rU2XsuGSfp/Zm9mi43CfgWNvj/LTfW/xD//vJL949Ri/PnKGRrP8moEKv7nxEt7/nkv4zY2reN9lK9m6bgWrByoOfTPrGof7HJwbq/Lim6fZ8+ZpXnzzNP94+DT/ePgMo7kulZcsK7N1/QquWDvApjXL2bx6OZvWLGfT6gEuX72MVcsqXdwDM+t17xTuS+IH1bkY6CsztHUtQ1vXTpTV6sH+Y2d5beQs+4+d5fVj59h/7CzPHzzFj/YcZrzWfKC8ZFmZ91y6jMFV/Qyu7OeyS5Zx2ar+iel1K/tZM1Bh9UAffeWe7rhkZu8yh/sslBJx9eBKrh5cOWVevR689fYoB06e5+CJ87x58jwHT57nyOkLHD0zyvDrJzh6ZvSiF1Ot7C+zeqDCmoE+Vg9UuHR5+rqkMVzWmC5PTF+yvMKqZWUqJR8YzKyZw32BJInSM/NLlnH9FWvaLhMRnL5QZeTMKCNnRjl2dpQTZ8c4cW6cE+fGOJkNT5wb58CJ85w+P86p8+NUp+mTv6ySsGpZGvQr+sos7ysxkL2WV8qT4xPDMgOVfFm5af5AJV2Hv02YFZfD/V0kaeKM/H2XTT37byciOD9e41QW9KfPV7PhOKcvjHPmQpUzF8Z5e7TK6QtVzo/VODdW5fjZMQ6cqE1MnxurNf1eMBPlRLkDRZnlleaDxEDjQFKZPGgsqyT0l0sTw/5yQn8lYVklHc8P+0oJfeWE/nJC2d8+zBaUw32Rk5SdWZfZeOnyea2rVk8PFOfGGgeB9NU4AKTzGmXVpvlnc+85faHKkdMXcu+tcX68Nq+6JSIL+vQbQ18pDf1G+E+ZV0mHlWy6UV7Jxisl0V/OT6dllVJ6IKkkolJOKCfK5iWUS6IvG1ZKCZUkN16Se0ZZoTjcl5BSIlb2l1nZv/D/7PV6cKFaY3S8zmi1zli1zmi1xoXxyeGF8drEMheqNcYmlkuHY7U6o+O1dNg6r1rn3FiVk+frTe8br00Ox6p1OnlXiVKi9ACRJC0HBqUHjMZ4ywGj6cCS5A8ykweTRnkpSddbSjRRnp8uJbmybFulbHvpvPbT5aSxncnpxvzEF+31pI6Fu6SbgP8GlIC/iYivdWpb1n1J0viG0d161OoxcaAYb7yqwVitxngtsrKgmg3H63XGq3Wq9dZ52Xi9nntfnWotJqar9Tpj1cYyLeutpQejdL3Z8rl51XowXq2n268FtS7e6ygRTWFfbnuQyR0QJg4y6TB9JbkDSzpdaTlI5eeV8+st5bc1/Xoa78nXt/Xg1Ti4VZJk4iDYmLdUvoF1JNwllYC/Aj4BHAB+KemRiHixE9szayhlvxMsp9TtqsxKvR7UIg35aj09SFTrk9O17EBUqwfVWqO8ni07OV3LDib56cllmtc5ZRvZAS6/zWpuO/npWu594+N1avVaulwtt/56Pav31HVXa539lvVOJr+9TB5MWg9A+d+G0lfzb0nN8xNW9pd577oVXHv5JR35ZjwXnarFDcC+iHgVQNKDwHbA4W7WRpKIBFEp1jFpXuq5g0rjwFDNjbcewNJlJ7/pNB8sWt+XX+fkwWk8W8fk+qYedMZr0dSEePrC+GSz4njadDiafTtsVU7Ee9cNkMzi28G//fAW/uO/uGohP9q0Lgu+xtQm4I3c9AHgn+UXkLQT2AlwxRVXdKgaZrZYJYnom2jvL95RrVaPid+TTp8f57W3zvL0/uO8fuzsrNazfmV/R+rXte8PEXEvcC+ktx/oVj3MzOailPudae2KPrauX8HvvP+ybldrQqc6Fx8EtuSmN2dlZmb2LuhUuP8SuEbSlZL6gNuBRzq0LTMza9GRZpmIqEr6AvAj0sa0ByJiTye2ZWZmU3WszT0iHgUe7dT6zczs4nxDDzOzHuRwNzPrQQ53M7Me5HA3M+tBi+IZqpJGgNfn+Pb1wFsLWJ0i8mfgzwD8GcDS+wzeGxGD7WYsinCfD0nDF3tA7FLhz8CfAfgzAH8GeW6WMTPrQQ53M7Me1Avhfm+3K7AI+DPwZwD+DMCfwYTCt7mbmdlUvXDmbmZmLRzuZmY9qNDhLukmSXsl7ZN0V7fr00mS9kt6XtKzkoazsrWSHpf0cjZck5VL0jezz2W3pOu7W/u5kfSApKOSXsiVzXqfJe3Iln9Z0o5u7MtcXGT/vyLpYPZ38KykW3Lzvpzt/15Jn8qVF/b/iaQtkp6U9KKkPZK+mJUvmb+DOYuIQr5IbyX8CnAV0Ac8B1zb7Xp1cH/3A+tbyv4rcFc2fhfwZ9n4LcD/AQTcCDzV7frPcZ8/ClwPvDDXfQbWAq9mwzXZ+Jpu79s89v8rwH9qs+y12f+BfuDK7P9Gqej/T4CNwPXZ+Crg19m+Lpm/g7m+inzmPvEQ7ogYAxoP4V5KtgO7svFdwK258m9H6hfAakkbu1C/eYmInwDHW4pnu8+fAh6PiOMRcQJ4HLip45VfABfZ/4vZDjwYEaMR8Rqwj/T/SKH/n0TEoYj4VTZ+BniJ9BnNS+bvYK6KHO7tHsK9qUt1eTcE8JikZ7KHiwNsiIhD2fhhYEM23sufzWz3uRc/iy9kTQ4PNJojWAL7L2krcB3wFP47mFaRw32p+UhEXA/cDNwp6aP5mZF+91xS/VqX4j4D9wBXA9uAQ8DXu1qbd4mklcD3gS9FxOn8vCX6dzCtIof7knoId0QczIZHgR+Sft0+0mhuyYZHs8V7+bOZ7T731GcREUciohYRdeA+0r8D6OH9l1QhDfbvRsQPsuIl/XcwE0UO9yXzEG5JKyStaowDnwReIN3fxq/+O4CHs/FHgM9lPQduBE7lvsIW3Wz3+UfAJyWtyZowPpmVFVLLbyefIf07gHT/b5fUL+lK4BrgaQr+/0SSgPuBlyLi7tysJf13MCPd/kV3Pi/SX8Z/Tdob4E+6XZ8O7udVpL0cngP2NPYVWAc8AbwM/D2wNisX8FfZ5/I8MNTtfZjjfn+PtOlhnLSN9I657DPwB6Q/MO4DPt/t/Zrn/n8n27/dpEG2Mbf8n2T7vxe4OVde2P8nwEdIm1x2A89mr1uW0t/BXF++/YCZWQ8qcrOMmZldhMPdzKwHOdzNzHqQw93MrAc53M3MepDD3cysBznczcx60P8HO4z9aHInY6oAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(s)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0f2fea7",
   "metadata": {},
   "source": [
    "The `Vh` vector is the topics vector. It is actually a matrix with each row containing significant topics for each entry in the dataset. We will limit ourselves to a few topics from each row, say 10. The below `show_topics` method extacts 10 topics from each row of the `Vh` vector and converts them back to words using the `vocab` vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "aa81bf99",
   "metadata": {},
   "outputs": [],
   "source": [
    "num_top_words = 10\n",
    "def show_topics(a):\n",
    "    top_words = lambda t: [vocab[i] for i in np.argsort(t)[:-num_top_words-1:-1]]\n",
    "    topic_words = ([top_words(t) for t in a])\n",
    "    return [' '.join(t) for t in topic_words]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "08f41c26",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['drl wovy frisco ites ios tuna headline camrys aesthetic corollas',\n",
       " 'wire wiring ground neutral subject circuit outlets gfci nec cable',\n",
       " '00 50 appears 40 art 10 dos 20 25 1st',\n",
       " '50 appears 40 art 10 80 wolverine 1st 70 cover',\n",
       " 'wire wiring appears 40 art 32 ground bit neutral os',\n",
       " 'car 40 appears brake oil 10 tires dot fluids abs',\n",
       " 'oil fluids brake dot abs tires braking tire bit 32',\n",
       " 'new copies price vs just like don know 10 45',\n",
       " 'wiring use subject gfci cec insulation cable circuits usually power',\n",
       " 'appears don 40 car just art know radar 00 rec']"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_topics(Vh[:10])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "dff8c9ad",
   "metadata": {},
   "outputs": [],
   "source": [
    "m,n = vectors.shape\n",
    "d = 10 # num topics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "cd245519",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/abishek/virtualenvs/notebooks-GQcUzVyT-py3.9/lib/python3.9/site-packages/sklearn/decomposition/_nmf.py:289: FutureWarning: The 'init' value, when 'init=None' and n_components is less than n_samples and n_features, will be changed from 'nndsvd' to 'nndsvda' in 1.1 (renaming of 0.26).\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "clf = decomposition.NMF(n_components=d, random_state=42)\n",
    "\n",
    "W1 = clf.fit_transform(vectors)\n",
    "H1 = clf.components_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "d9455242",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['wire wiring use neutral ground subject circuit cable outlets gfci',\n",
       " '32 bit mac os scsi ibm 16 93 pc 92',\n",
       " '00 50 appears 40 art 10 wolverine 1st 80 70',\n",
       " '00 dos mpc 15 20 25 35 30 world cd',\n",
       " 'car brake oil dot fluids abs tires braking cars tire',\n",
       " 'good 50 excellent missing tn de7 00 geoffrey uccxkvb fair',\n",
       " 'send requests request com edu list cars lists listserv mail',\n",
       " 'car don just like new know radar rec drive use',\n",
       " 'ground grounding conductor neutral current wire panel service run equipment',\n",
       " 'copies vs 10 copy new print annual cover issue left']"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_topics(H1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "72128f53",
   "metadata": {},
   "outputs": [],
   "source": [
    "vectorizer_2 = TfidfVectorizer(stop_words=\"english\")\n",
    "tfidf_vectors = vectorizer_2.fit_transform(newsgroups_train.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "576535fa",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/abishek/virtualenvs/notebooks-GQcUzVyT-py3.9/lib/python3.9/site-packages/sklearn/decomposition/_nmf.py:289: FutureWarning: The 'init' value, when 'init=None' and n_components is less than n_samples and n_features, will be changed from 'nndsvd' to 'nndsvda' in 1.1 (renaming of 0.26).\n",
      "  warnings.warn(\n",
      "/home/abishek/virtualenvs/notebooks-GQcUzVyT-py3.9/lib/python3.9/site-packages/sklearn/decomposition/_nmf.py:1637: ConvergenceWarning: Maximum number of iterations 200 reached. Increase it to improve convergence.\n",
      "  warnings.warn(\n"
     ]
    }
   ],
   "source": [
    "W2 = clf.fit_transform(tfidf_vectors)\n",
    "H2 = clf.components_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "a457f6e4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['use power ground current amp output voltage good input circuit',\n",
       " '00 50 dos 20 10 15 1st cover 25 new',\n",
       " 'car cars engine just like dealer good new think miles',\n",
       " 'drive hard scsi disk drives internal floppy cd external cable',\n",
       " 'offer sale shipping condition asking best sell email edu new',\n",
       " 'b30 ingr dtmedin catbyte 205 al huntsville ssd 35894 medin',\n",
       " 'card nubus video pds cards lc fpu cache slot vga',\n",
       " 'thanks does know mail advance info like looking appreciated hi',\n",
       " 'simms vram simm memory meg 256k need quadra ram pin',\n",
       " 'apple mac problem monitor duo se keyboard time hardware quadra']"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "show_topics(H2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "86e852a0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fe23a154d00>]"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8/fFQqAAAACXBIWXMAAAsTAAALEwEAmpwYAAAj9UlEQVR4nO3deZgU9Z0/8PdHjO4mcV2NPFlXXcck/uLD7nrOj9XfJq67P1dRo0ST/IK5TFYXY2R1n8Td4HokDyEe4BUJGgFPPBAUFWXkFFBuZriGgQGGYZgDhjkYZgaGuT+/P7p6qOmp7q7urrvfr+fhoae6uurb1VWf+tb3FFUFERGF3wl+J4CIiJzBgE5EFBEM6EREEcGATkQUEQzoREQRcaJfOz7jjDO0oKDAr90TEYVSSUlJk6oOt3rPt4BeUFCA4uJiv3ZPRBRKIrIv2XssciEiiggGdCKiiGBAJyKKCAZ0IqKIYEAnIooIBnQioohgQCciiggGdCIim7bVtWJzzWG/k5GUbx2LiIjC5ltTVgIAqh67weeUWGMOnYgoIhjQiYgiwlZAF5FRIrJTRCpEZLzF+z8VkUYR2Wz8u8P5pBIRUSppy9BFZBiAqQD+FUAtgA0iMk9Vtyes+raqjnMhjUREZIOdHPpIABWqWqmq3QBmARjtbrKIiChTdgL6WQBqTH/XGssSfUdEtorIOyJyjiOpIyIi25yqFP0QQIGqXghgMYBXrVYSkbEiUiwixY2NjQ7tmoic8tjH5Vi2s8HvZFCW7AT0OgDmHPfZxrIBqtqsql3GnzMAXGa1IVWdpqqFqlo4fLjlhBtE5KM/rdiDn728we9kUJbsBPQNAM4XkfNE5CQAYwDMM68gImea/rwJwA7nkkhERHakDeiq2gtgHICFiAXq2apaJiITROQmY7V7RKRMRLYAuAfAT91KMJEdRaUHcPNzq/xOBpGnbHX9V9UiAEUJyx42vb4fwP3OJo0oe794Y6PfSSDyHHuKEhFFBAM6EVFEMKATEUUEAzoRUUQwoBMRRQQDOhFRRDCgExFFBAM6EVFEMKATEUUEAzoRUUQwoFOkqarfSSDyDAM6EVFEMKATEUUEAzoRUUQwoBMRRQQDOhFRRDCgExFFBAM6EVFEMKATEUUEAzoRUUQwoBMRRQQDOhFRRDCgU6RxKBfKJwzoREQRwYBORBQRDOhERBHBgE5EFBEM6EREEcGATkQUEQzoREQRYSugi8goEdkpIhUiMj7Fet8RERWRQueSSEREdqQN6CIyDMBUANcBGAHgVhEZYbHeKQDuBbDO6USSfzZWt+CJhTv9TgYR2WAnhz4SQIWqVqpqN4BZAEZbrPc7AI8D6HQwfeSzW55bjT8uq/A7GURkg52AfhaAGtPftcayASJyKYBzVHV+qg2JyFgRKRaR4sbGxowTS5Qp9vynfJJzpaiInADgKQC/Sreuqk5T1UJVLRw+fHiuuyZKSsTvFBB5z05ArwNwjunvs41lcacA+DsAy0WkCsDlAOaxYpSIyFt2AvoGAOeLyHkichKAMQDmxd9U1VZVPUNVC1S1AMBaADeparErKSYiIktpA7qq9gIYB2AhgB0AZqtqmYhMEJGb3E4gERHZc6KdlVS1CEBRwrKHk6x7Ve7JIiKiTLGnKBFRRDCgExFFBAM6EVFEMKC77Fh3HwrGz8dLK/f6nRQiijgGdJe1dHQDAKZ/VulzSogo6hjQiYgiggGdIk2Vo7lQ/mBAp0jiUC5AzaEOdHT3+p0M8hADOlFEfXPSMtz20nq/k0EeYkAnirANVS1+J4E8xIBORBQRDOhERBHBgE5EFBEM6B5h6zkichsDuss4FRoReYUBnYgoIhjQiYgiggGdiCgiGNAp0lgXTfmEAZ0iSVgbTR6pOdSB55ZXBGIgOFuTRBMRkbXbXl6PysajuPmSs3DmqX/ua1qYQyciykFHV5/fSRjAgO4RZWmup4Lw+EvkNQZ0lwlH5iYijzCgExFFBAM6EVFEMKATEUUEAzpFSlXTUTS2dzm2rb5+Vq5SeLAdOkXKVU8sBwCckGNddHVzB656Yjnuuuqr+PWoC3JPGJEHbOXQRWSUiOwUkQoRGW/x/s9FpFRENovIShEZ4XxSiTKXbevFhvZOAMD6vYccTA2Ru9IGdBEZBmAqgOsAjABwq0XAflNV/15VLwYwCcBTTieUKBPs+k9eCVIfEzs59JEAKlS1UlW7AcwCMNq8gqq2mf78AjgmEhHlmSD0ObET0M8CUGP6u9ZYNoiI3C0iexDLod9jtSERGSsixSJS3NjYmE16Q8uvjovb6loxa321PzsnIk851spFVaeq6lcB/BrAg0nWmaaqhapaOHz4cKd2HWh+P/l/a8pKjJ9b6m8iiMgTdgJ6HYBzTH+fbSxLZhaAb+eQJiIiyoKdgL4BwPkicp6InARgDIB55hVE5HzTnzcA2O1cEomIyI607dBVtVdExgFYCGAYgJdUtUxEJgAoVtV5AMaJyNUAegC0ALjNzUQTeYWjNlKY2OpYpKpFAIoSlj1sen2vw+ki8pXfdR9E2WDXf5cxg0eUn2atr8a2ulZP98mu/x5hjo8ov8Rbl1U9doNn+2QOnYgoB0F6CmdA90iQfvR8EqRu2RRtQXgKZ0B3WRB+ZD+sqmjCil3+9QbO08NOeY5l6OSKH85YB8Db8kOifMccOlEKLLChMGFAJ7LEQhsKHwZ0IqKIYEAnIooIBnQioohgQCciiggGdCKiiGBA9wibvxFFU/zaDkK7KAZ0lwXhR6bscciGcLrr9RL8wyNLPNmXBiiis6coRVq2ATlfh2yIio+31fudBF8wh06R5HdA7uzpw5hpazwfD5v8tf/wMTQf6fJt/wzoRC7YVteKtZWH8Jt5ZX4nhTz0fx77BJdNHFzUs66y2bP9M6ATEbno+9PWerYvBnQiF5Xsa/E7CZRHGNApr3T29KFg/HzMWl+dcj0nW7f09PXjwfdL0dDe6dxGiSwwoFNeOdzRAwB4eskuW+s7Ubm6dMdBvL62Gr/5gOXp5C4GdI9Y5fgOd3SjYPx8LCrLzyZWYeBEs8f4NtimndzGgO62FDm8XQePAACmf1bpUWLILr+bPVKYBOdOzYBOROQACUBXUQZ0IqKIYEAncoX/uTXKPwzoREQRwYBOkZSuPJMtTiiKbAV0ERklIjtFpEJExlu8/0sR2S4iW0VkqYic63xSiXKXaeuVZHF/Y3ULFrK5aegt2HYAe5uO+p0Mx6QN6CIyDMBUANcBGAHgVhEZkbDaJgCFqnohgHcATHI6oUReShf3b3luNe6cWZL88yxCD4Wfv74R//zEcr+T4Rg7OfSRACpUtVJVuwHMAjDavIKqLlPVDuPPtQDOdjaZRESUjp2AfhaAGtPftcayZG4H8LHVGyIyVkSKRaS4sbHRfiojgYW2RMVVh7D7YLvfyYgsRytFReRHAAoBTLZ6X1WnqWqhqhYOHz7cyV0Hlp+dDZQ1fxQw3/3TGvzr05/6nYyc9fcrOnv6/E7GEHYCeh2Ac0x/n20sG0RErgbwAICbVNW/KTtowD9NXu53EvIWi9Cj7fEF5bjgoQU41t0XqBZTdgL6BgDni8h5InISgDEA5plXEJFLALyAWDBvcD6Z4aU+FrVUH+pIvxIRZWxOSS0AoKO7d2DZb+Zt8ys5A9IGdFXtBTAOwEIAOwDMVtUyEZkgIjcZq00G8EUAc0Rks4jMS7K5PMY8W5AEKFNFEVFU6n8z1hPtrKSqRQCKEpY9bHp9tcPpooBZVt6An72yAVsevganfv5zficnaxnfVoP0PE2UBnuKki1Tl1UAAHaGrIVC9uOZ5/ZElevng+C7z6/Gu0bRAoUDAzpFk8/x1O0WRrPWV6Oiwd2ba/G+FvxqzhZX9xFWQW1BZqvIhYiCY1tdK8bPLcWJJwgqHrne7+TktaA9iTGHThQy35qyEgDQ2x/MXKITWjt6sGT7Qb+TkVbQcuoM6J4J1g+fj2Z8VomRjywd+Lu7t9+1ziFWOTc/m7A6SVWxZk+zq8Hs7jc34o7XinGwrdOV7R/u6Ma+5uwH5QpazjyOAd1lQZiWimImzt8x8FoAXDV5GS54aIHr+w3otZ+1otJ63Dp9Ld5YV+3aPqqMYNvd25/TduoOH7NcfvVTKyLZ8Y4BnfKSAtjf6k7uL53Ve5rQcrTbl307obYl1mEtDB3XZnxWifbOniHLm47kdvzNTydBeu5iQM9za/Y0o2D8fDQf8W60htaOHvSHpPzX6VT29vXjB9PX4UcvrnN4y9GUa6nOy6uq8F9ztjqTGAsiEqhydAb0PDf9s0oAwOaaw57s79DRblw0YRGeXLzT3R3leI25VUoSv4/tCll7fq85WUxVezj4TxJOYUAnAEB9WydmF9ekXS/X3Miho7EngQXb/O8m7aaIFZtTEkHKnQNsh06GB96LDSx07Yi/suzaH7WKPb9EpaULELxg5iW2cskDW2oOY82eZr+TkZN+jy7SoIeCXNNX22LduiIuzK2fvIxlQb0BBvVmxoDuoNFTV+HW6Wv9TkZac4prUDB+Po529aZf2XHeBrJcA0I2qd3XfBR3v7nxeBqCee2jo7s3sJ13srnhdfX24Z2SWk+DbdBy6gzoHgnSRf388j0AYuXmQc1p5MzH6yxZ+2aBBOo8ePD9bbjjtWKU17f5nRRHPLVoF+6bswVLduTvlAwM6C4L2A08a0EKRE7w/fsE4LzY1xxr/XGk048nNXsSf6dXVu3F+5uGTJgGAGhoj1W4W7U7zxesFM1jbcd8PPH9Dqg2hSSZOVEAv/toOy48+1S/kzIgWUbotx9uBwB8+5Lk89Qn3gSyvXnP3lCDC885FRf81V8MPNUO3VewzhAG9Dx283OrcZrNySqcOm3D8sTidjKDcBjMaXhx5d6MPx+wWOb4Mf3vd2MdkqoeuwGPLyh3eOvuYJFLnmvpyCyXbq4Een9THa54dCn6QtLrMwiCFgSzEeYWOk4TkUA9xTGHToNkcnLeP7cUx3r60NXbh8+flNmpFKSLwG19FlE8SE8qQb7JZJO0AH8d1zGHHiKjnvkUD71vb2bxN10cCS/Rur2HbK8boDjmmXFvbvI7CZa8uKk0tnfh6w9+7M3QEvl4ciVgQA+R8vp2zFy7L+161c0d+J/3Sl1Jg1Ul0M9e3uDKvqIoiB1l3KzYW72nCV29/VmV0QPZxWgvKip7+oL3OwIM6JHU3ZdmDOksrhKrj+QSnILWOsBrTpVDL9l+0PJYttpowRSGsvBkZ8m2utaB1w1tnTjc0e3p9zniS6e89EIb0KuajmJjdYvfyQioNMHSx1gar1Stb+vEhA+3+1ihGo0byh2vFWNh2dDenmFvi50uNMen4QOAkY8sxcUTFruboJAIbUC/6onluOW51X4nwzYnw4eqYlFZfdIxxXPJ/Na4PGnBxn2xm3BnTz9eWrUX6zMof/eD0w8S8e05WX7daDGWfSbpjsat7biofZ9MhDagh4UbD4HvbarD2JkleG1NVVafr2xKPpfi6KmrbG8nm2D39JJdCdtw9/LLdvNuVRjGk+Po5rP+kk4mwn9BajnkFzZbDKGDbbEc2YEkE+gGOYfi1UXHa9u+TO4Hd860LuJxS1Y3/CBfAC5jDj3gFpXVo2x/q/WbPp+4TgRnJ79Cb5rKYL9zcG61cPHyNPAqmGcziqGTP++Upbs936cTGNADbuzMEtzw7MpBy9Kd67mWYizefhBLd6S/cM37+cUbJfhgs/WgSWZutkTYcSCTad2Op0NVMW/LfvSkax3kgCC27sn2Rre/NfmY7zWHOrCw7PisVF58byf38OTiXelXiu83QD8pA7oD4hMtp3LI5izv1c0dKK6yV1GY7DzKNSf4768V4/ZXi23tK66otB73ztqcdtuJwcPLiyFZ3FpYdhD3vLUJUz6p8C4tDj4uWB3DjCpFs/wNjnb1JX3vuj98hjtnluT8PQMUKy0FLX0sQ3fADGOiZSdcOXkZgNiAQMmku0SsLtBfvr0ZJ3/uBDx6y4U5pC6aWjpiN9uGJHUSUeVmcUGu7bSzSVvQij/8YCuHLiKjRGSniFSIyHiL968UkY0i0isi33U+mWQlk8fYuZvq8Nb69JNAp91nzltI3J5zW8xsW0PX9eJpwY1d5FqcYXXcDnfYe6IMolTnwfdfWIOJH233MDXeShvQRWQYgKkArgMwAsCtIjIiYbVqAD8F8KbTCQwDryvb3C5Dt9yn1X4c2I6qd+XKyR7/40utAoHTFZkD7dAd3GZvv6I1YdTMZOnu6evHG+v2oa9fB86jH0xfN2Q9pzrqZPPbLitvSNm0Nplk14U5Cev2HsKMLIchsNynY1tyhp0c+kgAFapaqardAGYBGG1eQVWrVHUrAPdrlfLEOyW1+HDL/pTreFn+HN/VvyeUrWcqMaj+5KX1vvfyswoErncjt9h8zaEOjHrmUzRZdBRKZeL8HbhowiJbwfPFlXvxwHvbMGtD7oO3ZXL6pStLV1U8+vEO7DjQhl/N2TKwvKEts2OR7+wE9LMAmJ/Va41lGRORsSJSLCLFjY2N2WwioJy/+O+bswX/8dbxUfrmmYJ7umDj5gBQ7aayUafKOe2MO+IFT26QKfbx4sq9KK9vx7zNqW/kuYjXF7Qdy30sEqeerGau3YcNVS14YUUl/t8Lawa9l82k60FqdeI1T1u5qOo0VS1U1cLhw4d7ueu0nl68K21LFT/dYwrucUlbuXh0Qme1G4fufffP3YqZCT1l031vvx+PzSM1pEpLtj+f+funOxb9qujq9e6BOlXwf+j9bQOBXDX73ykMg425zU4rlzoA55j+PttYFil/sNmRwEquZej1bZ1o7+zBKX9mbzo4PzrIdGdw8WeybrbiFbw/vqIg5Xp2guPug0dsr5sL8wiBVuK/qxd1CpMX7nR9H4CzzTPtyuMMuq0c+gYA54vIeSJyEoAxAOa5m6zwO9bdh4Z2e83gag4dw6hnPrO97fgIhebrvrLxiKsjF5amCUZmx7qTt092Wke3ddGBZbm4aVnTkeOtOHp9nkLvxikr8Z3nVyfNYda2dKTtBZsorEFNVbPOsPjdEzgI0gZ0Ve0FMA7AQgA7AMxW1TIRmSAiNwGAiPxvEakF8D0AL4hImZuJDhqr8+jW6Wsx8vdLbW+j7nDyXneJnlw0uBdbRcMR/MuTKwaeMjwrQ3RxP88u3Y1v2xgo7MpJy21vM91xsXq/qsnZ0Set6jdK61pRss96KOiG9k584/Fl+H3RDhvb9tZnu5vQ2ePGzdv7yFzRcGSgd3QQe/PaZatjkaoWAShKWPaw6fUGxIpi8lKLRZtdN6fcik9gEQ8OB40OMc8u3Y0/+9wJ+ObXglU/YWb3Un3KZtfrpiNdONLViy+e7E4fOScnMjDHCauiCKsc5mGjOeLK3U1Z7/fFlXvR2dOHu//5a1l9ftQzn+LHV5yLH/7DuUPeu+ChBdjym2tw6p/bKy5Mx4lQmk08vvqpFQBiHfraM/zNg3QDYNd/B2yoCs5EG5MW7EzZyqWrN/cclarihRV7ks+MZBGYOnv68LqN6fNS+XDLfrRZTNxwy3OrcOOUldjdcCTl59M9kr+7sRY/mjG0TbaTUl37x3OI9tYfum3Fmj3NeGrxrkFB5ncfbc+pzLy8vh0PvJd8LlsnOyF1dPcN+Z3sTtaR7vetb01fBJppk1EgWMVb7PqfZ77+4IKct1Fe345HPy7P6DOPLyjHy6uqst7n7oPt+I+3NuHav/0yXvhx4aD3dhmVmk9nMKBSMisrYjlh18dDt9h+VXOHsY6iq7dvSGchO+LN/E7/vDM5ZjvK9rfh3C99wbXttx6z32AASN5s95iN4qHCiUvwhzEX294XALR3Bmc6OubQI8jtJ8DeNBPkWgUru4OTJdNhVLTuP5z9eCuZNGtz+ximS8ldr2/EyEfs18EAwKLtx0fI/O2H3nVv/8UbG3E0TTFFRp2Qsk5J6k/a3e7GJPUZYRDZgP5J+UF8tNW9DhqpeFWmlmw3/QEq08tGRcPQYXAHmvSlCA2ZVCwH3SflDQCOf187TwzvlNSmXcetttp/+5uFloPUeVu96cx570ajJ69iUWQD+r+9Uoxxbw7tjOOFsv1tA6/9aIfrN6tvnMlR6OwZXDY/a3019jSmLh8PCzs3e7fux01HuvCnFXvc2ThiQxDcP3froCaWTnyVeVv248YpK9OvmMSO+tj1aPdSdCND5FUsYhm6C7zsgQdYDHjl6d6Hcnr/4+eWHt92LhsP0L3VjRt9upvFfaYxUtzy1voa/M3pQ8vTM/m2iYdm0gK7FbrWezk+IJq9VLjVLWHW+mrUthzDfdd+3Z0dIMI5dH9ZnxGqiupmZ9s1J99bNCR2HApzaVK9zfHWrb6inWCU7tCkK+d2yuMLMqswd1qyc8T+PdSdk2z83FL8cZm7k6iEPqBvqTmM8vq29Ctm4NDRblemI7v2mU9x5eRlKNlnb0aidJLlyBrbnRmhzq8y6fL642XoszcMHsM920utoa0zpyEJKhqO4H89+DFqDjl/Q06Uqtnigm31yFZYxjrJrPL6+AE6Xs+Sm/4MTpOglaiGvshltNGbMNUMP5m69HeLccPfn4mpP7zUsW0Cx5vX7Wk8isvOPd2x7SaeU/9pYyo4Ox58rzT9SjZlUsRgLhrIeQoz44K/5plPc9rOnJIadPf246OtB3DeGUOLFPr7FV/5nyKLTyakx/g/22/189dLsvxkDjvNwVvrMh+mN5Of3Nyyp89ofTUpwya1iTIpQw/aE2Poc+hxzy93prInHgDmlx5wZHtWkp2vmeb+kp1LTj1dZBtM3TzJ7bYgSszlHc6wTXeqr24OqqsqmrCv+Sj6bKZroDw3xfbdGv7Yj8zkmspmy+WrK5rwnedXZzxGTSJzR6Cq5tikGJn29Ezk89A+OYlMQPe73M4J35y0LKvPzUlorubUYFNOBoCsh0QN2CNtoh/OWId/mrwc729ybgBSqyIXkfQ3s3T3lHV7nSnqy0Zi0n41ZwtK9rXgoEXxYEYVqKa1030/u+fSuxvTN/8MqsgEdMCZ9t9ePEI51cIhntb3HAwmZlkn08FjmOsIkst3NuLyDDvoZOO/3tlqa71F22Nl4M1GR6sPNqf+7cyVwn4/3jt5HOOn1j8+9klu27F5jj65aCeaj4R3nlS7Ql+GbrawrB6j/u7MnLaRfNIIxZrKZlzxlS9lFJB7k9Sw3DhlJU4cFvDsZ5K80o1/TN4m+LGPyx1t62xu05+NifO342AW05itTVJUkKtXjOEPVGMDuN2bpr7jzpnHi3fSxXO3473dVjq5yuT6srvmlE8qsHRHQ3YJCpFI5dCPdrk3Dvf80gP4wfR1eCPDSp7bXxk6B6cgNmTqpurDOaXJzanmgOxy6E53XNnXPHiyYLu51Pi4HXZaTCROgdffr5g4P/1wtdkwH1M7zQibTUMmhL0HsNn+FANlZfKk/WQG4/ckHUwuQiIV0J044ZOdTLUtsSZ8mVZcZjIxRKb6FSic6N4Ey4tNLQiy9Yclu9Hfb39esTV7BueME0eyVChKa+0fUztjyCQWWc1YObgLu6qiq8ehiuYsaxPK69ttjOcejYCfKtibff+FNRk10Y3K8UklUgHd6Z9riSmgDdwsTNfj3I21Q+a1tMOpir6e3v5BM+8E0dNLdmHJjoOYuzF9Of+y8oa0kwIf7epLWeSTKJtc2SNFgyvY69s68crqqoy3Y8X823ckmdkpWeDJdYCzqPGzkjcuWczZnqKo8JqnV7iTGESsDN2JiG7exIHW4x1rrLoP/3K2dVfqdBmBZJ/LVGLrlqAaOzN92+m+fsX+1vQdmfx4bHarQ86kJC2znlhkXYwQ5iKXIwEaYtYL1z+bfErJeH8UN0Qqhx6XSxdn8zVj1cAi6M3owuqOVzfYaivux+F3sq7C3As2sew+V+kmofbTil2NsaK3HBS52DckWxuq/H9KMItUQP/vd2NNx6Z84sx4CeZH3/jj7gk2Isqc4nDknINk2c5GW7PqNDg0rEEmrng0t6Z1yWSaOUgXDluymBDDS1tqD+f0+V+8sTGnz2d7O5ldXJP0vTttPH16KVIBPS6XMTvMubH4q+rmDry4ci8Ae4/fb6c4AYjiMm1OGfZKvVueX+13ErLye5daPLkhkgE9l2IRq156V04+3oOTRS757VWHKkfzkd/3o8rGo+lXshCmaz5alaIOm/DRdlz8N385aFmIfltygVeda6z4HRDz1QkhiuiRzKGbNWc4i3fiRfNI4uOWSKSmOqPwuPm5cBZZhF14wnkeBPTLJi7B/XNLUdEwtKlQwfj5KBg/f9CyxBYNiU3FZnxWiX987BPsPjh03ksiNzVlmDkhZ4RpGslIBvT5Wwc3b3prfTXuSjGOdEOKx+iNCd3z451BqlyYeYiIgidMN9JIBnSrcs5hpvaG1c0dg8Zhft40/kiy3nuJ7nnLnwmoiYiSiVylaLJxPsrr2/Hrd7YONCm8xFTZ+bIxAh4ATPxou639xAd/IiIKilDm0FNNtJxqnA9z+/BkIx2+v3l/1ukiIvJTKAO6uV04ERHF2AroIjJKRHaKSIWIjLd4/2QRedt4f52IFDieUkMu47QQEQXBil2Nrmw3bUAXkWEApgK4DsAIALeKyIiE1W4H0KKqXwPwNIDHnU5oXGILFiKisHGrUYWdHPpIABWqWqmq3QBmARidsM5oAK8ar98B8H/FpcabD36wzY3NEhF55pjN1nSZshPQzwJgHm2q1lhmuY6q9gJoBfClxA2JyFgRKRaR4sbG7B45nvjeRVl9jogoKCZ/70JXtutps0VVnQZgGgAUFhZmNTLFTRf9NW666K8dTRcRURTYyaHXATjH9PfZxjLLdUTkRACnAnBn2nQiIrJkJ6BvAHC+iJwnIicBGANgXsI68wDcZrz+LoBPNOyDNxMRhUzaIhdV7RWRcQAWAhgG4CVVLRORCQCKVXUegBcBzBSRCgCHEAv6RETkIVtl6KpaBKAoYdnDptedAL7nbNKIiCgToewpSkREQzGgExFFBAM6EVFEMKATEUWE+NW6UEQaAezL8uNnAGhyMDlRxmNlD4+TPTxO9rh5nM5V1eFWb/gW0HMhIsWqWuh3OsKAx8oeHid7eJzs8es4sciFiCgiGNCJiCIirAF9mt8JCBEeK3t4nOzhcbLHl+MUyjJ0IiIaKqw5dCIiSsCATkQUEaEL6OkmrM4HIlIlIqUisllEio1lp4vIYhHZbfx/mrFcRORZ43htFZFLTdu5zVh/t4jclmx/YSEiL4lIg4hsMy1z7LiIyGXGca8wPuvKNItuS3KcfisidcY5tVlErje9d7/xnXeKyLWm5ZbXojHU9jpj+dvGsNuhIyLniMgyEdkuImUicq+xPLjnlKqG5h9iw/fuAfAVACcB2AJghN/p8uE4VAE4I2HZJADjjdfjATxuvL4ewMcABMDlANYZy08HUGn8f5rx+jS/v1uOx+VKAJcC2ObGcQGw3lhXjM9e5/d3dvA4/RbAfRbrjjCus5MBnGdcf8NSXYsAZgMYY7z+E4C7/P7OWR6nMwFcarw+BcAu43gE9pwKWw7dzoTV+co8UferAL5tWv6axqwF8JciciaAawEsVtVDqtoCYDGAUR6n2VGq+ili4/GbOXJcjPf+QlXXauxKfM20rVBJcpySGQ1glqp2qepeABWIXYeW16KRw/wXxCaLBwYf81BR1QOqutF43Q5gB2LzJwf2nApbQLczYXU+UACLRKRERMYay76sqgeM1/UAvmy8TnbM8uVYOnVczjJeJy6PknFGUcFL8WIEZH6cvgTgsMYmizcvDzURKQBwCYB1CPA5FbaATjHfUNVLAVwH4G4RudL8pnG3Z3vUBDwuKT0P4KsALgZwAMCTvqYmQETkiwDeBfCfqtpmfi9o51TYArqdCasjT1XrjP8bALyH2OPvQeMRDsb/DcbqyY5ZvhxLp45LnfE6cXkkqOpBVe1T1X4A0xE7p4DMj1MzYkUNJyYsDyUR+RxiwfwNVZ1rLA7sORW2gG5nwupIE5EviMgp8dcArgGwDYMn6r4NwAfG63kAfmLUwF8OoNV4XFwI4BoROc14vL7GWBY1jhwX4702EbncKCf+iWlboRcPUIabETungNhxGiMiJ4vIeQDOR6wiz/JaNHKsyxCbLB4YfMxDxfidXwSwQ1WfMr0V3HPK75rkLGqer0estnkPgAf8To8P3/8riLUo2AKgLH4MECu7XApgN4AlAE43lguAqcbxKgVQaNrWvyFWyVUB4Gd+fzcHjs1biBUX9CBWHnm7k8cFQCFigW4PgD/C6Gkdtn9JjtNM4zhsRSwwnWla/wHjO++EqRVGsmvROEfXG8dvDoCT/f7OWR6nbyBWnLIVwGbj3/VBPqfY9Z+IKCLCVuRCRERJMKATEUUEAzoRUUQwoBMRRQQDOhFRRDCgExFFBAM6EVFE/H/9SzzzHe0YYgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(clf.components_[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "42c56358",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "46.66649095524218"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.reconstruction_err_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "a1597ea7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/abishek/virtualenvs/notebooks-GQcUzVyT-py3.9/lib/python3.9/site-packages/sklearn/utils/extmath.py:368: FutureWarning: If 'random_state' is not supplied, the current default is to use 0 as a fixed seed. This will change to  None in version 1.2 leading to non-deterministic results that better reflect nature of the randomized_svd solver. If you want to silence this warning, set 'random_state' to an integer seed or to None explicitly depending if you want your code to be deterministic or not.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 538 ms, sys: 269 ms, total: 807 ms\n",
      "Wall time: 293 ms\n"
     ]
    }
   ],
   "source": [
    "%time u, s,v = decomposition.randomized_svd(vectors, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "d28b3a85",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 116 ms, sys: 92.4 ms, total: 208 ms\n",
      "Wall time: 69.9 ms\n"
     ]
    }
   ],
   "source": [
    "import fbpca\n",
    "\n",
    "%time u1, s1, v1 = fbpca.pca(vectors, 10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "f036523f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((2348, 10), (10,), (10, 20490), (2348, 20490))"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "u1.shape, s1.shape, v1.shape, vectors.shape"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.9"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": true,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": true,
   "user_envs_cfg": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}