Algorithm+of+PCA+Demonstration.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Algorithm of PCA demonstration\n",
    "\n",
    "In this demonstration, we'll utilise the algorithm of PCA to perform dimensionality reduction on the roadmap example that you saw earlier. This demo is on a 2-D example, but can be extended to any dimensions. You are advised to follow the same code for higher dimensional dataset as well to check the workings of PCA."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Let's import the necessary libraries\n",
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>1.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>6</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>7</td>\n",
       "      <td>3.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>8</td>\n",
       "      <td>4.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   X    Y\n",
       "0  2  1.0\n",
       "1  3  1.5\n",
       "2  4  2.0\n",
       "3  6  3.0\n",
       "4  7  3.5\n",
       "5  8  4.0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Let's write down the coordinates of all the locations\n",
    "#And store it in a dataframe.\n",
    "a = [[2,1],[3,1.5],[4,2],[6,3],[7,3.5],[8,4]]\n",
    "b = ['X','Y']\n",
    "data = pd.DataFrame(a,columns = b)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now proceed and use the algorithm of PCA to find the Principal Components"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Step 1: Covariance Matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[5.6, 2.8],\n",
       "       [2.8, 1.4]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Let's find the covariance matrix of the dataset\n",
    "\n",
    "C = np.cov(data.T)\n",
    "C"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see variance along X is 5.6, along Y is 1.4 and the cov(X,Y) = 2.8. Let's go ahead and perform it's eigendecomposition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Eigendecomposition of Covariance Matrix"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For finding the eigenvectors, we'll be using numpy.linalg.eig function.You're advised to take a look at its documentation to understand the workings of it.\n",
    "https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eig.html"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Let's find the eigenvectors and the eigenvalues of the covariance matrix\n",
    "eigenvalues, eigenvectors = np.linalg.eig(C)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.89442719, -0.4472136 ],\n",
       "       [ 0.4472136 ,  0.89442719]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Let's check the eigenvectors\n",
    "eigenvectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The eigenvectors are given along the column as per the documentation. Therefore the eigenvectors are given as v1 = (0.89442719,0.4472136 ) and v2 = (-0.4472136, 0.89442719)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 7.00000000e+00, -2.22044605e-16])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Let's check the eigenvalues corresponding to them\n",
    "eigenvalues"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, the eigenvalues corresponding to v1 and v2 are 7 and 0 respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Note that the eigenvalues are already sorted in decreasing order\n",
    "### Therefore you don't have to change the basis vector matrix \n",
    "### However if it isn't in the order, you can use the following code to do the sorting\n",
    "##idx = eigenvalues.argsort()[::-1]   \n",
    "##eigenvalues= eigenvalues[idx]\n",
    "##eigenvectors = eigenvectors[:,idx]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3:  Tranformation of Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now all we need to do is represent our data in the basis given by the eigenvectors. This involves taking the inverse of the eigenvectors matrix and multiply with the original dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Let's find the inverse of the eigenvectors matrix.\n",
    "M = np.linalg.inv(eigenvectors)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.89442719,  0.4472136 ],\n",
       "       [-0.4472136 ,  0.89442719]])"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "M"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Now we go ahead and obtain the new set of datapoints\n",
    "#By multiply M with each datapoint\n",
    "#We'll take a transpose of the orginal dataset to do this directly for all the points\n",
    "\n",
    "Data2 = M @ data.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 2.24, -0.  ],\n",
       "       [ 3.35, -0.  ],\n",
       "       [ 4.47, -0.  ],\n",
       "       [ 6.71, -0.  ],\n",
       "       [ 7.83, -0.  ],\n",
       "       [ 8.94, -0.  ]])"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Now let's do a transpose again to find the datapoints in the new basis representation\n",
    "NewData = Data2.T\n",
    "#Rounding off for better readability\n",
    "NewData.round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see the second column is completely redundant now.Because all the values are 0 now. If we drop it we're still retaining all the information"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 7., -0.],\n",
       "       [-0.,  0.]])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## Check the covariance matrix now\n",
    "np.cov(NewData.T).round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As we can see the covariance matrix has been diagonalised"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}