Assignment-Notebook.ipynb

{
    "nbformat_minor": 2, 
    "cells": [
        {
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src = \"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width = 400, align = \"center\"></a>\n\n<h1 align=center><font size = 5> Classification with Python</font></h1>", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 109, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [], 
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        }, 
        {
            "source": "### About dataset", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets download the dataset", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 110, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--2018-10-28 19:21:31--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: 'loan_train.csv'\n\nloan_train.csv      100%[===================>]  22.56K  --.-KB/s    in 0.001s  \n\n2018-10-28 19:21:32 (18.5 MB/s) - 'loan_train.csv' saved [23101/23101]\n\n"
                }
            ], 
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        }, 
        {
            "source": "### Load Data From CSV File  ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 111, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 111, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        }, 
        {
            "execution_count": 112, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "(346, 10)\nUnnamed: 0         int64\nUnnamed: 0.1       int64\nloan_status       object\nPrincipal          int64\nterms              int64\neffective_date    object\ndue_date          object\nage                int64\neducation         object\nGender            object\ndtype: object\n"
                }, 
                {
                    "execution_count": 112, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>count</th>\n      <td>346.000000</td>\n      <td>346.000000</td>\n      <td>346</td>\n      <td>346.000000</td>\n      <td>346.000000</td>\n      <td>346</td>\n      <td>346</td>\n      <td>346.000000</td>\n      <td>346</td>\n      <td>346</td>\n    </tr>\n    <tr>\n      <th>unique</th>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>2</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>7</td>\n      <td>23</td>\n      <td>NaN</td>\n      <td>4</td>\n      <td>2</td>\n    </tr>\n    <tr>\n      <th>top</th>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>PAIDOFF</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>9/11/2016</td>\n      <td>10/10/2016</td>\n      <td>NaN</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>freq</th>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>260</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>166</td>\n      <td>90</td>\n      <td>NaN</td>\n      <td>151</td>\n      <td>294</td>\n    </tr>\n    <tr>\n      <th>mean</th>\n      <td>202.167630</td>\n      <td>202.167630</td>\n      <td>NaN</td>\n      <td>943.641618</td>\n      <td>22.653179</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>30.939306</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>std</th>\n      <td>115.459715</td>\n      <td>115.459715</td>\n      <td>NaN</td>\n      <td>109.425530</td>\n      <td>7.991006</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>6.039418</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>min</th>\n      <td>0.000000</td>\n      <td>0.000000</td>\n      <td>NaN</td>\n      <td>300.000000</td>\n      <td>7.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>18.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>25%</th>\n      <td>107.250000</td>\n      <td>107.250000</td>\n      <td>NaN</td>\n      <td>900.000000</td>\n      <td>15.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>27.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>50%</th>\n      <td>204.500000</td>\n      <td>204.500000</td>\n      <td>NaN</td>\n      <td>1000.000000</td>\n      <td>30.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>30.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>75%</th>\n      <td>298.750000</td>\n      <td>298.750000</td>\n      <td>NaN</td>\n      <td>1000.000000</td>\n      <td>30.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>35.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n    <tr>\n      <th>max</th>\n      <td>399.000000</td>\n      <td>399.000000</td>\n      <td>NaN</td>\n      <td>1000.000000</td>\n      <td>30.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n      <td>51.000000</td>\n      <td>NaN</td>\n      <td>NaN</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "        Unnamed: 0  Unnamed: 0.1 loan_status    Principal       terms  \\\ncount   346.000000    346.000000         346   346.000000  346.000000   \nunique         NaN           NaN           2          NaN         NaN   \ntop            NaN           NaN     PAIDOFF          NaN         NaN   \nfreq           NaN           NaN         260          NaN         NaN   \nmean    202.167630    202.167630         NaN   943.641618   22.653179   \nstd     115.459715    115.459715         NaN   109.425530    7.991006   \nmin       0.000000      0.000000         NaN   300.000000    7.000000   \n25%     107.250000    107.250000         NaN   900.000000   15.000000   \n50%     204.500000    204.500000         NaN  1000.000000   30.000000   \n75%     298.750000    298.750000         NaN  1000.000000   30.000000   \nmax     399.000000    399.000000         NaN  1000.000000   30.000000   \n\n       effective_date    due_date         age             education Gender  \ncount             346         346  346.000000                   346    346  \nunique              7          23         NaN                     4      2  \ntop         9/11/2016  10/10/2016         NaN  High School or Below   male  \nfreq              166          90         NaN                   151    294  \nmean              NaN         NaN   30.939306                   NaN    NaN  \nstd               NaN         NaN    6.039418                   NaN    NaN  \nmin               NaN         NaN   18.000000                   NaN    NaN  \n25%               NaN         NaN   27.000000                   NaN    NaN  \n50%               NaN         NaN   30.000000                   NaN    NaN  \n75%               NaN         NaN   35.000000                   NaN    NaN  \nmax               NaN         NaN   51.000000                   NaN    NaN  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "print(df.shape)\nprint(df.dtypes)\ndf.describe(include='all')"
        }, 
        {
            "source": "### Convert to date time object ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 113, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 113, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        }, 
        {
            "source": "# Data visualization and pre-processing\n\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Let\u2019s see how many of each class is in our data set ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 114, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 114, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['loan_status'].value_counts()"
        }, 
        {
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets plot some columns to underestand data better:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 115, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "# notice: installing seaborn might takes a few minutes\n#!conda install -c anaconda seaborn -y"
        }, 
        {
            "execution_count": 116, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7fa70ddd01d0>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        }, 
        {
            "execution_count": 117, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7fa70dda05c0>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        }, 
        {
            "source": "# Pre-processing:  Feature selection/extraction", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "### Lets look at the day of the week people get the loan ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 118, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7fa70f404fd0>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        }, 
        {
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 119, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 119, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        }, 
        {
            "source": "## Convert Categorical features to numerical values", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets look at gender:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 120, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 120, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets convert male to 0 and female to 1:\n", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 121, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 121, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        }, 
        {
            "source": "## One Hot Encoding  \n#### How about education?", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 122, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 122, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        }, 
        {
            "source": "#### Feature befor One Hot Encoding", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 123, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 123, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        }, 
        {
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 124, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 124, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        }, 
        {
            "source": "### Feature selection", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Lets defind feature sets, X:", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 125, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 125, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X = Feature\nX[0:5]"
        }, 
        {
            "source": "What are our lables?", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 126, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 126, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'], dtype=object)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "y = df['loan_status'].values\ny[0:5]"
        }, 
        {
            "source": "## Normalize Data ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 127, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 127, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        }, 
        {
            "source": "# Classification ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells.", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__.", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 128, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.model_selection import train_test_split\nfrom sklearn.neighbors import KNeighborsClassifier\nfrom sklearn import metrics\n\nX_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=4)"
        }, 
        {
            "execution_count": 129, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "[ 0.67142857  0.65714286  0.71428571  0.68571429  0.75714286  0.71428571\n  0.78571429  0.75714286  0.75714286]\n"
                }, 
                {
                    "output_type": "display_data", 
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEKCAYAAADjDHn2AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvpW3flQAAIABJREFUeJzt3Xl8VPW9//HXJ3sCISEQ9kDY90UMKKKCOyBgFzfqBrZqW+1ia3vtbX9uvd4uttVba1tXcKdqtSaIIm6gKJqwDPtOgDCBBEKAELJ/fn/MoGNIyITMyZkkn+fjMQ+ZM2d5B0k+Od/zXURVMcYYY04lwu0Axhhjwp8VC2OMMQ2yYmGMMaZBViyMMcY0yIqFMcaYBlmxMMYY0yArFsYYYxpkxcIYY0yDrFgYY4xpUJTbAUKlc+fOmp6e7nYMY4xpUVasWHFAVVMb2q/VFIv09HRycnLcjmGMMS2KiOwKZj9rhjLGGNMgKxbGGGMaZMXCGGNMg6xYGGOMaZAVC2OMMQ2yYmGMMaZBViyMMcY0yIqFMSasZOcWsTbvsNsxTC1WLIwxYaOiqoZbnsvh6sc/s4IRZqxYGGPCxifbCikurUQEbn42m7xDpW5HMn5WLIwxYSNztZek+Ghe/f4EyiqruXleNkfKKt2OZbBiYYwJE8crqlm8YT9TR3RjeI8kHr/+THYUHuMHL6ygoqrG7XhtnhULY0xY+GBTAccqqpk5ugcA5wzozO+/PYpl2w7y6zfWoqouJ2zbHC0WIjJFRDaLyDYRubuOzx8WkdX+1xYRKQ747I8isl5ENorIX0VEnMxqjHFXpmcvXRJjOatfpy+3XXlmL35y0UBeXZHH3z7Y5mI649gU5SISCTwGXALkAdkikqmqG07so6p3Buz/I+AM/5/PASYCo/wffwJMAj5yKq8xxj1Hyir5cHMh153Vm8iIr/9e+NOLB7KnqJQ/L95CWkoC3zijp0sp2zYn7yzGA9tUdYeqVgDzgStOsf8s4GX/nxWIA2KAWCAa2O9gVmOMi95dv5+Kqhpm+JugAokIv//2KM7ul8IvX1vD8h0HXUhonCwWPYE9Ae/z/NtOIiJ9gL7ABwCq+hnwIZDvfy1S1Y0OZjXGuCjT4yUtJZ4z0pLr/DwmKoLHr88gLSWe255fwbaCkmZOaJwsFnU9Y6jvCdW1wGuqWg0gIgOAoUAvfAXmQhE5/6QLiNwqIjkiklNYWBii2MaY5nSwpJxl2w4wY1QPTvVoMikhmnlzxhMdKcyZ9wUHSsqbMaVxsljkAWkB73sB3nr2vZavmqAAvgksV9USVS0B3gbOrn2Qqj6hqhmqmpGa2uASssaYMLRw3T6qa7TOJqja0lISeOqmcRQeLed7z+ZQVlndDAkNOFsssoGBItJXRGLwFYTM2juJyGCgI/BZwObdwCQRiRKRaHwPt60ZyphWKGu1l4Fd2jOkW2JQ+49JS+aRa87Ak1fMT+evpqbGutQ2B8eKhapWAXcAi/D9oH9FVdeLyAMiMjNg11nAfP16J+rXgO3AWsADeFQ1y6msxhh35B8+zhe5RcwcfeomqNqmjOjGby4fxjvr9/G7t+33yObgWNdZAFVdCCyste2eWu/vq+O4auA2J7MZY9y3wJMPEFQTVG03T0xn98FjPPnxTnqnJHDDhPQQpzOBHC0WxhhzKpkeL6N6JZHeuV2jjxUR7pkxnL3Fx7k3cz09O8Zz4ZCuDqQ0YNN9GGNcsvPAMdbuPfzl9B6nIzJC+OusMxjeI4k7XlrFur02rblTrFgYY1yR5fEiApeP6t6k8yTERPH0TRl0TIjh5nnZeIuPhyihCWTFwhjT7FSVTI+XcekpdE+Kb/L5unSIY+6ccRyvqGbOXJvW3AlWLIwxzW5j/lG2FZQ0qQmqtkFdE/nH9WeyvbCE219cSWW1TWseSlYsjDHNLmuNl8gIYeqIbiE977kDO/O/3xrJx1sP8Js31tm05iFkvaGMMc1KVcnyeDl3QGc6tY8N+fmvzkhjT1Epj36wjd6dErj9ggEhv0ZbZHcWxphmtXJ3MXmHjoe0Caq2n10yiCvG9OChRZt5c/Vex67TltidhTGmWWV5vMRERXDpcOfGRIgIf7xyFPmHy/jFq2vonhTP+L4pjl2vLbA7C2NMs6muUd5am8+Fg7uQGBft6LVioyJ54oYz6ZUSz63P57Cj0KY1bworFsaYZrN8x0EKj5Yzc4xzTVCBkhNimDd7PJEizJmXzUGb1vy0WbEwxjSbLI+XdjGRXDikS7Nds3enBJ68KYN9h8u45Tmb1vx0WbEwxjSLiqoa3l63j0uHdyMuOrJZrz22d0ceuWYMq/YU87NXbFrz02HFwhjTLJZuKeTw8UpHe0GdytSR3fn1tKEsXLuPPyza5EqGlsx6QxljmkXWGi/JCdFMHNDZtQzfPbcvuw6W8viSHaR1TOD6s/u4lqWlsWJhjHHc8YpqFm/YzxVjehIT5V6Dhohw74xh7C0+zj1vrqNncjwXNOPzk5bMmqGMMY57b+N+SiuqXWuCChQVGcGjs85gaPcO3PHSStZ7bVrzYFixMMY4LtPjpUtibNgMjGsXG8Uzs8eRFB/NzfOyyT9s05o3xIqFMcZRh49XsmRzIdNH9SAyIvh1tp3WtUMcz8wZx7Fy37TmR21a81OyYmGMcdSi9fuoqK5ptoF4jTGkWwf+cf1YthWUcPtLq2xa81OwYmGMcVSWx0vvlARG90pyO0qdzhuYyoPfHMHSLYXc86ZNa14fKxbGGMccKCln2bYDzBjdHZHwaYKq7Zpxvbn9gv68/MUe/rlkh9txwpKjxUJEpojIZhHZJiJ31/H5wyKy2v/aIiLFAZ/1FpF3RWSjiGwQkXQnsxpjQm/h2nxqFGaO7ul2lAb9/JLBzBzdgz+8s4ksj9ftOGHHsXEWIhIJPAZcAuQB2SKSqaobTuyjqncG7P8j4IyAUzwHPKiqi0WkPWCNica0MJmrvQzumsjgboluR2lQRITw0FWj2He4jJ+/6qF7UhwZ6eHReyscOHlnMR7Ypqo7VLUCmA9ccYr9ZwEvA4jIMCBKVRcDqGqJqpY6mNUYE2J7i4+Ts+sQM0Z3dztK0GKjInn8hjPplRzPLc/lsPPAMbcjhQ0ni0VPYE/A+zz/tpOISB+gL/CBf9MgoFhEXheRVSLykP9OxRjTQizwN+XMCIOBeI3RsV0Mc+eMQ0SYM/cLio5VuB0pLDhZLOp6mlVfN4NrgddU9cTcwVHAecBdwDigHzD7pAuI3CoiOSKSU1hY2PTExpiQyfR4GZ2WTJ9O7dyO0mh9OrXjyRsz8Nq05l9ysljkAWkB73sB9T01uhZ/E1TAsav8TVhVwH+AsbUPUtUnVDVDVTNSU1NDFNsY01TbC0tY7z3CjFEtpwmqtjP7+KY1X7HrED9/1dPmpzV3slhkAwNFpK+IxOArCJm1dxKRwUBH4LNax3YUkRMV4EJgQ+1jjTHhKcvjRaTlNUHVNm1kd341dQhvrcnnoXc3ux3HVY71hlLVKhG5A1gERALPqOp6EXkAyFHVE4VjFjBfA0bCqGq1iNwFvC++ztkrgCedymqMk15fmUe/1PaMSUt2O0qzUFUyPV7O6ptC1w5xbsdpslvP78fuolL+8dF28ouP0yHe2bXDT0evjvHcen5/R6/h6BTlqroQWFhr2z213t9Xz7GLgVGOhTOmGRwsKecXr62hW4c43v/5pGZfIc4N671H2FF4jO+e29ftKCEhItw/czilFdV8tLnA7Th1GtEzqWUXC2PaurfX7aO6RtlbfJynPt7BHRcOdDuS47LWeImKEKaNaLnPK2qLiozg4WvGuB3DVTbdhzEOyvR4GdClPZcN78pjH25n3+EytyM5qqZGWeDJ57yBnenYLsbtOCaErFgY45D8w8fJzi1i5uge/HraMKpV+cM7rXvt55W7D7G3+HiLf7BtTmbFwhiHLPDko+rrEdS7UwK3nNeXN1btZcWuQ25Hc0yWx0tsVASXDu/mdhQTYlYsjHFI1hovI3sm0bezb1DaDycPoEtiLA9krW+Vffarqmt4a20+Fw3tQvtYexza2lixMMYBOw8cY03e4a+tOd0uNor/mjIET95h3li118V0zvhsx0EOlFQwY5Q1QbVGViyMccCJKa6n15pE75tn9GR0WjJ/eGcTJeVVbkRzTJbHS/vYKC4Y0sXtKMYBViyMCbETg9LGp6fQPSn+a59FRAj3zhhGwdFy/v7hNpcShl55VTVvr9vHpcO7tomxJG2RFQtjQmzTvqNsKyhhRj1rTo/t3ZFvndGTpz7eye6DrWPm/SWbCzlaVmW9oFoxKxbGhFimx0tkhDBtRP09gn45ZQhRkcKDC1vHlGdZa/LpmBDNuQM6ux3FOMSKhTEhpKpkebxMHNCZTu1j692vW1IcP5zcn0Xr9/PptgPNmDD0SiuqeG/DfqaN7E50pP1Iaa3s/6wxIbRqTzF5h45/rRdUfb53Xj96dYzngQUbqKpuuasGL96wn+OV1dYE1cpZsTAmhDJXe4mJiuCy4V0b3DcuOpJfTxvKpn1HeTl7T4P7h6ssTz7dOsQx3tarbtWsWBgTItU1yltr87lwcBcS44KbxnrKiG6c3S+Fv7y7meLSlrd85+HSSpZsKWD6qO5ERNS1OKZpLaxYGBMin+84SOHR8kY1x4gI90wfzuHjlTzy3lYH0znjnfX5VFarNUG1AVYsjAmRTI+XdjGRXDS0cYPShvXowLXje/P88l1s3X/UoXTOyPR46dMpgVG9ktyOYhxmxcKYEKioqvEPSut2WoPSfn7JIBJiInlgwQYCFo0MawVHy/hs+0Fmju6Bb0FL05pZsTAmBD7eWsjh45XMGH16C/50ah/LTy8exMdbD/DBpvBcja22hWvyqVGC6vllWj4rFsaEQKbHS3JCNOcOSD3tc9w4oQ/9U9vx2wUbqKgK/660mR4vQ7olMrBrottRTDOwYmFMEx2vqGbxhv1MHdGdmKjT/5aKjozg/00fRu7BUuZ9ujOECUNvT1EpK3cX24PtNsSKhTFN9P6m/ZRWVJ92E1SgyYO7cOGQLjz6/jYKj5aHIJ0zFqzJB6wJqi2xYmFME2Wu9tIlMZaz+nYKyfl+fflQjldW8+d3N4fkfE7I9HgZk5ZMWkqC21FMM3G0WIjIFBHZLCLbROTuOj5/WERW+19bRKS41ucdRGSviPzNyZzGnK7Dxyv5aHMh00f1IDJEg9L6p7Zn9jnp/CtnD+v2Hg7JOUNpW8FRNuYfsbuKNsaxYiEikcBjwFRgGDBLRIYF7qOqd6rqGFUdAzwKvF7rNL8FljiV0Zimenf9Piqqa0LSBBXoRxcNJCUhhvuz1oddV9pMTz4iMH1UaL9mE96cvLMYD2xT1R2qWgHMB644xf6zgJdPvBGRM4GuwLsOZjSmSTI9XnqnJDAmLTmk502Kj+auywaTnXvoy+cD4eDErLpn9+1Elw5xbscxzcjJYtETCJwdLc+/7SQi0gfoC3zgfx8B/Bn4hYP5jGmSAyXlfLr9IDNGd3dkUNrVGWkM696B3y3cyPGK6pCf/3Ss9x5h54FjzKxnYSfTejlZLOr67qnvfvpa4DVVPfEd8UNgoaqecipOEblVRHJEJKewsLAJUY1pvLfX5lNdo8wcXefvQE0W6V+C1Xu4jCeW7nDkGo2V6fESFSFMPcXCTqZ1crJY5AFpAe97Ad569r2WgCYoYAJwh4jkAn8CbhSR39c+SFWfUNUMVc1ITT39wVDGnI5Mj5dBXdszuJtzg9LO6teJy0d25x9LtuEtPu7YdYJRU+Nrgjp/UCrJCTGuZjHNz8likQ0MFJG+IhKDryBk1t5JRAYDHYHPTmxT1etUtbeqpgN3Ac+p6km9qYxxy97i42TnHmqWHkF3Tx2CKvz+7U2OX+tUcnYdIv9wmfWCaqMcKxaqWgXcASwCNgKvqOp6EXlARGYG7DoLmK/h1uXDmFNY4PHdJDfHCOa0lARuO78fmR4v2blFjl+vPlkeL3HREVwyrOGFnUzr4+g4C1VdqKqDVLW/qj7o33aPqmYG7HPfqe4aVHWeqt7hZE5jGitrjZfRvZLo06lds1zv+5P7061DHA9kbaCmpvl/r6qqrmHh2nwuGtKVdrFRzX594z4bwW1MI+0oLGHd3iPNOi9SQkwUv5o2hLV7D/Payrxmu+4Jy7Yf5OCxCpsLqg2zYmFMI2V6vP5Bac37g3Pm6B6M7Z3MH9/ZzNGyyma9dpbHS2JsFJMHW0eStqrBYiEid4hIx+YIY0y4OzEobXx6Ct2SmndQmohw74zhHCgp528fbmu265ZVVrOoCQs7mdYhmDuLbkC2iLzin+vJlsQybdaG/CNsL3RvUNrotGSuPLMXz3yyk50HjjXLNT/aXMjR8iobiNfGNVgsVPU3wEDgaWA2sFVE/ldE+juczZiw89WgNPfmRfrlZYOJiYzgwbc2Nsv1stZ4SWkXw8T+oZlV17RMQT2z8Hdr3ed/VeEbF/GaiPzRwWzGhBVVZYEnn3MHdialnXuD0rp0iOOOCwfy3sb9fLzV2ZkLjpVX8f7G/Uwb2Y2oSHvE2ZYF88zixyKyAvgjsAwYqao/AM4Evu1wPmPCxsrdh9hbfDwsBqXdfG46fTol8EDWBqqqnVuCdfGG/ZRV1jg2pYlpOYL5VaEz8C1VvUxVX1XVSgBVrQGmO5rOmDCSudpLbFR4DEqLjYrkv6cNZWtBCS9+vtux62R5vHRPiiOjj/VxaeuCKRYLgS+HjYpIooicBaCqzdNoaozLqqpreGttPhcO6UJiXLTbcQC4dFhXJg7oxF8Wb+HQsYqQn7+4tIKlWwuZPqo7ESFa2Mm0XMEUi38AJQHvj/m3GdNmLN9RxIGSirBogjpBRLhn+nCOllXy8HtbQn7+t9fto7LauVl1TcsSTLGQwHmb/M1PNt7ftCmZnr20j43igiFd3I7yNYO7JXL92X148fPdbN53NKTnzvJ46du5HSN6dgjpeU3LFEyx2OF/yB3tf/0ECI/J9Y1pBuVV1by9bh+XDu8aloPS7rx4EO1jo3hgQeiWYC04UsZnOw4yY3QPRxZ2Mi1PMMXi+8A5wF58a1ScBdzqZChjwsnSLQc4WlYVtvMidWwXw88uGcSybQdZvGF/SM65YE0+qjAzxGuLm5YrmEF5Bap6rap2UdWuqvodVS1ojnDGhINMj5eOCdGcO6Cz21Hqdd1ZvRnYpT3/89ZGyquavgRrpsfL0O4dGNDFuYWdTMsSzDiLOBG5XUT+LiLPnHg1Rzhj3FZaUcV7G/YzbWR3osN4UFpUZAT3zBjG7qJSnvkkt0nn2lNUyuo9xWH1MN+4L5h//c/jmx/qMmAJvuVRQ/skzZgw9d7GAo5XVodtE1Sg8wamcvHQrvztg60UHCk77fNk+hd2mj7KmqDMV4IpFgNU9f8Bx1T1WeByYKSzsYwJD5mrvXTrEMf49BS3owTlN5cPpaK6hocWbT7tc2R5vIztnUxaSkIIk5mWLphicWLi/GIRGQEkAemOJTImTBwurWTJloIWNSgtvXM7bj63L6+uyMOzp7jRx2/Zf5RN+45aE5Q5STDF4gn/eha/ATKBDcAfHE1lTBhYtN43KK0lNEEFuuOCAXRuH8v9WY3vSpvl8RIhMM2aoEwtpywWIhIBHFHVQ6q6VFX7+XtFPd5M+YxxTabHS59OCYzqleR2lEZJjIvml5cNZuXu4i+fPwRDVcn0eJnQvxNdEpt3YScT/k5ZLPyjte9opizGhI2Co2V8uv0AM1vooLQrz+zFyJ5J/G7hJkorqoI6Zu3ew+w6WGpNUKZOwTRDLRaRu0QkTURSTrwcT2aMi95eu48apcU1QZ0QESHcO2MY+46U8c8lwU24kLnaS3SkMGW4NUGZkwVTLG4GbgeWAiv8r5xgTu5fhnWziGwTkbvr+PxhEVntf20RkWL/9jEi8pmIrBeRNSJyTfBfkjFNl+nxMqRbIoO6ttxBaRnpKcwc3YPHl2wn71DpKfetqVEWrMln0qBUkhLCY1ZdE16CGcHdt45Xv4aOE5FI4DFgKjAMmCUiw2qd+05VHaOqY4BHgdf9H5UCN6rqcGAK8IiIJDfuSzPm9OQdKmXFrkMt9q4i0N1ThyACv3t70yn3y84tYt+RslbxNRtnNDh7rIjcWNd2VX2ugUPHA9tUdYf/PPOBK/D1pqrLLOBe/7m/nG9ZVb0iUgCkAo3vC2hMI2V58gGYMarl/+DskRzP9yf155H3tnLj2Qc5q1/d62hnerzERUdw8VD3F3Yy4SmYZqhxAa/zgPuAmUEc1xPYE/A+z7/tJCLSB+gLfFDHZ+OBGGB7HZ/dKiI5IpJTWOjsWsSm7cjyeBmTlkzvTq1jUNpt5/enR1Ic92dtoLrm5K60ldU1LFybz8VDu9Iu1lYfMHULphnqRwGvW4Az8P3wbkhdXUjq6/R9LfCaqn5tBjQR6Y5vupE5/p5ZtbM9oaoZqpqRmpoaRCRjTm1bQQkb8o+0qh5B8TGR/GraUDbkH+HVnD0nfb5s2wEOlVa2qq/ZhN7pzIxWCgwMYr88IC3gfS+gvk7f1wIvB24QkQ7AW8BvVHX5aeQ0ptEyPV5EWt+8SNNHdWdcekceWrSZI2WVX/ss0+MlMS6KSYPtFy5Tv2Bmnc0SkUz/awGwGXgziHNnAwNFpK+IxOArCJl1nH8w0BH4LGBbDPAG8Jyqvhrcl2JM06gqCzxezu7biS4dWtegNBHh3hnDKSqt4NH3t365vayymnfX72fK8G7ERoXfwk4mfATTQPmngD9XAbtUNa+hg1S1SkTuABYBkcAzqrpeRB4AclT1ROGYBcwPXLoVuBo4H+gkIrP922ar6uog8hpzWtZ7j7DjwDFuOb/Bzn4t0oieSVyTkcbcZblcO743/VPb8+GmAkrKq5g5xpqgzKkFUyx2A/mqWgYgIvEikq6quQ0dqKoLgYW1tt1T6/19dRz3AvBCENmMCZlMj5eoCGHqiG5uR3HMzy8dzII1+Tz41kaemT2OrDVeOrePYUI9vaSMOSGYZxavAoEPl6v924xpNWpqfE1Q5w9KJTkhmP4bLVNqYiw/vmgAH2wq4K01+by/sYBpI7sTFcYLO5nwEMy/kChVrTjxxv/n1vvdZNqkFbsP4T1c1iZ6BM0+py99O7fjzldWU15V0ya+ZtN0wRSLQhH5clyFiFwBHHAukglnqsr/vbeVz3ccdDtKSGWu9g1Ku2RY6x+UFhMV4VskqaqGHklxjO3d0e1IpgUI5pnF94EXReRv/vd5QJ2juk3rt3xHEQ+/t4V2MZG8+v1zGNajg9uRmqzKPyjtoiFtZ1DahUO6MPucdIZ179BiFnYy7mrwO0NVtwNni0h7QFTV1t9uw+Yu20nHhGjioiO5eV42/7l9It2SWnY300+3H+TgsYo2NS+SiHDfzOFuxzAtSDDjLP5XRJJVtURVj4pIRxH5n+YIZ8LLnqJSFm/cz3fO6s0zs8dRUl7FnHnZlJQHt15CuMr0eEmMjWKyDUozpl7BPLOYqqpfTuCnqoeAac5FMuHq2U9ziRThhrPTGdq9A49dN5Yt+49y+4srqao+aTaWFqG8qppF6/Zx6fBuxEXboDRj6hNMsYgUkdgTb0QkHog9xf6mFTpWXsW/cvYwdWT3L5udJg1K5X++MYIlWwq5J7Px6z2Hg482F3LUBqUZ06Bgnua9ALwvInP97+cAzzoXyYSjf6/M42hZFXMmpn9t+6zxvdlTVMrfP9pOn5QEbpvU352ApynT46VTuxgm9rdBacacSjAPuP8oImuAi/HNJPsO0MfpYCZ81NQo85blMjotuc5ulnddOpjdRaX87u1N9OqYwOUtZBK+Y+VVvL9xP1edmWaD0oxpQLDfIfvwjeL+NnARsNGxRCbsLN1ayI4Dx5hzTnqdn0dECH+6ajQZfTpy5yurWbGrqHkDnqb3Nu6nrLKmTfWCMuZ01VssRGSQiNwjIhuBv+FbyEhU9QJV/Vt9x5nWZ+6yXLokxjJtZP13DHHRkTxxYwY9k+P53rM55B441owJT0/mai/dk+LI6GOD0oxpyKnuLDbhu4uYoarnquqj+OaFMm3ItoISlmwp5Pqz+xATdeob0ZR2McydPQ6AOfOyOXSs4pT7u6m4tIKlWwuZMbqHDUozJgin+u7/Nr7mpw9F5EkRuYi6V78zrdizn+YSExnBd87qHdT+6Z3b8dRNGewtPs6tz+dQVhmev1+8s24fldXaKtbZNqY51FssVPUNVb0GGAJ8BNwJdBWRf4jIpc2Uz7jo8PFK/r0yj5ljetC5ffC9pc/sk8Jfrh5Ndu4hfvHaGmrqWPfZbZkeL307t2NEz5Y/XYkxzSGYNbiPqeqLqjod39Koq4G7HU9mXPdK9h5KK6pP6i4bjOmjenD31CFkebz8efHm0IdrgoIjZXy24yAzRvdAxG6WjQlGo2ZNU9Ui4HH/y7Ri1TXKs5/lMr5vCsN7JJ3WOW47vx+7Dpby2IfbSeuYwLXjg2vKctpba/NRhZmjW0YXX2PCgXUuN3V6b+N+8g4dr7e7bDBEhN9eMZxJg1L59X/WsXRLYegCNkGmx8vQ7h0Y0CXR7SjGtBhWLEyd5i7bSc/k+Cav7xAVGcFj141lUNdEfvjiSjbmHwlRwtOzp6iUVbuLbcEfYxrJioU5ycb8IyzfUcSNE/qEZGRz+9gonpmdQfvYKG6el83+I2UhSHl6stZ4AZjeQkaZGxMurFiYk8xdtpP46EiuHRe6Zwzdk+J5ZvY4jhyvZM5c96Y1z1ztZWzvZNJSEly5vjEtlaPFQkSmiMhmEdkmIif1oBKRh0Vktf+1RUSKAz67SUS2+l83OZnTfOVgSTn/We3lW2N7kpQQHdJzD+vRgb9dN5bN+4/yo5eaf1rzrfuPsmnfUWuCMuY0OFYsRCQSeAyYCgwDZonIsMB9VPVOVR2jqmOAR4HX/cemAPcCZwHjgXtFxOZkaAYvf7Gbiqqa0+ouG4wLBnfht1eM4MPNhdyX1bzTmmd6vEQITLMmKGMazck7i/HANlXdoaoVwHzgilPsPwt42f/ny4DFqlrkX2xpMTDFwawGqKyu4fnluzhvYGdHewp956ze3DapHy8s381/gxJUAAAWGElEQVRTH+907DqBVJUsj5cJ/TvRJbFlLwNrjBucLBY98U0+eEKef9tJRKQP0Bf4oLHHmtB5e90+9h8pd+yuItB/XTaEy0d258GFG1m4Nt/x663de5jcg6XWBGXMaXKyWNQ1NLa+NodrgddU9cREQkEdKyK3ikiOiOQUFoZHH/6WbO6ynfTt3I7Jg7o4fq2ICOHPV4/mzD4dufNfq1m5+5Cj18tc7SU6Upgy3JqgjDkdThaLPCAt4H0vwFvPvtfyVRNU0Meq6hOqmqGqGampqU2M27at3lPMqt3F3DShT7PNwhoXHcmTN2bQPSmOW57NYddBZ6Y1r6lRFqzJZ9Kg1JA/tDemrXCyWGQDA0Wkr4jE4CsImbV3EpHBQEfgs4DNi4BLRaSj/8H2pf5txiFzl+0kMTaKKzPSGt45hFLaxTB3zniqVZkzN5vi0tBPa56dW8S+I2W2yJExTeBYsVDVKuAOfD/kNwKvqOp6EXlARGYG7DoLmK8B3WL8c1D9Fl/ByQYe8G8zDth/pIy31uRzVUYa7WMbNV1YSPTt3I4nb8wg79Bxbn1uBeVVoZ3WPNPjJT46ssmj0Y1pyxz9yaCqC4GFtbbdU+v9ffUc+wzwjGPhzJdeWL6LalVmN2EeqKYal57Cn64ezY9fXsUvX1vDI9eMCcmMsJXVNby9bh8XDe1CQkzzF0JjWgv77mnjyiqreenz3Vw0pCu9O7k7qnnm6B7sKSrloUWb6Z2SwM8vHdzkcy7bdoCiYxXWC8qYJrJi0cZlebwcPFbRLN1lg/HDyf3ZU1TKox9sI61jAlePa9ozlEyPl8S4KCYNtg4QxjSFFYs2TFWZuyyXwV0TOad/J7fjAP5pzb8xgr3Fx/nvN9bSIzmecwd2Pq1zlVVW8+76/Uwd0Y3YqMgQJzWmbbGJBNuwL3YWsSH/CLMnpofVinHRkRH8/bqxDOjSnh+8sILN+46e1nk+2lxASXkVM8dYE5QxTWXFog2buyyX5IRovjEm/AbHJ8ZF88zscSTERjJn7henNa15psdL5/YxTOgXHndNxrRkVizaqD1Fpby7YR+zxvcmPiY8m2h6JMfz9E3jKD5eyXefzeZYI6Y1P1pWyfsbC5g2sntI1uQwpq2z76I26vnluxARbji7j9tRTmlEzyQe+85YNniP8OOXV1FdE9wste9t3E95VY31gjImRKxYtEGlFVXM/2I3U4Z3o0dyvNtxGnTBkC7cf8UI3t9UwP1BTmueudpLz+R4xva2me2NCQXrDdUGvb5yL0fKqsKmu2wwbji7D3uKSnli6Q56pyTwvfP61bvvoWMVfLz1AN89r2+zzXNlTGtnxaKNUVXmfZrLyJ5JnNmnZf3WffeUIewpKuXBhRvp1TGeKSPqnkH27XX7qKpRZoyyJihjQsWaodqYj7ceYFtBCXPCrLtsMCIihIevGcOYtGR+Mn81q+qZ1jzTs5d+qe0Y3qNDMyc0pvWyYtHGzF22k87tY7m8hS4tGhcdyVM3ZtC1QxzfezaH3QdLv/b5vsNlfL6ziJmje7S4YmhMOLNi0YbsKCzhw82FXH927xY9orlT+1jmzhlHVY0ye94XX5vW/K21+ahi05EbE2JWLNqQZz/NJTpSuO6s8O4uG4z+qe154oYzySs6zm3PfzWteabHy/AeHeif2t7lhMa0LlYs2ogjZZW8tiKPGaN6kJoY63ackDirXyceumoUn+8s4u5/ryX3wDE8e4ptbIUxDrDeUG3EK9l7OFZRzZyJfd2OElJXjOnJnqJS/vTuFtbkFQMw3YqFMSFnxaINqK5RnvtsFxl9OjKyV5LbcULu9gsGsLuolFdy8sjo05GeLWCgoTEtjRWLNuCDTQXsLirlv6YMcTuKI0SEB785kqT4aC4eakunGuMEKxZtwNxlO+mRFMdlw1vvD9LoyAh+ffkwt2MY02rZA+5WbtO+I3y6/SA3TEi32VeNMafNfnq0cvOW5RIXHcGs8U1bntQY07ZZsWjFio5V8MaqvXzzjJ4kJ8S4HccY04I5WixEZIqIbBaRbSJydz37XC0iG0RkvYi8FLD9j/5tG0Xkr2JzNzTay1/spryqhtnntK7ussaY5ufYA24RiQQeAy4B8oBsEclU1Q0B+wwEfgVMVNVDItLFv/0cYCIwyr/rJ8Ak4COn8rY2ldU1vLB8FxMHdGJwt0S34xhjWjgn7yzGA9tUdYeqVgDzgStq7XML8JiqHgJQ1QL/dgXigBggFogG9juYtdVZtH4f+YfLmGN3FcaYEHCyWPQE9gS8z/NvCzQIGCQiy0RkuYhMAVDVz4APgXz/a5GqbnQwa6szd1kufTolcOGQLm5HMca0Ak4Wi7qeMdReDzMKGAhMBmYBT4lIsogMAIYCvfAVmAtF5PyTLiByq4jkiEhOYWFhSMO3ZGvyilmx6xA3TUi3leKMMSHhZLHIAwL7a/YCvHXs86aqVqrqTmAzvuLxTWC5qpaoagnwNnB27Quo6hOqmqGqGampqY58ES3R3GW5tIuJ5KqMXm5HMca0Ek4Wi2xgoIj0FZEY4Fogs9Y+/wEuABCRzviapXYAu4FJIhIlItH4Hm5bM1QQCo6UsWCNl6sy0kiMi3Y7jjGmlXCsWKhqFXAHsAjfD/pXVHW9iDwgIjP9uy0CDorIBnzPKH6hqgeB14DtwFrAA3hUNcuprK3JC5/vpqpGuemcdLejGGNaEUfnhlLVhcDCWtvuCfizAj/zvwL3qQZuczJba1ReVc1Ln+/igsFd6Nu5ndtxjDGtiI3gbkUWePI5UFLBnInpbkcxxrQyVixaCVVl7qc7GdilPecO6Ox2HGNMK2PFopXI2XWIdXuPMHtiOjYzijEm1KxYtBJzl+0kKT6ab55Re9yjMcY0nRWLVmBv8XEWrd/PtePSSIix9ayMMaFnxaIVeO6zXFSVGyb0cTuKMaaVsmIB1NTUnoWk5TheUc38L/Zw2fBu9OqY4HYcY0wr1eaLxbHyKqb831Ke+WQnldU1bsdptDdW7eXw8UrmTLTZZY0xzmnzxeJoWRVdO8TxwIINTHlkKR9tLmj4oDChqsz7dCfDe3RgXHpHt+MYY1qxNl8suiXF8dzN43n6pgyqa5TZc7O5eV42OwpL3I7WoGXbDrJlfwlzJva17rLGGEe1+WIBICJcNLQr7945if+eNoQvdhZx2SNLefCtDRwpq3Q7Xr3mLttJ5/YxzBjd3e0oxphWzopFgJioCG49vz8f3jWZb4/txVOf7OSChz5i/he7qQ6zh+C5B47xweYCvjO+N7FRkW7HMca0clYs6pCaGMvvvz2KrDvOpV9qO+5+fS0zHv2Ez3ccdDval+Z9mktUhHD92dZd1hjjPCsWpzCiZxKv3DaBR2edQXFpBdc8sZzbX1pJ3qFSV3MdLavktRV5XD6yO106xLmaxRjTNlixaICIMGN0D97/+WTuvHgQ72/cz0V/XsJfFm+htKLKlUyvrcijpLzKussaY5qNFYsgxcdE8pOLB/LBzydz2fBu/PX9rVz4pyW8uXovvmU5mkdNjfLsp7mM7Z3M6LTkZruuMaZts2LRSD2S4/nrrDN49fsTSE2M5SfzV3PlPz9jTV5xs1z/w80F5B4stbsKY0yzsmJxmsalp/Dm7RP545Wj2HWwlJl/W8YvXvVQcLTM0evOXZZLtw5xTBnRzdHrGGNMICsWTRARIVydkcaHd03itkn9+M/qvVzw0Ef846PtlFdVh/x6W/Yf5ZNtB7hhQh+iI+1/nTGm+dhPnBBIjIvmV1OHsvjOSZwzoDN/eGcTlz68lHfX7wvp84y5y3KJjYpg1vjeITunMcYEw4pFCKV3bseTN2bw/HfHExMZwa3Pr+D6pz9n876jTT53cWkFb6zK4xtjepLSLiYEaY0xJnhWLBxw3sBU3v7Jedw/czjr9h5h6v8t5Z4313HoWMVpn3N+9h7KKmuYc2566IIaY0yQHC0WIjJFRDaLyDYRubuefa4WkQ0isl5EXgrY3ltE3hWRjf7P053MGmpRkRHcdE46H901mRvO7sOLn+9m8p8+4tlPc6lq5FToVdU1PPdpLhP6dWJItw4OJTbGmPo5VixEJBJ4DJgKDANmiciwWvsMBH4FTFTV4cBPAz5+DnhIVYcC44GWM3d4gI7tYrj/ihEs/PF5jOjZgXsz1zPtrx/z8dbCoM/x7ob9eA+XMWdiunNBjTHmFJy8sxgPbFPVHapaAcwHrqi1zy3AY6p6CEBVCwD8RSVKVRf7t5eoqrtzbDTR4G6JvPDds3jihjMpq6zhhqe/4HvP5pB74FiDx85dtpO0lHguGtq1GZIaY8zJnCwWPYE9Ae/z/NsCDQIGicgyEVkuIlMCtheLyOsiskpEHvLfqbRoIsKlw7ux+Gfnc/fUIXy2/QCXPLyE3729kaP1TIW+bu9hsnMPcdOEdCIjbM0KY4w7nCwWdf1kq92PNAoYCEwGZgFPiUiyf/t5wF3AOKAfMPukC4jcKiI5IpJTWBh8s47bYqMi+f6k/nz4i8l8Y0xPHl+ygwv+tIRXsvectB74M8t2khATyVUZaS6lNcYYZ4tFHhD4E64X4K1jnzdVtVJVdwKb8RWPPGCVvwmrCvgPMLb2BVT1CVXNUNWM1NRUR74IJ3VJjOOhq0bz5u0T6Z0Szy//vYYrHltGTm4RAIVHy1ngyefKM3uRFB/tclpjTFsW5eC5s4GBItIX2AtcC3yn1j7/wXdHMU9EOuNrftoBFAMdRSRVVQuBC4EcB7O6anRaMv/+wTlkerz8/u1NXPnPz5gxugfJ8dFUVNdw0znpbkc0xrRxjhULVa0SkTuARUAk8IyqrheRB4AcVc30f3apiGwAqoFfqOpBABG5C3hffItLrwCedCprOBARrhjTk0uGdeWfS3bw+JLtlFfVMHlwKv1T27sdzxjTxklzTq/tpIyMDM3JaT03H3mHSpm7LJerM9IY3C3R7TjGmFZKRFaoakZD+znZDGWaoFfHBP7f9GEN72iMMc3ApvswxhjTICsWxhhjGmTFwhhjTIOsWBhjjGmQFQtjjDENsmJhjDGmQVYsjDHGNMiKhTHGmAa1mhHcIlII7GrCKToDB0IUJ5QsV+NYrsaxXI3TGnP1UdUGZ2JtNcWiqUQkJ5gh783NcjWO5Wocy9U4bTmXNUMZY4xpkBULY4wxDbJi8ZUn3A5QD8vVOJarcSxX47TZXPbMwhhjTIPszsIYY0yD2nyxEJFnRKRARNa5neUEEUkTkQ9FZKOIrBeRn7idCUBE4kTkCxHx+HPd73amQCISKSKrRGSB21lOEJFcEVkrIqtFJGxW5xKRZBF5TUQ2+f+dTXA7E4CIDPb/XZ14HRGRn4ZBrjv9/+bXicjLIhLndiYAEfmJP9N6p/+e2nwzlIicD5QAz6nqCLfzAIhId6C7qq4UkUR8y8p+Q1U3uJxLgHaqWiIi0cAnwE9UdbmbuU4QkZ8BGUAHVZ3udh7wFQsgQ1XDqm++iDwLfKyqT4lIDJCgqsVu5wokIpHAXuAsVW3KGKqm5uiJ79/6MFU9LiKvAAtVdZ5bmfy5RgDzgfFABfAO8ANV3erE9dr8nYWqLgWK3M4RSFXzVXWl/89HgY1AT3dTgfqU+N9G+19h8duGiPQCLgeecjtLuBORDsD5wNMAqloRboXC7yJgu5uFIkAUEC8iUUAC4HU5D8BQYLmqlqpqFbAE+KZTF2vzxSLciUg6cAbwubtJfPxNPauBAmCxqoZFLuAR4JdAjdtBalHgXRFZISK3uh3Grx9QCMz1N9s9JSLt3A5Vh2uBl90Ooap7gT8Bu4F84LCqvutuKgDWAeeLSCcRSQCmAWlOXcyKRRgTkfbAv4GfquoRt/MAqGq1qo4BegHj/bfCrhKR6UCBqq5wO0sdJqrqWGAqcLu/2dNtUcBY4B+qegZwDLjb3Uhf528amwm8GgZZOgJXAH2BHkA7Ebne3VSgqhuBPwCL8TVBeYAqp65nxSJM+Z8J/Bt4UVVfdztPbf5mi4+AKS5HAZgIzPQ/H5gPXCgiL7gbyUdVvf7/FgBv4GtfdlsekBdwV/gavuIRTqYCK1V1v9tBgIuBnapaqKqVwOvAOS5nAkBVn1bVsap6Pr7mdEeeV4AVi7Dkf5D8NLBRVf/idp4TRCRVRJL9f47H9020yd1UoKq/UtVeqpqOr+niA1V1/Tc/EWnn76CAv5nnUnxNB65S1X3AHhEZ7N90EeBq54k6zCIMmqD8dgNni0iC/3vzInzPEV0nIl38/+0NfAsH/86inDpxSyEiLwOTgc4ikgfcq6pPu5uKicANwFr/8wGA/1bVhS5mAugOPOvvpRIBvKKqYdNNNQx1Bd7w/XwhCnhJVd9xN9KXfgS86G/u2QHMcTnPl/zt75cAt7mdBUBVPxeR14CV+Jp5VhE+I7n/LSKdgErgdlU95NSF2nzXWWOMMQ2zZihjjDENsmJhjDGmQVYsjDHGNMiKhTHGmAZZsTDGGNMgKxamVRIRFZE/B7y/S0Tua8TxsSLynn/m02uamKWHv+tlQ/uV1LN9nohc2ZQMxjSVFQvTWpUD3xKRzqd5/BlAtKqOUdV/NSWIqnpV1ZUf9v6J74xpMisWprWqwjdw6s5T7SQiKSLyHxFZIyLLRWSUf1TsC8AY/51F/1rHfCQif/Cv7bFFRM7zb48UkYdEJNt/vtv829PFv16KfxTwK/7P/yUin4tIRsC5HxTfeiHLRaRrwGUvFpGP/deb7t83TkTmim+9jFUicoF/+2wReVVEsvBNYthdRJb6v5Z1J/Ia0xhWLExr9hhwnYgknWKf+4FVqjoK+G9865oUAN/Dt97DGFXdXsdxUao6HvgpcK9/23fxzUg6DhgH3CIifWsd90PgkP96vwXODPisHb4pp0cDS4FbAj5LBybhm4b9n+JbfOd2AFUdiW96jGflq0V5JgA3qeqFwHeARf4JIEcDqzGmkaxYmFbLP1Pvc8CPT7HbucDz/v0/ADo1UFxOODG54wp8P8jBN/fTjf4pWj4HOgED67jefP/11gFrAj6rAE5MnxJ4XvBNrVLjX9hmBzCkVvZNwC5gkH//xap6Yp2WbGCO/5nNSP8aKcY0ihUL09o9gu83/vrWa5A6tgUzB065/7/VfDXHmgA/8t+NjFHVvnWse1DX9U6o1K/m3wk8b12ZtIFzHftyR98CX+fjW3XueRG58RTHGVMnKxamVfP/dv0KvoJRl6XAdQAiMhk40IS1QxYBP/BPL4+IDKpjUaFPgKv9nw8DRgZ57qtEJML//KQfsLlW9kFAb//2rxGRPvjW+3gS32zG4TYduWkBrKeEaQv+DNxRz2f34Vsxbg1QCtzUhOs8ha/paKV/KutC4Bu19vk7vmcLa/DNXroGOBzEuTfjWzazK/B9VS0Tkb/je36xFt8D/dmqWu6f5TbQZOAXIlKJb715u7MwjWazzhrTjPzTu0f7f9j3B94HBqlqhcvRjDklu7MwpnklAB/6m6oE+IEVCtMS2J2FMcaYBtkDbmOMMQ2yYmGMMaZBViyMMcY0yIqFMcaYBlmxMMYY0yArFsYYYxr0/wHnLWLCfJsP5gAAAABJRU5ErkJggg==\n", 
                        "text/plain": "<matplotlib.figure.Figure at 0x7fa70bc2bfd0>"
                    }, 
                    "metadata": {}
                }
            ], 
            "source": "acc_list = np.zeros(9)\n\nfor n in range(1, 10):\n    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train, y_train)\n    acc_list[n-1] = metrics.accuracy_score(y_test, neigh.predict(X_test))\n\nprint(acc_list)\n\nplt.plot(range(1,10), acc_list)\nplt.xlabel('N of neighbors')\nplt.ylabel('Accuracy')\nplt.show()"
        }, 
        {
            "execution_count": 130, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 130, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n           metric_params=None, n_jobs=1, n_neighbors=7, p=2,\n           weights='uniform')"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# k=7 give the best accuracy\nK_neighbors_model = KNeighborsClassifier(n_neighbors = 7)\nK_neighbors_model.fit(X_train, y_train)"
        }, 
        {
            "source": "# Decision Tree", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 131, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.tree import DecisionTreeClassifier"
        }, 
        {
            "execution_count": 132, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 132, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=None,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "D_tree_model = DecisionTreeClassifier(criterion='entropy')\nD_tree_model.fit(X_train, y_train)"
        }, 
        {
            "execution_count": 133, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 133, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "0.69999999999999996"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "metrics.accuracy_score(y_test, D_tree_model.predict(X_test))"
        }, 
        {
            "source": "# Support Vector Machine", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 134, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.svm import SVC\nkernels = ['linear', 'poly', 'rbf', 'sigmoid']"
        }, 
        {
            "execution_count": 135, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/ibm/conda/miniconda3/lib/python3.5/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                }, 
                {
                    "execution_count": 135, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Accuracy</th>\n      <th>F1</th>\n      <th>Jaccard</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>linear</th>\n      <td>0.785714</td>\n      <td>0.691429</td>\n      <td>0.785714</td>\n    </tr>\n    <tr>\n      <th>poly</th>\n      <td>0.771429</td>\n      <td>0.706479</td>\n      <td>0.771429</td>\n    </tr>\n    <tr>\n      <th>rbf</th>\n      <td>0.742857</td>\n      <td>0.727588</td>\n      <td>0.742857</td>\n    </tr>\n    <tr>\n      <th>sigmoid</th>\n      <td>0.742857</td>\n      <td>0.689286</td>\n      <td>0.742857</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "         Accuracy        F1   Jaccard\nlinear   0.785714  0.691429  0.785714\npoly     0.771429  0.706479  0.771429\nrbf      0.742857  0.727588  0.742857\nsigmoid  0.742857  0.689286  0.742857"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "acc_list = np.zeros(4)\nf1_list = np.zeros(4)\njacc_list = np.zeros(4)\n\nfor i in range(len(kernels)):\n    svm_ = SVC(kernel=kernels[i]).fit(X_train, y_train)\n    y_hat = svm_.predict(X_test)\n    acc_list[i] = metrics.accuracy_score(y_test, y_hat)\n    f1_list[i] = metrics.f1_score(y_test, y_hat, average='weighted')\n    jacc_list[i] = metrics.jaccard_similarity_score(y_test, y_hat)\n\nsvm_score_matrix = pd.DataFrame()\nsvm_score_matrix['Accuracy'] = acc_list\nsvm_score_matrix['F1'] = f1_list\nsvm_score_matrix['Jaccard'] = jacc_list\nsvm_score_matrix.index = kernels\nsvm_score_matrix"
        }, 
        {
            "execution_count": 136, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 136, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n  decision_function_shape='ovr', degree=3, gamma='auto', kernel='linear',\n  max_iter=-1, probability=False, random_state=None, shrinking=True,\n  tol=0.001, verbose=False)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# linear kernel give the best result\nSVM_model = SVC(kernel='linear')\nSVM_model.fit(X_train, y_train)"
        }, 
        {
            "source": "# Logistic Regression", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 137, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.linear_model import LogisticRegression"
        }, 
        {
            "execution_count": 138, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/ibm/conda/miniconda3/lib/python3.5/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n/opt/ibm/conda/miniconda3/lib/python3.5/site-packages/sklearn/linear_model/sag.py:326: ConvergenceWarning: The max_iter was reached which means the coef_ did not converge\n  \"the coef_ did not converge\", ConvergenceWarning)\n"
                }, 
                {
                    "execution_count": 138, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>newton-cg</th>\n      <th>lbfgs</th>\n      <th>liblinear</th>\n      <th>sag</th>\n      <th>saga</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0.001</th>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518</td>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518</td>\n      <td>Acc: 0.671; F1: 0.675; Jac: 0.671; LogLos: 0.671</td>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518</td>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518</td>\n    </tr>\n    <tr>\n      <th>0.010</th>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492</td>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492</td>\n      <td>Acc: 0.686; F1: 0.667; Jac: 0.686; LogLos: 0.577</td>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492</td>\n      <td>Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492</td>\n    </tr>\n    <tr>\n      <th>0.100</th>\n      <td>Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483</td>\n      <td>Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483</td>\n      <td>Acc: 0.743; F1: 0.705; Jac: 0.743; LogLos: 0.498</td>\n      <td>Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483</td>\n      <td>Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483</td>\n    </tr>\n    <tr>\n      <th>1.000</th>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500</td>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500</td>\n      <td>Acc: 0.714; F1: 0.686; Jac: 0.714; LogLos: 0.500</td>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500</td>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500</td>\n    </tr>\n    <tr>\n      <th>10.000</th>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.519</td>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.519</td>\n      <td>Acc: 0.714; F1: 0.686; Jac: 0.714; LogLos: 0.519</td>\n      <td>Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.518</td>\n      <td>Acc: 0.714; F1: 0.686; Jac: 0.714; LogLos: 0.514</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "                                               newton-cg  \\\n0.001   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518   \n0.010   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492   \n0.100   Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483   \n1.000   Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500   \n10.000  Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.519   \n\n                                                   lbfgs  \\\n0.001   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518   \n0.010   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492   \n0.100   Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483   \n1.000   Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500   \n10.000  Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.519   \n\n                                               liblinear  \\\n0.001   Acc: 0.671; F1: 0.675; Jac: 0.671; LogLos: 0.671   \n0.010   Acc: 0.686; F1: 0.667; Jac: 0.686; LogLos: 0.577   \n0.100   Acc: 0.743; F1: 0.705; Jac: 0.743; LogLos: 0.498   \n1.000   Acc: 0.714; F1: 0.686; Jac: 0.714; LogLos: 0.500   \n10.000  Acc: 0.714; F1: 0.686; Jac: 0.714; LogLos: 0.519   \n\n                                                     sag  \\\n0.001   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518   \n0.010   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492   \n0.100   Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483   \n1.000   Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500   \n10.000  Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.518   \n\n                                                    saga  \n0.001   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.518  \n0.010   Acc: 0.786; F1: 0.691; Jac: 0.786; LogLos: 0.492  \n0.100   Acc: 0.771; F1: 0.706; Jac: 0.771; LogLos: 0.483  \n1.000   Acc: 0.729; F1: 0.695; Jac: 0.729; LogLos: 0.500  \n10.000  Acc: 0.714; F1: 0.686; Jac: 0.714; LogLos: 0.514  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "solvers = ['newton-cg', 'lbfgs', 'liblinear', 'sag', 'saga'];\nregularizations = [0.001, 0.01, 0.1, 1, 10]\nLR_score_matrix = pd.DataFrame()\n\nfor i in range(len(solvers)):\n    solv_col = np.zeros(len(regularizations), dtype=np.object)\n    for j in range(len(regularizations)):\n        logreg = LogisticRegression(C=regularizations[j], solver=solvers[i]).fit(X_train, y_train)\n        y_hat = logreg.predict(X_test)\n        y_hat_prob = logreg.predict_proba(X_test)\n        acc_score = metrics.accuracy_score(y_test, y_hat)\n        f1_score = metrics.f1_score(y_test, y_hat, average='weighted')\n        jacc_score = metrics.jaccard_similarity_score(y_test, y_hat)\n        logloss = metrics.log_loss(y_test, y_hat_prob)\n        solv_col[j] = \"Acc: {:.3f}; F1: {:.3f}; Jac: {:.3f}; LogLos: {:.3f}\".format(acc_score, f1_score, jacc_score, logloss)\n    LR_score_matrix[solvers[i]] = solv_col\nLR_score_matrix.index = regularizations\nLR_score_matrix"
        }, 
        {
            "execution_count": 139, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 139, 
                    "metadata": {}, 
                    "data": {
                        "text/plain": "LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,\n          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n          penalty='l2', random_state=None, solver='sag', tol=0.0001,\n          verbose=0, warm_start=False)"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "# C=0.01 and solver='sag' give best result\nLogisticRegression_model = LogisticRegression(C=0.01, solver='sag')\nLogisticRegression_model.fit(X_train, y_train)"
        }, 
        {
            "source": "# Model Evaluation using Test set", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 140, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        }, 
        {
            "source": "First, download and load the test set:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "execution_count": 141, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "--2018-10-28 19:22:26--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.193\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.193|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: 'loan_test.csv'\n\nloan_test.csv       100%[===================>]   3.56K  --.-KB/s    in 0s      \n\n2018-10-28 19:22:26 (305 MB/s) - 'loan_test.csv' saved [3642/3642]\n\n"
                }
            ], 
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        }, 
        {
            "source": "### Load Test set for evaluation ", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "execution_count": 149, 
            "cell_type": "code", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }, 
            "outputs": [
                {
                    "execution_count": 149, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30       9/8/2016   \n1           5             5     PAIDOFF        300      7       9/9/2016   \n2          21            21     PAIDOFF       1000     30      9/10/2016   \n3          24            24     PAIDOFF       1000     30      9/10/2016   \n4          35            35     PAIDOFF        800     15      9/11/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   50              Bechalor  female  \n1  9/15/2016   35       Master or Above    male  \n2  10/9/2016   43  High School or Below  female  \n3  10/9/2016   26               college    male  \n4  9/25/2016   29              Bechalor    male  "
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "test_df = pd.read_csv('loan_test.csv')\ntest_df.head()"
        }, 
        {
            "execution_count": 150, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)"
        }, 
        {
            "execution_count": 151, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\ntest_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\ntest_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x > 3) else 0)"
        }, 
        {
            "execution_count": 152, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [], 
            "source": "X_testset = test_df[['Principal','terms','age','Gender','weekend']]\nX_testset = pd.concat([X_testset, pd.get_dummies(test_df['education'])], axis=1)\nX_testset.drop(['Master or Above'], axis = 1,inplace=True)\nX_testset = preprocessing.StandardScaler().fit(X_testset).transform(X_testset)\n\ny_testset = test_df['loan_status'].values"
        }, 
        {
            "execution_count": 154, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stdout", 
                    "text": "(54, 8) (54,)\n"
                }
            ], 
            "source": "print(X_testset.shape, y_testset.shape)"
        }, 
        {
            "execution_count": 155, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "output_type": "stream", 
                    "name": "stderr", 
                    "text": "/opt/ibm/conda/miniconda3/lib/python3.5/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.\n  'precision', 'predicted', average, warn_for)\n"
                }
            ], 
            "source": "report = pd.DataFrame()\nreport['Algorithm'] = ['KNN', 'Decision Tree', 'SVM', 'Logistic Regression']\n\nreport['Jaccard'] = [jaccard_similarity_score(y_testset, K_neighbors_model.predict(X_testset)),\n                    jaccard_similarity_score(y_testset, D_tree_model.predict(X_testset)),\n                    jaccard_similarity_score(y_testset, SVM_model.predict(X_testset)),\n                    jaccard_similarity_score(y_testset, LogisticRegression_model.predict(X_testset))]\n\nreport['F1-score'] = [f1_score(y_testset, K_neighbors_model.predict(X_testset), average='weighted'),\n                    f1_score(y_testset, D_tree_model.predict(X_testset), average='weighted'),\n                    f1_score(y_testset, SVM_model.predict(X_testset), average='weighted'),\n                    f1_score(y_testset, LogisticRegression_model.predict(X_testset), average='weighted')]\n\nreport['LogLoss'] = ['NA',\n                    'NA',\n                    'NA',\n                    log_loss(y_testset, LogisticRegression_model.predict_proba(X_testset))]"
        }, 
        {
            "execution_count": 156, 
            "cell_type": "code", 
            "metadata": {}, 
            "outputs": [
                {
                    "execution_count": 156, 
                    "metadata": {}, 
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Algorithm</th>\n      <th>Jaccard</th>\n      <th>F1-score</th>\n      <th>LogLoss</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>KNN</td>\n      <td>0.666667</td>\n      <td>0.632840</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>Decision Tree</td>\n      <td>0.722222</td>\n      <td>0.725253</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>SVM</td>\n      <td>0.740741</td>\n      <td>0.630418</td>\n      <td>NA</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>Logistic Regression</td>\n      <td>0.740741</td>\n      <td>0.630418</td>\n      <td>0.516359</td>\n    </tr>\n  </tbody>\n</table>\n</div>", 
                        "text/plain": "             Algorithm   Jaccard  F1-score   LogLoss\n0                  KNN  0.666667  0.632840        NA\n1        Decision Tree  0.722222  0.725253        NA\n2                  SVM  0.740741  0.630418        NA\n3  Logistic Regression  0.740741  0.630418  0.516359"
                    }, 
                    "output_type": "execute_result"
                }
            ], 
            "source": "report"
        }, 
        {
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | ?       | ?        | NA      |\n| Decision Tree      | ?       | ?        | NA      |\n| SVM                | ?       | ?        | NA      |\n| LogisticRegression | ?       | ?        | ?       |", 
            "cell_type": "markdown", 
            "metadata": {}
        }, 
        {
            "source": "## Want to learn more?\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: [SPSS Modeler](http://cocl.us/ML0101EN-SPSSModeler).\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at [Watson Studio](https://cocl.us/ML0101EN_DSX)\n\n\n<hr>\nCopyright &copy; 2018 [Cognitive Class](https://cocl.us/DX0108EN_CC). This notebook and its source code are released under the terms of the [MIT License](https://bigdatauniversity.com/mit-license/).\u200b", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }, 
        {
            "source": "### Thanks for completing this lesson!\n\nNotebook created by: <a href = \"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>", 
            "cell_type": "markdown", 
            "metadata": {
                "button": false, 
                "new_sheet": false, 
                "run_control": {
                    "read_only": false
                }
            }
        }
    ], 
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.5 with Spark", 
            "name": "python3", 
            "language": "python3"
        }, 
        "language_info": {
            "mimetype": "text/x-python", 
            "nbconvert_exporter": "python", 
            "version": "3.5.4", 
            "name": "python", 
            "file_extension": ".py", 
            "pygments_lexer": "ipython3", 
            "codemirror_mode": {
                "version": 3, 
                "name": "ipython"
            }
        }
    }, 
    "nbformat": 4
}