Weather Data Classification using Decision Trees.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<p style=\"font-family: Arial; font-size:2.75em;color:purple; font-style:bold\">\n",
    "\n",
    "Classification of Weather Data <br><br>\n",
    "using scikit-learn\n",
    "<br><br>\n",
    "</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "Daily Weather Data Analysis</p>\n",
    "\n",
    "In this notebook, we will use scikit-learn to perform a decision tree based classification of weather data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Importing the Necessary Libraries<br></p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Creating a Pandas DataFrame from a CSV file<br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "data = pd.read_csv('./weather/daily_weather.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\">Daily Weather Data Description</p>\n",
    "<br>\n",
    "The file **daily_weather.csv** is a comma-separated file that contains weather data.  This data comes from a weather station located in San Diego, California.  The weather station is equipped with sensors that capture weather-related measurements such as air temperature, air pressure, and relative humidity.  Data was collected for a period of three years, from September 2011 to September 2014, to ensure that sufficient data for different seasons and weather conditions is captured.<br><br>\n",
    "Let's now check all the columns in the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['number', 'air_pressure_9am', 'air_temp_9am', 'avg_wind_direction_9am',\n",
       "       'avg_wind_speed_9am', 'max_wind_direction_9am', 'max_wind_speed_9am',\n",
       "       'rain_accumulation_9am', 'rain_duration_9am', 'relative_humidity_9am',\n",
       "       'relative_humidity_3pm'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<br>Each row in daily_weather.csv captures weather data for a separate day.  <br><br>\n",
    "Sensor measurements from the weather station were captured at one-minute intervals.  These measurements were then processed to generate values to describe daily weather. Since this dataset was created to classify low-humidity days vs. non-low-humidity days (that is, days with normal or high humidity), the variables included are weather measurements in the morning, with one measurement, namely relatively humidity, in the afternoon.  The idea is to use the morning weather values to predict whether the day will be low-humidity or not based on the afternoon measurement of relative humidity.\n",
    "\n",
    "Each row, or sample, consists of the following variables:\n",
    "\n",
    "* **number:** unique number for each row\n",
    "* **air_pressure_9am:** air pressure averaged over a period from 8:55am to 9:04am (*Unit: hectopascals*)\n",
    "* **air_temp_9am:** air temperature averaged over a period from 8:55am to 9:04am (*Unit: degrees Fahrenheit*)\n",
    "* **air_wind_direction_9am:** wind direction averaged over a period from 8:55am to 9:04am (*Unit: degrees, with 0 means coming from the North, and increasing clockwise*)\n",
    "* **air_wind_speed_9am:** wind speed averaged over a period from 8:55am to 9:04am (*Unit: miles per hour*)\n",
    "* ** max_wind_direction_9am:** wind gust direction averaged over a period from 8:55am to 9:10am (*Unit: degrees, with 0 being North and increasing clockwise*)\n",
    "* **max_wind_speed_9am:** wind gust speed averaged over a period from 8:55am to 9:04am (*Unit: miles per hour*)\n",
    "* **rain_accumulation_9am:** amount of rain accumulated in the 24 hours prior to 9am (*Unit: millimeters*)\n",
    "* **rain_duration_9am:** amount of time rain was recorded in the 24 hours prior to 9am (*Unit: seconds*)\n",
    "* **relative_humidity_9am:** relative humidity averaged over a period from 8:55am to 9:04am (*Unit: percent*)\n",
    "* **relative_humidity_3pm:** relative humidity averaged over a period from 2:55pm to 3:04pm (*Unit: percent *)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>number</th>\n",
       "      <th>air_pressure_9am</th>\n",
       "      <th>air_temp_9am</th>\n",
       "      <th>avg_wind_direction_9am</th>\n",
       "      <th>avg_wind_speed_9am</th>\n",
       "      <th>max_wind_direction_9am</th>\n",
       "      <th>max_wind_speed_9am</th>\n",
       "      <th>rain_accumulation_9am</th>\n",
       "      <th>rain_duration_9am</th>\n",
       "      <th>relative_humidity_9am</th>\n",
       "      <th>relative_humidity_3pm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>918.060000</td>\n",
       "      <td>74.822000</td>\n",
       "      <td>271.100000</td>\n",
       "      <td>2.080354</td>\n",
       "      <td>295.400000</td>\n",
       "      <td>2.863283</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>42.420000</td>\n",
       "      <td>36.160000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>917.347688</td>\n",
       "      <td>71.403843</td>\n",
       "      <td>101.935179</td>\n",
       "      <td>2.443009</td>\n",
       "      <td>140.471548</td>\n",
       "      <td>3.533324</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>24.328697</td>\n",
       "      <td>19.426597</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>923.040000</td>\n",
       "      <td>60.638000</td>\n",
       "      <td>51.000000</td>\n",
       "      <td>17.067852</td>\n",
       "      <td>63.700000</td>\n",
       "      <td>22.100967</td>\n",
       "      <td>0.00</td>\n",
       "      <td>20.0</td>\n",
       "      <td>8.900000</td>\n",
       "      <td>14.460000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>920.502751</td>\n",
       "      <td>70.138895</td>\n",
       "      <td>198.832133</td>\n",
       "      <td>4.337363</td>\n",
       "      <td>211.203341</td>\n",
       "      <td>5.190045</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.189102</td>\n",
       "      <td>12.742547</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>921.160000</td>\n",
       "      <td>44.294000</td>\n",
       "      <td>277.800000</td>\n",
       "      <td>1.856660</td>\n",
       "      <td>136.500000</td>\n",
       "      <td>2.863283</td>\n",
       "      <td>8.90</td>\n",
       "      <td>14730.0</td>\n",
       "      <td>92.410000</td>\n",
       "      <td>76.740000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5</td>\n",
       "      <td>915.300000</td>\n",
       "      <td>78.404000</td>\n",
       "      <td>182.800000</td>\n",
       "      <td>9.932014</td>\n",
       "      <td>189.000000</td>\n",
       "      <td>10.983375</td>\n",
       "      <td>0.02</td>\n",
       "      <td>170.0</td>\n",
       "      <td>35.130000</td>\n",
       "      <td>33.930000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6</td>\n",
       "      <td>915.598868</td>\n",
       "      <td>70.043304</td>\n",
       "      <td>177.875407</td>\n",
       "      <td>3.745587</td>\n",
       "      <td>186.606696</td>\n",
       "      <td>4.589632</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>10.657422</td>\n",
       "      <td>21.385657</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>7</td>\n",
       "      <td>918.070000</td>\n",
       "      <td>51.710000</td>\n",
       "      <td>242.400000</td>\n",
       "      <td>2.527742</td>\n",
       "      <td>271.600000</td>\n",
       "      <td>3.646212</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>80.470000</td>\n",
       "      <td>74.920000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>8</td>\n",
       "      <td>920.080000</td>\n",
       "      <td>80.582000</td>\n",
       "      <td>40.700000</td>\n",
       "      <td>4.518619</td>\n",
       "      <td>63.000000</td>\n",
       "      <td>5.883152</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>29.580000</td>\n",
       "      <td>24.030000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>9</td>\n",
       "      <td>915.010000</td>\n",
       "      <td>47.498000</td>\n",
       "      <td>163.100000</td>\n",
       "      <td>4.943637</td>\n",
       "      <td>195.900000</td>\n",
       "      <td>6.576604</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>88.600000</td>\n",
       "      <td>68.050000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>10</td>\n",
       "      <td>919.650000</td>\n",
       "      <td>77.036000</td>\n",
       "      <td>70.600000</td>\n",
       "      <td>3.825167</td>\n",
       "      <td>85.500000</td>\n",
       "      <td>4.764682</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>22.070000</td>\n",
       "      <td>32.130000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>11</td>\n",
       "      <td>915.640000</td>\n",
       "      <td>45.716000</td>\n",
       "      <td>241.600000</td>\n",
       "      <td>5.860783</td>\n",
       "      <td>265.800000</td>\n",
       "      <td>8.030615</td>\n",
       "      <td>0.55</td>\n",
       "      <td>1770.0</td>\n",
       "      <td>90.560000</td>\n",
       "      <td>79.090000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>12</td>\n",
       "      <td>917.390000</td>\n",
       "      <td>49.784000</td>\n",
       "      <td>204.100000</td>\n",
       "      <td>1.275056</td>\n",
       "      <td>211.800000</td>\n",
       "      <td>2.013246</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>73.150000</td>\n",
       "      <td>58.430000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>13</td>\n",
       "      <td>920.820000</td>\n",
       "      <td>62.438000</td>\n",
       "      <td>213.600000</td>\n",
       "      <td>2.617220</td>\n",
       "      <td>165.700000</td>\n",
       "      <td>3.310671</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>43.640000</td>\n",
       "      <td>27.990000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>14</td>\n",
       "      <td>911.000000</td>\n",
       "      <td>86.432000</td>\n",
       "      <td>202.900000</td>\n",
       "      <td>1.207948</td>\n",
       "      <td>162.900000</td>\n",
       "      <td>1.677705</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>15.190000</td>\n",
       "      <td>24.370000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>15</td>\n",
       "      <td>922.383131</td>\n",
       "      <td>70.865263</td>\n",
       "      <td>36.174175</td>\n",
       "      <td>1.847278</td>\n",
       "      <td>58.428632</td>\n",
       "      <td>2.529142</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>12.110889</td>\n",
       "      <td>14.801706</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>917.890000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>169.200000</td>\n",
       "      <td>2.192201</td>\n",
       "      <td>196.800000</td>\n",
       "      <td>2.930391</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48.990000</td>\n",
       "      <td>51.190000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>17</td>\n",
       "      <td>916.915255</td>\n",
       "      <td>77.018961</td>\n",
       "      <td>234.539345</td>\n",
       "      <td>2.274725</td>\n",
       "      <td>229.474199</td>\n",
       "      <td>2.906513</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>21.031462</td>\n",
       "      <td>20.755683</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>18</td>\n",
       "      <td>918.800000</td>\n",
       "      <td>67.082000</td>\n",
       "      <td>176.100000</td>\n",
       "      <td>4.876529</td>\n",
       "      <td>183.400000</td>\n",
       "      <td>5.569981</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>18.900000</td>\n",
       "      <td>45.870000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>19</td>\n",
       "      <td>922.040000</td>\n",
       "      <td>68.576000</td>\n",
       "      <td>58.300000</td>\n",
       "      <td>9.551734</td>\n",
       "      <td>81.900000</td>\n",
       "      <td>12.571603</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.540000</td>\n",
       "      <td>7.740000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>20</td>\n",
       "      <td>919.992262</td>\n",
       "      <td>62.964383</td>\n",
       "      <td>54.799094</td>\n",
       "      <td>12.680436</td>\n",
       "      <td>74.254223</td>\n",
       "      <td>15.452306</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>18.809518</td>\n",
       "      <td>14.649909</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>21</td>\n",
       "      <td>917.230000</td>\n",
       "      <td>67.676000</td>\n",
       "      <td>177.800000</td>\n",
       "      <td>2.460634</td>\n",
       "      <td>93.200000</td>\n",
       "      <td>3.288302</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.640000</td>\n",
       "      <td>41.340000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>22</td>\n",
       "      <td>921.125626</td>\n",
       "      <td>68.818772</td>\n",
       "      <td>71.799092</td>\n",
       "      <td>2.576538</td>\n",
       "      <td>95.472334</td>\n",
       "      <td>3.487444</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.742201</td>\n",
       "      <td>17.281161</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>23</td>\n",
       "      <td>920.350000</td>\n",
       "      <td>47.570000</td>\n",
       "      <td>192.100000</td>\n",
       "      <td>6.263432</td>\n",
       "      <td>205.700000</td>\n",
       "      <td>7.605596</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>54.550000</td>\n",
       "      <td>66.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>24</th>\n",
       "      <td>24</td>\n",
       "      <td>921.788229</td>\n",
       "      <td>71.659572</td>\n",
       "      <td>217.405520</td>\n",
       "      <td>1.946447</td>\n",
       "      <td>253.758003</td>\n",
       "      <td>2.719712</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.194250</td>\n",
       "      <td>16.331716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>25</td>\n",
       "      <td>918.030000</td>\n",
       "      <td>50.666000</td>\n",
       "      <td>128.900000</td>\n",
       "      <td>2.527742</td>\n",
       "      <td>117.400000</td>\n",
       "      <td>4.004123</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>76.880000</td>\n",
       "      <td>47.030000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>26</td>\n",
       "      <td>914.490000</td>\n",
       "      <td>49.892000</td>\n",
       "      <td>163.000000</td>\n",
       "      <td>4.854160</td>\n",
       "      <td>189.600000</td>\n",
       "      <td>6.733189</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>92.100000</td>\n",
       "      <td>90.990000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>27</th>\n",
       "      <td>27</td>\n",
       "      <td>914.900000</td>\n",
       "      <td>78.620000</td>\n",
       "      <td>203.300000</td>\n",
       "      <td>1.811921</td>\n",
       "      <td>240.000000</td>\n",
       "      <td>2.751436</td>\n",
       "      <td>0.00</td>\n",
       "      <td>220.0</td>\n",
       "      <td>47.890000</td>\n",
       "      <td>43.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>28</th>\n",
       "      <td>28</td>\n",
       "      <td>915.840000</td>\n",
       "      <td>40.118000</td>\n",
       "      <td>171.900000</td>\n",
       "      <td>10.535987</td>\n",
       "      <td>188.000000</td>\n",
       "      <td>12.929513</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>86.430000</td>\n",
       "      <td>84.390000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>29</th>\n",
       "      <td>29</td>\n",
       "      <td>916.310000</td>\n",
       "      <td>45.428000</td>\n",
       "      <td>183.100000</td>\n",
       "      <td>8.343786</td>\n",
       "      <td>194.600000</td>\n",
       "      <td>10.088599</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>86.150000</td>\n",
       "      <td>90.580000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1065</th>\n",
       "      <td>1065</td>\n",
       "      <td>915.600000</td>\n",
       "      <td>69.584000</td>\n",
       "      <td>185.500000</td>\n",
       "      <td>4.630466</td>\n",
       "      <td>198.200000</td>\n",
       "      <td>5.480503</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>28.970000</td>\n",
       "      <td>39.370000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1066</th>\n",
       "      <td>1066</td>\n",
       "      <td>919.564869</td>\n",
       "      <td>73.726732</td>\n",
       "      <td>68.704694</td>\n",
       "      <td>3.551777</td>\n",
       "      <td>102.571616</td>\n",
       "      <td>4.861315</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.657314</td>\n",
       "      <td>17.331823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1067</th>\n",
       "      <td>1067</td>\n",
       "      <td>917.690000</td>\n",
       "      <td>64.994000</td>\n",
       "      <td>178.300000</td>\n",
       "      <td>2.975130</td>\n",
       "      <td>193.900000</td>\n",
       "      <td>3.735690</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>54.880000</td>\n",
       "      <td>52.070000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1068</th>\n",
       "      <td>1068</td>\n",
       "      <td>920.330000</td>\n",
       "      <td>68.864000</td>\n",
       "      <td>122.400000</td>\n",
       "      <td>2.035615</td>\n",
       "      <td>182.000000</td>\n",
       "      <td>3.086977</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>52.380000</td>\n",
       "      <td>62.620000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1069</th>\n",
       "      <td>1069</td>\n",
       "      <td>918.260000</td>\n",
       "      <td>82.220000</td>\n",
       "      <td>186.600000</td>\n",
       "      <td>1.096101</td>\n",
       "      <td>221.500000</td>\n",
       "      <td>1.722444</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>32.460000</td>\n",
       "      <td>56.930000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1070</th>\n",
       "      <td>1070</td>\n",
       "      <td>911.600000</td>\n",
       "      <td>46.346000</td>\n",
       "      <td>110.200000</td>\n",
       "      <td>2.304048</td>\n",
       "      <td>96.100000</td>\n",
       "      <td>3.154085</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>90.510000</td>\n",
       "      <td>62.490000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1071</th>\n",
       "      <td>1071</td>\n",
       "      <td>919.200000</td>\n",
       "      <td>70.250000</td>\n",
       "      <td>156.700000</td>\n",
       "      <td>2.013246</td>\n",
       "      <td>172.300000</td>\n",
       "      <td>2.617220</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>42.060000</td>\n",
       "      <td>41.850000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1072</th>\n",
       "      <td>1072</td>\n",
       "      <td>923.100000</td>\n",
       "      <td>75.596000</td>\n",
       "      <td>178.300000</td>\n",
       "      <td>1.409272</td>\n",
       "      <td>205.200000</td>\n",
       "      <td>1.856660</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.880000</td>\n",
       "      <td>19.810000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1073</th>\n",
       "      <td>1073</td>\n",
       "      <td>917.920000</td>\n",
       "      <td>84.650000</td>\n",
       "      <td>30.000000</td>\n",
       "      <td>4.831790</td>\n",
       "      <td>40.100000</td>\n",
       "      <td>5.838413</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>16.220000</td>\n",
       "      <td>27.450000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1074</th>\n",
       "      <td>1074</td>\n",
       "      <td>913.870000</td>\n",
       "      <td>66.938000</td>\n",
       "      <td>171.900000</td>\n",
       "      <td>6.509495</td>\n",
       "      <td>182.300000</td>\n",
       "      <td>7.471380</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>57.430000</td>\n",
       "      <td>53.380000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1075</th>\n",
       "      <td>1075</td>\n",
       "      <td>922.858110</td>\n",
       "      <td>64.989361</td>\n",
       "      <td>63.483047</td>\n",
       "      <td>10.261187</td>\n",
       "      <td>73.170504</td>\n",
       "      <td>11.949206</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.525109</td>\n",
       "      <td>15.511157</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1076</th>\n",
       "      <td>1076</td>\n",
       "      <td>925.550000</td>\n",
       "      <td>50.918000</td>\n",
       "      <td>32.600000</td>\n",
       "      <td>5.413395</td>\n",
       "      <td>57.500000</td>\n",
       "      <td>8.120092</td>\n",
       "      <td>0.00</td>\n",
       "      <td>10.0</td>\n",
       "      <td>29.090000</td>\n",
       "      <td>23.720000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1077</th>\n",
       "      <td>1077</td>\n",
       "      <td>920.130000</td>\n",
       "      <td>73.292000</td>\n",
       "      <td>193.600000</td>\n",
       "      <td>4.115970</td>\n",
       "      <td>211.300000</td>\n",
       "      <td>4.966007</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>33.520000</td>\n",
       "      <td>38.120000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1078</th>\n",
       "      <td>1078</td>\n",
       "      <td>924.017856</td>\n",
       "      <td>68.370307</td>\n",
       "      <td>56.304083</td>\n",
       "      <td>15.162800</td>\n",
       "      <td>76.585117</td>\n",
       "      <td>19.803578</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.599240</td>\n",
       "      <td>9.373013</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1079</th>\n",
       "      <td>1079</td>\n",
       "      <td>917.200000</td>\n",
       "      <td>65.264000</td>\n",
       "      <td>137.600000</td>\n",
       "      <td>1.767183</td>\n",
       "      <td>176.700000</td>\n",
       "      <td>2.550112</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>58.370000</td>\n",
       "      <td>65.590000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1080</th>\n",
       "      <td>1080</td>\n",
       "      <td>916.260000</td>\n",
       "      <td>79.772000</td>\n",
       "      <td>185.700000</td>\n",
       "      <td>3.511996</td>\n",
       "      <td>201.500000</td>\n",
       "      <td>4.160708</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27.680000</td>\n",
       "      <td>20.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1081</th>\n",
       "      <td>1081</td>\n",
       "      <td>920.856913</td>\n",
       "      <td>69.884338</td>\n",
       "      <td>47.335966</td>\n",
       "      <td>12.238670</td>\n",
       "      <td>66.729162</td>\n",
       "      <td>16.067860</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>11.956852</td>\n",
       "      <td>12.696233</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1082</th>\n",
       "      <td>1082</td>\n",
       "      <td>916.130000</td>\n",
       "      <td>50.108000</td>\n",
       "      <td>211.400000</td>\n",
       "      <td>2.550112</td>\n",
       "      <td>231.800000</td>\n",
       "      <td>3.534365</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>89.800000</td>\n",
       "      <td>53.860000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1083</th>\n",
       "      <td>1083</td>\n",
       "      <td>916.320000</td>\n",
       "      <td>48.308000</td>\n",
       "      <td>46.900000</td>\n",
       "      <td>4.854160</td>\n",
       "      <td>61.700000</td>\n",
       "      <td>5.860783</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>91.030000</td>\n",
       "      <td>62.400000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1084</th>\n",
       "      <td>1084</td>\n",
       "      <td>917.130000</td>\n",
       "      <td>80.240000</td>\n",
       "      <td>183.300000</td>\n",
       "      <td>1.632966</td>\n",
       "      <td>38.500000</td>\n",
       "      <td>2.259309</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>28.260000</td>\n",
       "      <td>34.690000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1085</th>\n",
       "      <td>1085</td>\n",
       "      <td>914.840000</td>\n",
       "      <td>47.354000</td>\n",
       "      <td>190.900000</td>\n",
       "      <td>3.713320</td>\n",
       "      <td>204.400000</td>\n",
       "      <td>4.652835</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>92.300000</td>\n",
       "      <td>88.160000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1086</th>\n",
       "      <td>1086</td>\n",
       "      <td>921.260000</td>\n",
       "      <td>52.646000</td>\n",
       "      <td>261.900000</td>\n",
       "      <td>2.035615</td>\n",
       "      <td>260.500000</td>\n",
       "      <td>3.042238</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>91.110000</td>\n",
       "      <td>81.890000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1087</th>\n",
       "      <td>1087</td>\n",
       "      <td>914.000000</td>\n",
       "      <td>66.650000</td>\n",
       "      <td>173.800000</td>\n",
       "      <td>8.366156</td>\n",
       "      <td>181.000000</td>\n",
       "      <td>9.439887</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.920000</td>\n",
       "      <td>47.340000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1088</th>\n",
       "      <td>1088</td>\n",
       "      <td>912.900000</td>\n",
       "      <td>71.870000</td>\n",
       "      <td>129.200000</td>\n",
       "      <td>1.431642</td>\n",
       "      <td>160.000000</td>\n",
       "      <td>2.057985</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>51.840000</td>\n",
       "      <td>55.490000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1089</th>\n",
       "      <td>1089</td>\n",
       "      <td>915.000000</td>\n",
       "      <td>55.040000</td>\n",
       "      <td>191.800000</td>\n",
       "      <td>5.368656</td>\n",
       "      <td>220.900000</td>\n",
       "      <td>7.068730</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>73.550000</td>\n",
       "      <td>69.670000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1090</th>\n",
       "      <td>1090</td>\n",
       "      <td>918.900000</td>\n",
       "      <td>63.104000</td>\n",
       "      <td>192.900000</td>\n",
       "      <td>3.869906</td>\n",
       "      <td>207.300000</td>\n",
       "      <td>5.212070</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>26.020000</td>\n",
       "      <td>38.180000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1091</th>\n",
       "      <td>1091</td>\n",
       "      <td>918.710000</td>\n",
       "      <td>49.568000</td>\n",
       "      <td>241.600000</td>\n",
       "      <td>1.811921</td>\n",
       "      <td>227.400000</td>\n",
       "      <td>2.371156</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>90.350000</td>\n",
       "      <td>73.340000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1092</th>\n",
       "      <td>1092</td>\n",
       "      <td>916.600000</td>\n",
       "      <td>71.096000</td>\n",
       "      <td>189.300000</td>\n",
       "      <td>3.064608</td>\n",
       "      <td>200.800000</td>\n",
       "      <td>3.892276</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>45.590000</td>\n",
       "      <td>52.310000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1093</th>\n",
       "      <td>1093</td>\n",
       "      <td>912.600000</td>\n",
       "      <td>58.406000</td>\n",
       "      <td>172.700000</td>\n",
       "      <td>3.825167</td>\n",
       "      <td>189.100000</td>\n",
       "      <td>4.764682</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>64.840000</td>\n",
       "      <td>58.280000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1094</th>\n",
       "      <td>1094</td>\n",
       "      <td>921.530000</td>\n",
       "      <td>77.702000</td>\n",
       "      <td>97.100000</td>\n",
       "      <td>3.265932</td>\n",
       "      <td>125.900000</td>\n",
       "      <td>4.451511</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.560000</td>\n",
       "      <td>15.100000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>1095 rows × 11 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      number  air_pressure_9am  air_temp_9am  avg_wind_direction_9am  \\\n",
       "0          0        918.060000     74.822000              271.100000   \n",
       "1          1        917.347688     71.403843              101.935179   \n",
       "2          2        923.040000     60.638000               51.000000   \n",
       "3          3        920.502751     70.138895              198.832133   \n",
       "4          4        921.160000     44.294000              277.800000   \n",
       "5          5        915.300000     78.404000              182.800000   \n",
       "6          6        915.598868     70.043304              177.875407   \n",
       "7          7        918.070000     51.710000              242.400000   \n",
       "8          8        920.080000     80.582000               40.700000   \n",
       "9          9        915.010000     47.498000              163.100000   \n",
       "10        10        919.650000     77.036000               70.600000   \n",
       "11        11        915.640000     45.716000              241.600000   \n",
       "12        12        917.390000     49.784000              204.100000   \n",
       "13        13        920.820000     62.438000              213.600000   \n",
       "14        14        911.000000     86.432000              202.900000   \n",
       "15        15        922.383131     70.865263               36.174175   \n",
       "16        16        917.890000           NaN              169.200000   \n",
       "17        17        916.915255     77.018961              234.539345   \n",
       "18        18        918.800000     67.082000              176.100000   \n",
       "19        19        922.040000     68.576000               58.300000   \n",
       "20        20        919.992262     62.964383               54.799094   \n",
       "21        21        917.230000     67.676000              177.800000   \n",
       "22        22        921.125626     68.818772               71.799092   \n",
       "23        23        920.350000     47.570000              192.100000   \n",
       "24        24        921.788229     71.659572              217.405520   \n",
       "25        25        918.030000     50.666000              128.900000   \n",
       "26        26        914.490000     49.892000              163.000000   \n",
       "27        27        914.900000     78.620000              203.300000   \n",
       "28        28        915.840000     40.118000              171.900000   \n",
       "29        29        916.310000     45.428000              183.100000   \n",
       "...      ...               ...           ...                     ...   \n",
       "1065    1065        915.600000     69.584000              185.500000   \n",
       "1066    1066        919.564869     73.726732               68.704694   \n",
       "1067    1067        917.690000     64.994000              178.300000   \n",
       "1068    1068        920.330000     68.864000              122.400000   \n",
       "1069    1069        918.260000     82.220000              186.600000   \n",
       "1070    1070        911.600000     46.346000              110.200000   \n",
       "1071    1071        919.200000     70.250000              156.700000   \n",
       "1072    1072        923.100000     75.596000              178.300000   \n",
       "1073    1073        917.920000     84.650000               30.000000   \n",
       "1074    1074        913.870000     66.938000              171.900000   \n",
       "1075    1075        922.858110     64.989361               63.483047   \n",
       "1076    1076        925.550000     50.918000               32.600000   \n",
       "1077    1077        920.130000     73.292000              193.600000   \n",
       "1078    1078        924.017856     68.370307               56.304083   \n",
       "1079    1079        917.200000     65.264000              137.600000   \n",
       "1080    1080        916.260000     79.772000              185.700000   \n",
       "1081    1081        920.856913     69.884338               47.335966   \n",
       "1082    1082        916.130000     50.108000              211.400000   \n",
       "1083    1083        916.320000     48.308000               46.900000   \n",
       "1084    1084        917.130000     80.240000              183.300000   \n",
       "1085    1085        914.840000     47.354000              190.900000   \n",
       "1086    1086        921.260000     52.646000              261.900000   \n",
       "1087    1087        914.000000     66.650000              173.800000   \n",
       "1088    1088        912.900000     71.870000              129.200000   \n",
       "1089    1089        915.000000     55.040000              191.800000   \n",
       "1090    1090        918.900000     63.104000              192.900000   \n",
       "1091    1091        918.710000     49.568000              241.600000   \n",
       "1092    1092        916.600000     71.096000              189.300000   \n",
       "1093    1093        912.600000     58.406000              172.700000   \n",
       "1094    1094        921.530000     77.702000               97.100000   \n",
       "\n",
       "      avg_wind_speed_9am  max_wind_direction_9am  max_wind_speed_9am  \\\n",
       "0               2.080354              295.400000            2.863283   \n",
       "1               2.443009              140.471548            3.533324   \n",
       "2              17.067852               63.700000           22.100967   \n",
       "3               4.337363              211.203341            5.190045   \n",
       "4               1.856660              136.500000            2.863283   \n",
       "5               9.932014              189.000000           10.983375   \n",
       "6               3.745587              186.606696            4.589632   \n",
       "7               2.527742              271.600000            3.646212   \n",
       "8               4.518619               63.000000            5.883152   \n",
       "9               4.943637              195.900000            6.576604   \n",
       "10              3.825167               85.500000            4.764682   \n",
       "11              5.860783              265.800000            8.030615   \n",
       "12              1.275056              211.800000            2.013246   \n",
       "13              2.617220              165.700000            3.310671   \n",
       "14              1.207948              162.900000            1.677705   \n",
       "15              1.847278               58.428632            2.529142   \n",
       "16              2.192201              196.800000            2.930391   \n",
       "17              2.274725              229.474199            2.906513   \n",
       "18              4.876529              183.400000            5.569981   \n",
       "19              9.551734               81.900000           12.571603   \n",
       "20             12.680436               74.254223           15.452306   \n",
       "21              2.460634               93.200000            3.288302   \n",
       "22              2.576538               95.472334            3.487444   \n",
       "23              6.263432              205.700000            7.605596   \n",
       "24              1.946447              253.758003            2.719712   \n",
       "25              2.527742              117.400000            4.004123   \n",
       "26              4.854160              189.600000            6.733189   \n",
       "27              1.811921              240.000000            2.751436   \n",
       "28             10.535987              188.000000           12.929513   \n",
       "29              8.343786              194.600000           10.088599   \n",
       "...                  ...                     ...                 ...   \n",
       "1065            4.630466              198.200000            5.480503   \n",
       "1066            3.551777              102.571616            4.861315   \n",
       "1067            2.975130              193.900000            3.735690   \n",
       "1068            2.035615              182.000000            3.086977   \n",
       "1069            1.096101              221.500000            1.722444   \n",
       "1070            2.304048               96.100000            3.154085   \n",
       "1071            2.013246              172.300000            2.617220   \n",
       "1072            1.409272              205.200000            1.856660   \n",
       "1073            4.831790               40.100000            5.838413   \n",
       "1074            6.509495              182.300000            7.471380   \n",
       "1075           10.261187               73.170504           11.949206   \n",
       "1076            5.413395               57.500000            8.120092   \n",
       "1077            4.115970              211.300000            4.966007   \n",
       "1078           15.162800               76.585117           19.803578   \n",
       "1079            1.767183              176.700000            2.550112   \n",
       "1080            3.511996              201.500000            4.160708   \n",
       "1081           12.238670               66.729162           16.067860   \n",
       "1082            2.550112              231.800000            3.534365   \n",
       "1083            4.854160               61.700000            5.860783   \n",
       "1084            1.632966               38.500000            2.259309   \n",
       "1085            3.713320              204.400000            4.652835   \n",
       "1086            2.035615              260.500000            3.042238   \n",
       "1087            8.366156              181.000000            9.439887   \n",
       "1088            1.431642              160.000000            2.057985   \n",
       "1089            5.368656              220.900000            7.068730   \n",
       "1090            3.869906              207.300000            5.212070   \n",
       "1091            1.811921              227.400000            2.371156   \n",
       "1092            3.064608              200.800000            3.892276   \n",
       "1093            3.825167              189.100000            4.764682   \n",
       "1094            3.265932              125.900000            4.451511   \n",
       "\n",
       "      rain_accumulation_9am  rain_duration_9am  relative_humidity_9am  \\\n",
       "0                      0.00                0.0              42.420000   \n",
       "1                      0.00                0.0              24.328697   \n",
       "2                      0.00               20.0               8.900000   \n",
       "3                      0.00                0.0              12.189102   \n",
       "4                      8.90            14730.0              92.410000   \n",
       "5                      0.02              170.0              35.130000   \n",
       "6                      0.00                0.0              10.657422   \n",
       "7                      0.00                0.0              80.470000   \n",
       "8                      0.00                0.0              29.580000   \n",
       "9                      0.00                0.0              88.600000   \n",
       "10                     0.00                0.0              22.070000   \n",
       "11                     0.55             1770.0              90.560000   \n",
       "12                     0.00                0.0              73.150000   \n",
       "13                     0.00                0.0              43.640000   \n",
       "14                     0.00                0.0              15.190000   \n",
       "15                     0.00                0.0              12.110889   \n",
       "16                     0.00                0.0              48.990000   \n",
       "17                     0.00                0.0              21.031462   \n",
       "18                     0.00                0.0              18.900000   \n",
       "19                     0.00                0.0               7.540000   \n",
       "20                     0.00                0.0              18.809518   \n",
       "21                     0.00                0.0              40.640000   \n",
       "22                     0.00                0.0              11.742201   \n",
       "23                     0.00                0.0              54.550000   \n",
       "24                     0.00                0.0              11.194250   \n",
       "25                     0.00                0.0              76.880000   \n",
       "26                     0.00                0.0              92.100000   \n",
       "27                     0.00              220.0              47.890000   \n",
       "28                     0.00                0.0              86.430000   \n",
       "29                     0.00                0.0              86.150000   \n",
       "...                     ...                ...                    ...   \n",
       "1065                   0.00                0.0              28.970000   \n",
       "1066                    NaN                0.0              11.657314   \n",
       "1067                   0.00                0.0              54.880000   \n",
       "1068                   0.00                0.0              52.380000   \n",
       "1069                   0.00                0.0              32.460000   \n",
       "1070                   0.00                0.0              90.510000   \n",
       "1071                   0.00                0.0              42.060000   \n",
       "1072                   0.00                0.0              11.880000   \n",
       "1073                   0.00                0.0              16.220000   \n",
       "1074                   0.00                0.0              57.430000   \n",
       "1075                   0.00                0.0              14.525109   \n",
       "1076                   0.00               10.0              29.090000   \n",
       "1077                   0.00                0.0              33.520000   \n",
       "1078                   0.00                0.0              11.599240   \n",
       "1079                   0.00                0.0              58.370000   \n",
       "1080                   0.00                0.0              27.680000   \n",
       "1081                   0.00                0.0              11.956852   \n",
       "1082                   0.00                0.0              89.800000   \n",
       "1083                   0.00                0.0              91.030000   \n",
       "1084                   0.00                0.0              28.260000   \n",
       "1085                   0.00                0.0              92.300000   \n",
       "1086                   0.00                0.0              91.110000   \n",
       "1087                   0.00                0.0              30.920000   \n",
       "1088                   0.00                0.0              51.840000   \n",
       "1089                   0.00                0.0              73.550000   \n",
       "1090                   0.00                0.0              26.020000   \n",
       "1091                   0.00                0.0              90.350000   \n",
       "1092                   0.00                0.0              45.590000   \n",
       "1093                   0.00                0.0              64.840000   \n",
       "1094                   0.00                0.0              14.560000   \n",
       "\n",
       "      relative_humidity_3pm  \n",
       "0                 36.160000  \n",
       "1                 19.426597  \n",
       "2                 14.460000  \n",
       "3                 12.742547  \n",
       "4                 76.740000  \n",
       "5                 33.930000  \n",
       "6                 21.385657  \n",
       "7                 74.920000  \n",
       "8                 24.030000  \n",
       "9                 68.050000  \n",
       "10                32.130000  \n",
       "11                79.090000  \n",
       "12                58.430000  \n",
       "13                27.990000  \n",
       "14                24.370000  \n",
       "15                14.801706  \n",
       "16                51.190000  \n",
       "17                20.755683  \n",
       "18                45.870000  \n",
       "19                 7.740000  \n",
       "20                14.649909  \n",
       "21                41.340000  \n",
       "22                17.281161  \n",
       "23                66.000000  \n",
       "24                16.331716  \n",
       "25                47.030000  \n",
       "26                90.990000  \n",
       "27                43.900000  \n",
       "28                84.390000  \n",
       "29                90.580000  \n",
       "...                     ...  \n",
       "1065              39.370000  \n",
       "1066              17.331823  \n",
       "1067              52.070000  \n",
       "1068              62.620000  \n",
       "1069              56.930000  \n",
       "1070              62.490000  \n",
       "1071              41.850000  \n",
       "1072              19.810000  \n",
       "1073              27.450000  \n",
       "1074              53.380000  \n",
       "1075              15.511157  \n",
       "1076              23.720000  \n",
       "1077              38.120000  \n",
       "1078               9.373013  \n",
       "1079              65.590000  \n",
       "1080              20.400000  \n",
       "1081              12.696233  \n",
       "1082              53.860000  \n",
       "1083              62.400000  \n",
       "1084              34.690000  \n",
       "1085              88.160000  \n",
       "1086              81.890000  \n",
       "1087              47.340000  \n",
       "1088              55.490000  \n",
       "1089              69.670000  \n",
       "1090              38.180000  \n",
       "1091              73.340000  \n",
       "1092              52.310000  \n",
       "1093              58.280000  \n",
       "1094              15.100000  \n",
       "\n",
       "[1095 rows x 11 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>number</th>\n",
       "      <th>air_pressure_9am</th>\n",
       "      <th>air_temp_9am</th>\n",
       "      <th>avg_wind_direction_9am</th>\n",
       "      <th>avg_wind_speed_9am</th>\n",
       "      <th>max_wind_direction_9am</th>\n",
       "      <th>max_wind_speed_9am</th>\n",
       "      <th>rain_accumulation_9am</th>\n",
       "      <th>rain_duration_9am</th>\n",
       "      <th>relative_humidity_9am</th>\n",
       "      <th>relative_humidity_3pm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>16</td>\n",
       "      <td>917.890000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>169.200000</td>\n",
       "      <td>2.192201</td>\n",
       "      <td>196.800000</td>\n",
       "      <td>2.930391</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>48.990000</td>\n",
       "      <td>51.190000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>111</td>\n",
       "      <td>915.290000</td>\n",
       "      <td>58.820000</td>\n",
       "      <td>182.600000</td>\n",
       "      <td>15.613841</td>\n",
       "      <td>189.000000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>21.500000</td>\n",
       "      <td>29.690000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>177</th>\n",
       "      <td>177</td>\n",
       "      <td>915.900000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>183.300000</td>\n",
       "      <td>4.719943</td>\n",
       "      <td>189.900000</td>\n",
       "      <td>5.346287</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>29.260000</td>\n",
       "      <td>46.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>262</th>\n",
       "      <td>262</td>\n",
       "      <td>923.596607</td>\n",
       "      <td>58.380598</td>\n",
       "      <td>47.737753</td>\n",
       "      <td>10.636273</td>\n",
       "      <td>67.145843</td>\n",
       "      <td>13.671423</td>\n",
       "      <td>0.000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>17.990876</td>\n",
       "      <td>16.461685</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>277</th>\n",
       "      <td>277</td>\n",
       "      <td>920.480000</td>\n",
       "      <td>62.600000</td>\n",
       "      <td>194.400000</td>\n",
       "      <td>2.751436</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.869906</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>52.580000</td>\n",
       "      <td>54.030000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334</th>\n",
       "      <td>334</td>\n",
       "      <td>916.230000</td>\n",
       "      <td>75.740000</td>\n",
       "      <td>149.100000</td>\n",
       "      <td>2.751436</td>\n",
       "      <td>187.500000</td>\n",
       "      <td>4.183078</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1480.000000</td>\n",
       "      <td>31.880000</td>\n",
       "      <td>32.900000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>358</th>\n",
       "      <td>358</td>\n",
       "      <td>917.440000</td>\n",
       "      <td>58.514000</td>\n",
       "      <td>55.100000</td>\n",
       "      <td>10.021491</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.705819</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>13.880000</td>\n",
       "      <td>25.930000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>361</th>\n",
       "      <td>361</td>\n",
       "      <td>920.444946</td>\n",
       "      <td>65.801845</td>\n",
       "      <td>49.823346</td>\n",
       "      <td>21.520177</td>\n",
       "      <td>61.886944</td>\n",
       "      <td>25.549112</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.364018</td>\n",
       "      <td>12.278715</td>\n",
       "      <td>7.618649</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>381</td>\n",
       "      <td>918.480000</td>\n",
       "      <td>66.542000</td>\n",
       "      <td>90.900000</td>\n",
       "      <td>3.467257</td>\n",
       "      <td>89.400000</td>\n",
       "      <td>4.406772</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>20.640000</td>\n",
       "      <td>14.350000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>409</th>\n",
       "      <td>409</td>\n",
       "      <td>NaN</td>\n",
       "      <td>67.853833</td>\n",
       "      <td>65.880616</td>\n",
       "      <td>4.328594</td>\n",
       "      <td>78.570923</td>\n",
       "      <td>5.216734</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>18.487385</td>\n",
       "      <td>20.356594</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>517</th>\n",
       "      <td>517</td>\n",
       "      <td>920.570000</td>\n",
       "      <td>53.600000</td>\n",
       "      <td>100.100000</td>\n",
       "      <td>4.697574</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.285801</td>\n",
       "      <td>4.712</td>\n",
       "      <td>14842.000000</td>\n",
       "      <td>79.880000</td>\n",
       "      <td>84.530000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>519</th>\n",
       "      <td>519</td>\n",
       "      <td>916.250000</td>\n",
       "      <td>55.670000</td>\n",
       "      <td>176.400000</td>\n",
       "      <td>6.666081</td>\n",
       "      <td>188.200000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>72.550000</td>\n",
       "      <td>74.390000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>546</th>\n",
       "      <td>546</td>\n",
       "      <td>NaN</td>\n",
       "      <td>42.746000</td>\n",
       "      <td>251.100000</td>\n",
       "      <td>12.929513</td>\n",
       "      <td>274.400000</td>\n",
       "      <td>17.604718</td>\n",
       "      <td>14.627</td>\n",
       "      <td>7825.000000</td>\n",
       "      <td>87.870000</td>\n",
       "      <td>70.770000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>620</th>\n",
       "      <td>620</td>\n",
       "      <td>921.200000</td>\n",
       "      <td>56.786000</td>\n",
       "      <td>192.300000</td>\n",
       "      <td>9.551734</td>\n",
       "      <td>201.400000</td>\n",
       "      <td>11.005745</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>59.790000</td>\n",
       "      <td>77.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625</th>\n",
       "      <td>625</td>\n",
       "      <td>912.400000</td>\n",
       "      <td>50.774000</td>\n",
       "      <td>171.600000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181.400000</td>\n",
       "      <td>4.831790</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>86.840000</td>\n",
       "      <td>64.740000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>656</th>\n",
       "      <td>656</td>\n",
       "      <td>920.830000</td>\n",
       "      <td>66.344000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>15.457255</td>\n",
       "      <td>189.400000</td>\n",
       "      <td>16.486248</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>23.770000</td>\n",
       "      <td>51.630000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>670</th>\n",
       "      <td>670</td>\n",
       "      <td>910.920000</td>\n",
       "      <td>48.362000</td>\n",
       "      <td>156.500000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>177.500000</td>\n",
       "      <td>16.128337</td>\n",
       "      <td>4.970</td>\n",
       "      <td>10560.000000</td>\n",
       "      <td>80.560000</td>\n",
       "      <td>88.220000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>672</th>\n",
       "      <td>672</td>\n",
       "      <td>922.448945</td>\n",
       "      <td>72.863773</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.682370</td>\n",
       "      <td>214.196160</td>\n",
       "      <td>4.849450</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>16.753670</td>\n",
       "      <td>17.804720</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>705</th>\n",
       "      <td>705</td>\n",
       "      <td>911.900000</td>\n",
       "      <td>59.072000</td>\n",
       "      <td>199.800000</td>\n",
       "      <td>1.275056</td>\n",
       "      <td>239.500000</td>\n",
       "      <td>1.834291</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>77.630000</td>\n",
       "      <td>59.130000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>731</th>\n",
       "      <td>731</td>\n",
       "      <td>922.970166</td>\n",
       "      <td>51.391847</td>\n",
       "      <td>33.810942</td>\n",
       "      <td>NaN</td>\n",
       "      <td>59.290089</td>\n",
       "      <td>11.111555</td>\n",
       "      <td>0.000</td>\n",
       "      <td>4.735034</td>\n",
       "      <td>34.807753</td>\n",
       "      <td>18.418179</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>737</th>\n",
       "      <td>737</td>\n",
       "      <td>917.895130</td>\n",
       "      <td>76.804690</td>\n",
       "      <td>104.771020</td>\n",
       "      <td>1.632705</td>\n",
       "      <td>97.178763</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>13.771311</td>\n",
       "      <td>16.792455</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>788</th>\n",
       "      <td>788</td>\n",
       "      <td>917.923442</td>\n",
       "      <td>73.249717</td>\n",
       "      <td>42.101739</td>\n",
       "      <td>4.132698</td>\n",
       "      <td>64.284969</td>\n",
       "      <td>5.345258</td>\n",
       "      <td>0.000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6.939692</td>\n",
       "      <td>18.793825</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>840</th>\n",
       "      <td>840</td>\n",
       "      <td>918.043767</td>\n",
       "      <td>NaN</td>\n",
       "      <td>181.774042</td>\n",
       "      <td>0.964376</td>\n",
       "      <td>185.618601</td>\n",
       "      <td>1.570007</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>11.911222</td>\n",
       "      <td>18.154358</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>848</th>\n",
       "      <td>848</td>\n",
       "      <td>915.250000</td>\n",
       "      <td>37.562000</td>\n",
       "      <td>246.500000</td>\n",
       "      <td>11.587349</td>\n",
       "      <td>258.700000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3.171</td>\n",
       "      <td>2891.000000</td>\n",
       "      <td>91.000000</td>\n",
       "      <td>90.780000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>861</th>\n",
       "      <td>861</td>\n",
       "      <td>919.065408</td>\n",
       "      <td>NaN</td>\n",
       "      <td>172.303728</td>\n",
       "      <td>2.639600</td>\n",
       "      <td>193.058141</td>\n",
       "      <td>3.326949</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>12.497839</td>\n",
       "      <td>13.438518</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>869</th>\n",
       "      <td>869</td>\n",
       "      <td>NaN</td>\n",
       "      <td>45.104000</td>\n",
       "      <td>259.000000</td>\n",
       "      <td>3.265932</td>\n",
       "      <td>275.000000</td>\n",
       "      <td>4.026492</td>\n",
       "      <td>0.000</td>\n",
       "      <td>80.000000</td>\n",
       "      <td>85.270000</td>\n",
       "      <td>90.260000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>998</th>\n",
       "      <td>998</td>\n",
       "      <td>914.140000</td>\n",
       "      <td>71.240000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.722444</td>\n",
       "      <td>232.900000</td>\n",
       "      <td>2.326418</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>24.200000</td>\n",
       "      <td>41.380000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1031</th>\n",
       "      <td>1031</td>\n",
       "      <td>922.669195</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.946284</td>\n",
       "      <td>7.969686</td>\n",
       "      <td>65.770066</td>\n",
       "      <td>10.262337</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>18.920805</td>\n",
       "      <td>19.641841</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1035</th>\n",
       "      <td>1035</td>\n",
       "      <td>919.670000</td>\n",
       "      <td>77.576000</td>\n",
       "      <td>171.800000</td>\n",
       "      <td>6.554234</td>\n",
       "      <td>191.000000</td>\n",
       "      <td>8.164831</td>\n",
       "      <td>0.000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>56.860000</td>\n",
       "      <td>50.650000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1063</th>\n",
       "      <td>1063</td>\n",
       "      <td>917.300185</td>\n",
       "      <td>65.790001</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.879553</td>\n",
       "      <td>222.498226</td>\n",
       "      <td>2.692862</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>14.972668</td>\n",
       "      <td>20.966267</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1066</th>\n",
       "      <td>1066</td>\n",
       "      <td>919.564869</td>\n",
       "      <td>73.726732</td>\n",
       "      <td>68.704694</td>\n",
       "      <td>3.551777</td>\n",
       "      <td>102.571616</td>\n",
       "      <td>4.861315</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>11.657314</td>\n",
       "      <td>17.331823</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      number  air_pressure_9am  air_temp_9am  avg_wind_direction_9am  \\\n",
       "16        16        917.890000           NaN              169.200000   \n",
       "111      111        915.290000     58.820000              182.600000   \n",
       "177      177        915.900000           NaN              183.300000   \n",
       "262      262        923.596607     58.380598               47.737753   \n",
       "277      277        920.480000     62.600000              194.400000   \n",
       "334      334        916.230000     75.740000              149.100000   \n",
       "358      358        917.440000     58.514000               55.100000   \n",
       "361      361        920.444946     65.801845               49.823346   \n",
       "381      381        918.480000     66.542000               90.900000   \n",
       "409      409               NaN     67.853833               65.880616   \n",
       "517      517        920.570000     53.600000              100.100000   \n",
       "519      519        916.250000     55.670000              176.400000   \n",
       "546      546               NaN     42.746000              251.100000   \n",
       "620      620        921.200000     56.786000              192.300000   \n",
       "625      625        912.400000     50.774000              171.600000   \n",
       "656      656        920.830000     66.344000                     NaN   \n",
       "670      670        910.920000     48.362000              156.500000   \n",
       "672      672        922.448945     72.863773                     NaN   \n",
       "705      705        911.900000     59.072000              199.800000   \n",
       "731      731        922.970166     51.391847               33.810942   \n",
       "737      737        917.895130     76.804690              104.771020   \n",
       "788      788        917.923442     73.249717               42.101739   \n",
       "840      840        918.043767           NaN              181.774042   \n",
       "848      848        915.250000     37.562000              246.500000   \n",
       "861      861        919.065408           NaN              172.303728   \n",
       "869      869               NaN     45.104000              259.000000   \n",
       "998      998        914.140000     71.240000                     NaN   \n",
       "1031    1031        922.669195           NaN               47.946284   \n",
       "1035    1035        919.670000     77.576000              171.800000   \n",
       "1063    1063        917.300185     65.790001                     NaN   \n",
       "1066    1066        919.564869     73.726732               68.704694   \n",
       "\n",
       "      avg_wind_speed_9am  max_wind_direction_9am  max_wind_speed_9am  \\\n",
       "16              2.192201              196.800000            2.930391   \n",
       "111            15.613841              189.000000                 NaN   \n",
       "177             4.719943              189.900000            5.346287   \n",
       "262            10.636273               67.145843           13.671423   \n",
       "277             2.751436                     NaN            3.869906   \n",
       "334             2.751436              187.500000            4.183078   \n",
       "358            10.021491                     NaN           12.705819   \n",
       "361            21.520177               61.886944           25.549112   \n",
       "381             3.467257               89.400000            4.406772   \n",
       "409             4.328594               78.570923            5.216734   \n",
       "517             4.697574                     NaN            6.285801   \n",
       "519             6.666081              188.200000                 NaN   \n",
       "546            12.929513              274.400000           17.604718   \n",
       "620             9.551734              201.400000           11.005745   \n",
       "625                  NaN              181.400000            4.831790   \n",
       "656            15.457255              189.400000           16.486248   \n",
       "670                  NaN              177.500000           16.128337   \n",
       "672             3.682370              214.196160            4.849450   \n",
       "705             1.275056              239.500000            1.834291   \n",
       "731                  NaN               59.290089           11.111555   \n",
       "737             1.632705               97.178763                 NaN   \n",
       "788             4.132698               64.284969            5.345258   \n",
       "840             0.964376              185.618601            1.570007   \n",
       "848            11.587349              258.700000                 NaN   \n",
       "861             2.639600              193.058141            3.326949   \n",
       "869             3.265932              275.000000            4.026492   \n",
       "998             1.722444              232.900000            2.326418   \n",
       "1031            7.969686               65.770066           10.262337   \n",
       "1035            6.554234              191.000000            8.164831   \n",
       "1063            1.879553              222.498226            2.692862   \n",
       "1066            3.551777              102.571616            4.861315   \n",
       "\n",
       "      rain_accumulation_9am  rain_duration_9am  relative_humidity_9am  \\\n",
       "16                    0.000           0.000000              48.990000   \n",
       "111                   0.000           0.000000              21.500000   \n",
       "177                   0.000           0.000000              29.260000   \n",
       "262                   0.000                NaN              17.990876   \n",
       "277                   0.000           0.000000              52.580000   \n",
       "334                     NaN        1480.000000              31.880000   \n",
       "358                   0.000           0.000000              13.880000   \n",
       "361                     NaN          40.364018              12.278715   \n",
       "381                     NaN           0.000000              20.640000   \n",
       "409                   0.000           0.000000              18.487385   \n",
       "517                   4.712       14842.000000              79.880000   \n",
       "519                   0.000           0.000000              72.550000   \n",
       "546                  14.627        7825.000000              87.870000   \n",
       "620                     NaN           0.000000              59.790000   \n",
       "625                   0.000           0.000000              86.840000   \n",
       "656                   0.000           0.000000              23.770000   \n",
       "670                   4.970       10560.000000              80.560000   \n",
       "672                   0.000           0.000000              16.753670   \n",
       "705                     NaN           0.000000              77.630000   \n",
       "731                   0.000           4.735034              34.807753   \n",
       "737                   0.000           0.000000              13.771311   \n",
       "788                   0.000                NaN               6.939692   \n",
       "840                   0.000           0.000000              11.911222   \n",
       "848                   3.171        2891.000000              91.000000   \n",
       "861                   0.000           0.000000              12.497839   \n",
       "869                   0.000          80.000000              85.270000   \n",
       "998                   0.000           0.000000              24.200000   \n",
       "1031                  0.000           0.000000              18.920805   \n",
       "1035                  0.000                NaN              56.860000   \n",
       "1063                  0.000           0.000000              14.972668   \n",
       "1066                    NaN           0.000000              11.657314   \n",
       "\n",
       "      relative_humidity_3pm  \n",
       "16                51.190000  \n",
       "111               29.690000  \n",
       "177               46.500000  \n",
       "262               16.461685  \n",
       "277               54.030000  \n",
       "334               32.900000  \n",
       "358               25.930000  \n",
       "361                7.618649  \n",
       "381               14.350000  \n",
       "409               20.356594  \n",
       "517               84.530000  \n",
       "519               74.390000  \n",
       "546               70.770000  \n",
       "620               77.750000  \n",
       "625               64.740000  \n",
       "656               51.630000  \n",
       "670               88.220000  \n",
       "672               17.804720  \n",
       "705               59.130000  \n",
       "731               18.418179  \n",
       "737               16.792455  \n",
       "788               18.793825  \n",
       "840               18.154358  \n",
       "848               90.780000  \n",
       "861               13.438518  \n",
       "869               90.260000  \n",
       "998               41.380000  \n",
       "1031              19.641841  \n",
       "1035              50.650000  \n",
       "1063              20.966267  \n",
       "1066              17.331823  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data.isnull().any(axis=1)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Data Cleaning Steps<br><br></p>\n",
    "\n",
    "We will not need to number for each row so we can clean it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "del data['number']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's drop null values using the *pandas dropna* function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1095\n"
     ]
    }
   ],
   "source": [
    "before_rows = data.shape[0]\n",
    "print(before_rows)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "data = data.dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1064\n"
     ]
    }
   ],
   "source": [
    "after_rows = data.shape[0]\n",
    "print(after_rows)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "How many rows dropped due to cleaning?<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "31"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "before_rows - after_rows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\">\n",
    "Convert to a Classification Task <br><br></p>\n",
    "Binarize the relative_humidity_3pm to 0 or 1.<br>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0       1\n",
      "1       0\n",
      "2       0\n",
      "3       0\n",
      "4       1\n",
      "5       1\n",
      "6       0\n",
      "7       1\n",
      "8       0\n",
      "9       1\n",
      "10      1\n",
      "11      1\n",
      "12      1\n",
      "13      1\n",
      "14      0\n",
      "15      0\n",
      "17      0\n",
      "18      1\n",
      "19      0\n",
      "20      0\n",
      "21      1\n",
      "22      0\n",
      "23      1\n",
      "24      0\n",
      "25      1\n",
      "26      1\n",
      "27      1\n",
      "28      1\n",
      "29      1\n",
      "30      1\n",
      "       ..\n",
      "1064    1\n",
      "1065    1\n",
      "1067    1\n",
      "1068    1\n",
      "1069    1\n",
      "1070    1\n",
      "1071    1\n",
      "1072    0\n",
      "1073    1\n",
      "1074    1\n",
      "1075    0\n",
      "1076    0\n",
      "1077    1\n",
      "1078    0\n",
      "1079    1\n",
      "1080    0\n",
      "1081    0\n",
      "1082    1\n",
      "1083    1\n",
      "1084    1\n",
      "1085    1\n",
      "1086    1\n",
      "1087    1\n",
      "1088    1\n",
      "1089    1\n",
      "1090    1\n",
      "1091    1\n",
      "1092    1\n",
      "1093    1\n",
      "1094    0\n",
      "Name: high_humidity_label, dtype: int64\n"
     ]
    }
   ],
   "source": [
    "clean_data = data.copy()\n",
    "clean_data['high_humidity_label'] = (clean_data['relative_humidity_3pm'] > 24.99)*1\n",
    "print(clean_data['high_humidity_label'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Target is stored in 'y'.\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "y=clean_data[['high_humidity_label']].copy()\n",
    "#y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    36.160000\n",
       "1    19.426597\n",
       "2    14.460000\n",
       "3    12.742547\n",
       "4    76.740000\n",
       "Name: relative_humidity_3pm, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clean_data['relative_humidity_3pm'].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>high_humidity_label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   high_humidity_label\n",
       "0                    1\n",
       "1                    0\n",
       "2                    0\n",
       "3                    0\n",
       "4                    1"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Use 9am Sensor Signals as Features to Predict Humidity at 3pm\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "morning_features = ['air_pressure_9am','air_temp_9am','avg_wind_direction_9am','avg_wind_speed_9am',\n",
    "        'max_wind_direction_9am','max_wind_speed_9am','rain_accumulation_9am',\n",
    "        'rain_duration_9am']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "X = clean_data[morning_features].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['air_pressure_9am', 'air_temp_9am', 'avg_wind_direction_9am',\n",
       "       'avg_wind_speed_9am', 'max_wind_direction_9am', 'max_wind_speed_9am',\n",
       "       'rain_accumulation_9am', 'rain_duration_9am'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['high_humidity_label'], dtype='object')"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Perform Test and Train split\n",
    "\n",
    "<br><br></p>\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## REMINDER: Training Phase\n",
    "\n",
    "In the **training phase**, the learning algorithm uses the training data to adjust the model’s parameters to minimize errors.  At the end of the training phase, you get the trained model.\n",
    "\n",
    "<img src=\"TrainingVSTesting.jpg\" align=\"middle\" style=\"width:550px;height:360px;\"/>\n",
    "\n",
    "<BR>\n",
    "In the **testing phase**, the trained model is applied to test data.  Test data is separate from the training data, and is previously unseen by the model.  The model is then evaluated on how it performs on the test data.  The goal in building a classifier model is to have the model perform well on training as well as test data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=324)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#type(X_train)\n",
    "#type(X_test)\n",
    "#type(y_train)\n",
    "#type(y_test)\n",
    "#X_train.head()\n",
    "#y_train.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Fit on Train Set\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
       "            max_features=None, max_leaf_nodes=10, min_impurity_split=1e-07,\n",
       "            min_samples_leaf=1, min_samples_split=2,\n",
       "            min_weight_fraction_leaf=0.0, presort=False, random_state=0,\n",
       "            splitter='best')"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "humidity_classifier = DecisionTreeClassifier(max_leaf_nodes=10, random_state=0)\n",
    "humidity_classifier.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "sklearn.tree.tree.DecisionTreeClassifier"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(humidity_classifier)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Predict on Test Set \n",
    "\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "predictions = humidity_classifier.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 0, 1, 1, 1, 1, 0, 0, 0, 1])"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "predictions[:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "456     0\n",
       "845     0\n",
       "693     1\n",
       "259     1\n",
       "723     1\n",
       "224     1\n",
       "300     1\n",
       "442     0\n",
       "585     1\n",
       "1057    1\n",
       "Name: high_humidity_label, dtype: int64"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test['high_humidity_label'][:10]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p style=\"font-family: Arial; font-size:1.75em;color:purple; font-style:bold\"><br>\n",
    "\n",
    "Measure Accuracy of the Classifier\n",
    "<br><br></p>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.81534090909090906"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "accuracy_score(y_true = y_test, y_pred = predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}