outlier_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Outlier Analysis\n",
    "\n",
    "Please work in your section and suggest:\n",
    "- Why the data is missing?\n",
    "- Is there a pattern for missing data (by ZIP, day, year, etc)?\n",
    "- Is it important?\n",
    "- What are the next steps?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Drop these columns:\n",
    "\n",
    "[\"Categorical_Snow_surface\",\n",
    "\"Composite_reflectivity_entire_atmosphere\",\n",
    "\"Graupel_snow_pellets_hybrid\",\n",
    "\"Graupel_snow_pellets_isobaric\",\n",
    "\"Snow_mixing_ratio_hybrid\",\n",
    "\"Snow_mixing_ratio_isobaric\"\n",
    "'Geopotential_height_potential_vorticity_surface',\n",
    "'u_component_of_wind_potential_vorticity_surface',\n",
    "'v_component_of_wind_potential_vorticity_surface',\n",
    "'Categorical_Ice_Pellets_surface',\n",
    "'Ice_water_mixing_ratio_hybrid',\n",
    "'Ice_water_mixing_ratio_isobaric',\n",
    "'Precipitation_rate_surface',\n",
    "'Vertical_velocity_geometric_isobaric',\n",
    "'Ice_growth_rate_altitude_above_msl',\n",
    "Land_sea_coverage_nearest_neighbor_land1sea0_surface]\n",
    "\n",
    "\n",
    "Spatial Inputation:\n",
    "[Snow_depth_surface,\n",
    "'Haines_Index_surface',\n",
    "'u_component_of_wind_altitude_above_msl',\n",
    "'v_component_of_wind_altitude_above_msl']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wendy\n",
    "\n",
    "- Pressure_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- Soil_temperature_depth_below_surface_layer has 5544 (1.4%) missing values\n",
    "- Temperature_altitude_above_msl has 4321 (1.1%) missing values\n",
    "- Temperature_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- Vertical_Speed_Shear_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- Visibility_surface has 69098 (17.7%) missing values\n",
    "- Volumetric_Soil_Moisture_Content_depth_below_surface_layer has 5544 (1.4%) missing values\n",
    "- Categorical_Freezing_Rain_surface has 316016 (81.2%) missing values\n",
    "- Categorical_Rain_surface has 316016 (81.2%) missing values\n",
    "- Cloud_mixing_ratio_hybrid has 316016 (81.2%) missing values\n",
    "- Cloud_mixing_ratio_hybrid is highly skewed (γ1 = 27.69373963)\n",
    "- Cloud_mixing_ratio_hybrid has 72199 (18.5%) zeros\n",
    "- Rain_mixing_ratio_hybrid has 316016 (81.2%) missing values\n",
    "- Rain_mixing_ratio_hybrid has 70260 (18.0%) zeros\n",
    "- Rain_mixing_ratio_isobaric has 316016 (81.2%) missing values\n",
    "- Total_cloud_cover_isobaric has 316016 (81.2%) missing values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Ngan\n",
    "- Geopotential_height_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- Haines_Index_surface has 6726 (1.7%) missing values\n",
    "- u_component_of_wind_altitude_above_msl has 4321 (1.1%) missing values\n",
    "- u_component_of_wind_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- v_component_of_wind_altitude_above_msl has 4321 (1.1%) missing values\n",
    "- v_component_of_wind_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- Categorical_Ice_Pellets_surface has 316016 (81.2%) missing values\n",
    "- Ice_water_mixing_ratio_hybrid has 316016 (81.2%) missing values\n",
    "- Ice_water_mixing_ratio_hybrid is highly skewed (γ1 = 21.89943017)\n",
    "- Ice_water_mixing_ratio_hybrid has 72756 (18.7%) zeros\n",
    "- Ice_water_mixing_ratio_isobaric has 316016 (81.2%) missing values\n",
    "- Precipitation_rate_surface has 316016 (81.2%) missing values\n",
    "- Precipitation_rate_surface has 61001 (15.7%) zeros\n",
    "- Vertical_velocity_geometric_isobaric has 316016 (81.2%) missing values\n",
    "- Ice_growth_rate_altitude_above_msl has 369711 (95.0%) missing values\n",
    "- Ice_growth_rate_altitude_above_msl has 19572 (5.0%) zeros!\n",
    "\n",
    "We will drop variables with more than 80% of missing data. There are 9 of such including:\n",
    "- Geopotential_height_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- u_component_of_wind_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- v_component_of_wind_potential_vorticity_surface has 385709 (99.1%) missing values\n",
    "- Categorical_Ice_Pellets_surface has 316016 (81.2%) missing values\n",
    "- Ice_water_mixing_ratio_hybrid has 316016 (81.2%) missing values\n",
    "- Ice_water_mixing_ratio_isobaric has 316016 (81.2%) missing values\n",
    "- Precipitation_rate_surface has 316016 (81.2%) missing values\n",
    "- Vertical_velocity_geometric_isobaric has 316016 (81.2%) missing values\n",
    "- Ice_growth_rate_altitude_above_msl has 369711 (95.0%) missing values\n",
    "\n",
    "The remaining variables will be analyzed:\n",
    "- Haines_Index_surface has 6726 (1.7%) missing values:\n",
    "    - Observations:\n",
    "        - zipcode 7008 have all values missing\n",
    "        - zipcode 2722, 98421 and 98032 all have around 18% data missing\n",
    "    - Imputation method: spatial imputation\n",
    "        - Values seem to be the same for locations close to each other. Ex: zipcode 2722 has (lat = 41.5, long = -71) and have data very close to (lat = 41.5, long = -76).\n",
    "        - It was suggested to impute missing data based on the closest lat long data given same date and time. \n",
    "    - Summary Data:\n",
    "    \n",
    "No. zip5\tHaines_Index_surface    \n",
    "0\t7008\t1.000000\n",
    "1\t2722\t0.187225\n",
    "2\t98421\t0.184851\n",
    "3\t98032\t0.184679\n",
    "\n",
    "- u_component_of_wind_altitude_above_msl has 4321 (1.1%) missing values & - v_component_of_wind_altitude_above_msl has 4321 (1.1%) missing values:\n",
    "    - Observations:\n",
    "        - (u,v) are missing in pair. \n",
    "        - zipcode 89506 have all values missing\n",
    "    - Imputation method: spatial imputation\n",
    "        - Values seem to be the same for locations close to each other. Ex: (lat = 30, long = -98) and have data very close to (lat = 29.5, long = -98.5) on January 2017.\n",
    "        - It was suggested to impute missing data based on the closest lat long data given same date and time. \n",
    "    - Summary Data:\n",
    "\n",
    "No  zip5\tu_component_of_wind_altitude_above_msl\n",
    "0\t89506\t1.0\n",
    "\n",
    "Spatial vs Timewise Imputation\n",
    "![title](spatialVStimewise_imputation.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from IPython.display import Image\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the merged data:\n",
    "data = pd.read_hdf('data.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_key</th>\n",
       "      <th>zip5</th>\n",
       "      <th>impact_score</th>\n",
       "      <th>grid_lat</th>\n",
       "      <th>grid_lon</th>\n",
       "      <th>Date</th>\n",
       "      <th>Time</th>\n",
       "      <th>ForecastRange</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>...</th>\n",
       "      <th>Snow_mixing_ratio_hybrid</th>\n",
       "      <th>Snow_mixing_ratio_isobaric</th>\n",
       "      <th>Total_cloud_cover_isobaric</th>\n",
       "      <th>Vertical_velocity_geometric_isobaric</th>\n",
       "      <th>Ice_growth_rate_altitude_above_msl</th>\n",
       "      <th>density</th>\n",
       "      <th>state</th>\n",
       "      <th>Region</th>\n",
       "      <th>USR</th>\n",
       "      <th>Weekday</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>578</td>\n",
       "      <td>97</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>578</td>\n",
       "      <td>97</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>578</td>\n",
       "      <td>97</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>18</td>\n",
       "      <td>0</td>\n",
       "      <td>578</td>\n",
       "      <td>97</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>2722</td>\n",
       "      <td>16.868994</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>578</td>\n",
       "      <td>97</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 128 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    date_key  zip5  impact_score  grid_lat  grid_lon       Date  Time  \\\n",
       "0 2017-01-01  2722     20.268081      41.5     -71.0 2017-01-01     0   \n",
       "1 2017-01-01  2722     20.268081      41.5     -71.0 2017-01-01     6   \n",
       "2 2017-01-01  2722     20.268081      41.5     -71.0 2017-01-01    12   \n",
       "3 2017-01-01  2722     20.268081      41.5     -71.0 2017-01-01    18   \n",
       "4 2017-01-02  2722     16.868994      41.5     -71.0 2017-01-02     0   \n",
       "\n",
       "   ForecastRange    x   y  ...  Snow_mixing_ratio_hybrid  \\\n",
       "0              0  578  97  ...                       NaN   \n",
       "1              0  578  97  ...                       NaN   \n",
       "2              0  578  97  ...                       NaN   \n",
       "3              0  578  97  ...                       NaN   \n",
       "4              0  578  97  ...                       NaN   \n",
       "\n",
       "   Snow_mixing_ratio_isobaric  Total_cloud_cover_isobaric  \\\n",
       "0                         NaN                         NaN   \n",
       "1                         NaN                         NaN   \n",
       "2                         NaN                         NaN   \n",
       "3                         NaN                         NaN   \n",
       "4                         NaN                         NaN   \n",
       "\n",
       "   Vertical_velocity_geometric_isobaric  Ice_growth_rate_altitude_above_msl  \\\n",
       "0                                   NaN                                 NaN   \n",
       "1                                   NaN                                 NaN   \n",
       "2                                   NaN                                 NaN   \n",
       "3                                   NaN                                 NaN   \n",
       "4                                   NaN                                 NaN   \n",
       "\n",
       "   density  state                 Region    USR  Weekday  \n",
       "0      0.0     MA  New England Northeast  Rural        6  \n",
       "1      0.0     MA  New England Northeast  Rural        6  \n",
       "2      0.0     MA  New England Northeast  Rural        6  \n",
       "3      0.0     MA  New England Northeast  Rural        6  \n",
       "4      0.0     MA  New England Northeast  Rural        0  \n",
       "\n",
       "[5 rows x 128 columns]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data size before dropping high-data-misisng variables: (389331, 128)\n",
      "Data size AFTER dropping high-data-misisng variables: (389331, 119)\n"
     ]
    }
   ],
   "source": [
    "# Drop 9 variables with more than 80% missing data\n",
    "highMissingList = ['Geopotential_height_potential_vorticity_surface',\\\n",
    "                   'u_component_of_wind_potential_vorticity_surface',\\\n",
    "                   'v_component_of_wind_potential_vorticity_surface',\\\n",
    "                   'Categorical_Ice_Pellets_surface',\\\n",
    "                   'Ice_water_mixing_ratio_hybrid',\\\n",
    "                   'Ice_water_mixing_ratio_isobaric',\\\n",
    "                   'Precipitation_rate_surface',\\\n",
    "                   'Vertical_velocity_geometric_isobaric',\\\n",
    "                   'Ice_growth_rate_altitude_above_msl']\n",
    "print('Data size before dropping high-data-misisng variables:', np.shape(data))\n",
    "data = data.drop(highMissingList, axis = 1)\n",
    "print('Data size AFTER dropping high-data-misisng variables:', np.shape(data))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get list of interested features (those that have missing values less than 80%)\n",
    "# nanList = [line.rstrip('\\n') for line in open('nan_features_Ngan.txt')]\n",
    "nanList = ['Haines_Index_surface',\\\n",
    "            'u_component_of_wind_altitude_above_msl',\\\n",
    "            'v_component_of_wind_altitude_above_msl']\n",
    "\n",
    "# Append primary keys to the nanList\n",
    "nanList = ['date_key','grid_lat','grid_lon','Time','zip5','state','Region','USR','impact_score'] + nanList"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_key</th>\n",
       "      <th>grid_lat</th>\n",
       "      <th>grid_lon</th>\n",
       "      <th>Time</th>\n",
       "      <th>zip5</th>\n",
       "      <th>state</th>\n",
       "      <th>Region</th>\n",
       "      <th>USR</th>\n",
       "      <th>impact_score</th>\n",
       "      <th>Haines_Index_surface</th>\n",
       "      <th>u_component_of_wind_altitude_above_msl</th>\n",
       "      <th>v_component_of_wind_altitude_above_msl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>4.0</td>\n",
       "      <td>25.209999</td>\n",
       "      <td>14.56</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>6</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>3.0</td>\n",
       "      <td>23.070000</td>\n",
       "      <td>15.06</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>12</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>3.0</td>\n",
       "      <td>16.990000</td>\n",
       "      <td>0.85</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>18</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>4.0</td>\n",
       "      <td>15.710000</td>\n",
       "      <td>-5.89</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>16.868994</td>\n",
       "      <td>5.0</td>\n",
       "      <td>9.880000</td>\n",
       "      <td>-8.27</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    date_key  grid_lat  grid_lon  Time  zip5 state                 Region  \\\n",
       "0 2017-01-01      41.5     -71.0     0  2722    MA  New England Northeast   \n",
       "1 2017-01-01      41.5     -71.0     6  2722    MA  New England Northeast   \n",
       "2 2017-01-01      41.5     -71.0    12  2722    MA  New England Northeast   \n",
       "3 2017-01-01      41.5     -71.0    18  2722    MA  New England Northeast   \n",
       "4 2017-01-02      41.5     -71.0     0  2722    MA  New England Northeast   \n",
       "\n",
       "     USR  impact_score  Haines_Index_surface  \\\n",
       "0  Rural     20.268081                   4.0   \n",
       "1  Rural     20.268081                   3.0   \n",
       "2  Rural     20.268081                   3.0   \n",
       "3  Rural     20.268081                   4.0   \n",
       "4  Rural     16.868994                   5.0   \n",
       "\n",
       "   u_component_of_wind_altitude_above_msl  \\\n",
       "0                               25.209999   \n",
       "1                               23.070000   \n",
       "2                               16.990000   \n",
       "3                               15.710000   \n",
       "4                                9.880000   \n",
       "\n",
       "   v_component_of_wind_altitude_above_msl  \n",
       "0                                   14.56  \n",
       "1                                   15.06  \n",
       "2                                    0.85  \n",
       "3                                   -5.89  \n",
       "4                                   -8.27  "
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nanData = data[nanList]\n",
    "nanData.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\MyPC\\Anaconda3\\envs\\weather\\lib\\site-packages\\ipykernel_launcher.py:1: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  \"\"\"Entry point for launching an IPython kernel.\n"
     ]
    }
   ],
   "source": [
    "# Extract year, month and date from date_key\n",
    "nanData['year'] = nanData['date_key'].apply(lambda x: x.strftime('%Y'))\n",
    "nanData['month'] = nanData['date_key'].apply(lambda x: x.strftime('%B'))\n",
    "nanData['date'] = nanData['date_key'].apply(lambda x: x.strftime('%d'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_key</th>\n",
       "      <th>grid_lat</th>\n",
       "      <th>grid_lon</th>\n",
       "      <th>Time</th>\n",
       "      <th>zip5</th>\n",
       "      <th>state</th>\n",
       "      <th>Region</th>\n",
       "      <th>USR</th>\n",
       "      <th>impact_score</th>\n",
       "      <th>Haines_Index_surface</th>\n",
       "      <th>u_component_of_wind_altitude_above_msl</th>\n",
       "      <th>v_component_of_wind_altitude_above_msl</th>\n",
       "      <th>year</th>\n",
       "      <th>month</th>\n",
       "      <th>date</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>4.0</td>\n",
       "      <td>25.209999</td>\n",
       "      <td>14.56</td>\n",
       "      <td>2017</td>\n",
       "      <td>January</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>6</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>3.0</td>\n",
       "      <td>23.070000</td>\n",
       "      <td>15.06</td>\n",
       "      <td>2017</td>\n",
       "      <td>January</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>12</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>3.0</td>\n",
       "      <td>16.990000</td>\n",
       "      <td>0.85</td>\n",
       "      <td>2017</td>\n",
       "      <td>January</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>18</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>4.0</td>\n",
       "      <td>15.710000</td>\n",
       "      <td>-5.89</td>\n",
       "      <td>2017</td>\n",
       "      <td>January</td>\n",
       "      <td>01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>2722</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>Rural</td>\n",
       "      <td>16.868994</td>\n",
       "      <td>5.0</td>\n",
       "      <td>9.880000</td>\n",
       "      <td>-8.27</td>\n",
       "      <td>2017</td>\n",
       "      <td>January</td>\n",
       "      <td>02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    date_key  grid_lat  grid_lon  Time  zip5 state                 Region  \\\n",
       "0 2017-01-01      41.5     -71.0     0  2722    MA  New England Northeast   \n",
       "1 2017-01-01      41.5     -71.0     6  2722    MA  New England Northeast   \n",
       "2 2017-01-01      41.5     -71.0    12  2722    MA  New England Northeast   \n",
       "3 2017-01-01      41.5     -71.0    18  2722    MA  New England Northeast   \n",
       "4 2017-01-02      41.5     -71.0     0  2722    MA  New England Northeast   \n",
       "\n",
       "     USR  impact_score  Haines_Index_surface  \\\n",
       "0  Rural     20.268081                   4.0   \n",
       "1  Rural     20.268081                   3.0   \n",
       "2  Rural     20.268081                   3.0   \n",
       "3  Rural     20.268081                   4.0   \n",
       "4  Rural     16.868994                   5.0   \n",
       "\n",
       "   u_component_of_wind_altitude_above_msl  \\\n",
       "0                               25.209999   \n",
       "1                               23.070000   \n",
       "2                               16.990000   \n",
       "3                               15.710000   \n",
       "4                                9.880000   \n",
       "\n",
       "   v_component_of_wind_altitude_above_msl  year    month date  \n",
       "0                                   14.56  2017  January   01  \n",
       "1                                   15.06  2017  January   01  \n",
       "2                                    0.85  2017  January   01  \n",
       "3                                   -5.89  2017  January   01  \n",
       "4                                   -8.27  2017  January   02  "
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nanData.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "#nanData.to_csv('nanData.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Get percentage of missing data group by zipcode\n",
    "missingPerZip = (1-nanData.set_index(\"zip5\").notnull().groupby(level=0).mean()).reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>zip5</th>\n",
       "      <th>u_component_of_wind_altitude_above_msl</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>89506</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "    zip5  u_component_of_wind_altitude_above_msl\n",
       "0  89506                                     1.0"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Get list of zipcode and their corresponding data-missing percentage in descending order.\n",
    "name = 'Haines_Index_surface'\n",
    "name = 'u_component_of_wind_altitude_above_msl'\n",
    "\n",
    "sort_missingPerZip = missingPerZip.sort_values([name], ascending = [0]).head(10)[['zip5',name]].reset_index(drop = True)\n",
    "sort_missingPerZip[sort_missingPerZip[name] != 0]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAEbCAYAAAB0jZg1AAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAE88SURBVHhe7b2Jfxy1/f//+5P6gdJ+e31LW6AchRZoP8W0hVL6bSGUtuRwnHjj3CEhcQ4CCSEJJuSAcCTmSMhJQpyb3Ledg9zkcOzEiY/YjvXzazxvrVae2Z3ZnfUefj0fj/djdyWNpJE0mtdqNNL/pwghhBBCSE6hICOEEEIIyTEUZIQQQgghOYaCjBBCCCEkx1CQEUIIIYTkGAoyQgghhJAcQ0FGCCGEEJJjKMgIIYQQQnIMBRkhhBBCSI6hICOEEEIIyTEUZIQQQgghOYaCjBBCCCEkx1CQEUIIIYTkGAoyQgghhJAcQ0FGCCGEEJJjKMgIIRmzZ88eVVlZqZ577jlVUlLifM6aNUtduXLFDdEDwo0bN84JAxswYIBasmSJ6xsHxyE+CVdTU+P69CDp+FkyJH18EkJIvkBBRgjJiLq6ul6CSCwWi7mhesL5Camqqio3lHIEmh0uSkGGPCGMmTdCCMk1FGSEkIzAqNfAgQO1aMKnlziSES+MikGcYRRMRqsgsAQIJYStrq7WcdiCzAsRWshLMijICCH5CAUZISRyRPTABBnVMkfDVq9ercNB2JmYwi6VIDPDQsglw0uQIR/4bY68QSyaeUK+4Y4RPHyXsBCAdt4JISQsFGSEkMgRsSIjXxgNE6FjiivzcactusIIMq+RNj+8BJm42WbGJ4LMyxDOni9HCCFhoCAjhEQKRpBEqMhoWDJx5eceVJCZos4cffPDS5BhVA2jZCKqMOJlp20KMkkn7GNVQgjxg4KMEBIZEDIyOmbO5Uomrvzckx1jYr6NCXGWCi9BJm91Yn6bxCUmaYsgM48DEi7Vo1JCCEkGBRkhJBJsMWY+wvN7NOn3KBMEEWQ4XtKEoAqClyDzEmJiQQVZkNE5Qgjxg4KMEJIxycSY4CVczEn99uhWEEEmIilZGBtbkPmlY7tRkBFCsgkFGSEkIyC+IMIgSvzEGBAhhNEohIGZbjZ+QsnE6/FoMpCmjIZ5CTJ5W9KcB0dBRgjpCyjICCEZYYoXLxOhYgof2zBSJohI8zKkJZgT6lPN3zIn6YtJXBBpIuy8jIKMENIXUJARQjLCFEZeZgoVCC8ZTYPhuynGQFBBJvFATKXCFGQIj22dTOBvxgd/yYctyOwtl0TMmXkjhJCwUJARQgghhOQYCjJCCCGEkBxDQUYIIYQQkmMoyAghhBBCcgwFGSGEEEJIjqEgI4QQQgjJMRRkhBBCCCE5hoKMEEIIISTHUJARQgghhOQYCjJCCCGEkBxDQUYIyXuwPZNsCu61EXm+I/t92tsuRUE2484E2VJKNmzva/K1XHINyyV/oSDLQ5LtDSj76hHSX6irq+t1HWQb2dfS3vMyXfw2Jo+CMHHjfBAW55dtkA4sV31WNsu8kGG55C8UZHmIXDBeFnXnJv+WgmzQTIiJuWF3NkdB0OYlnStXrjgCLdsUuiDD6Afc7FEQL0HmFzZTECeMgiy/YLnkLxRkeYgpyLJNX6ZFigtTKGXzpit/Ggq5jWbzJugVN74HTS9M2DAgThgFWX7BcslfKMjykCAiCSMFlZWVep4GPvHvF+4C4pHOVsLgGAkj/4xtM/3sf80SRkZEJK/4RNz2MXCXuT/iZ45w4DvyBfMbZfEbxZNzg39YVq9erUdB7LKTsjXzjbTsG4sci3xLXsy4cO74LcebdWMea9aDXYcA8UheYfhun7OkjzxKPcAQtx2fee4Sn3luUqdIwzwHhJM6MkWSaWFvvsiL5B2GMjfLwPQTg1sQ5NE/8m8iceLcBJyXxA+kTsxylri86tsEeTfrAN8RBt+D5l2Q8pE6gCFvUg9A6kviNsOKiZ/UG+IAycJKGUhYQcKZeUA+zTl++C3hzDaBYyReCWu35SCELRd8l/zZ7R1IviQ+fOK39FUSF9xNcBzcYQLyluz6CoqkhzRwDvI7SP9i5kGOEez2QvIHCrI8RC4YmBe48MwL3jTzIvPyh0kHa3aMpgHpAOyLVsJIB2N2VOInx5g3JdMQVjoPxCPufp2WX3lIHuEfBj8xgRt4srKFoaMTvPzFzPIQ86obr3Bm5+lXhjAznJSFl5npBpmfaJa3bVJ3UQiyZHlBHQCvNmqeTzLMuWdSb8i7uEkaQM5Z3LzalhznZWa4ZO0naN4Fv3pFPQiSd4k7WdsLE1bSlt+ChJO6RtmKm5dJOAgLr/RgZvkFQfJmm1e5eBnCidhKli8JZwovs42L0JZ2E+X8X684xLzyK/WEdGw/mKRvtwGSP1CQ5SHJOhIgN0N0AtKpmBeh+a/O7ATMm6hgpmUStDM2j8d3YHZg6DgkLNzlXyo6LnFDGBiO8SJVHiXdoEhnhps9btAwCB/kySwjr3ybN3EJJ/WAeCQcDHEB6bRhgvw2jzXFBzBvAnKOdjj8BlIWODcRH6aYE+TcJW9mfAgPvOrUvNFIueDTdguDXQ/ATFvOw3QLi9SHiFdbPKDsgZSfhPNqW3IM6gx1gzzjO9zkOjHLRI5FmnKuEi4oKHccL+XjJQykfMy4Jf92emHC+rnDDSbpSziUtVdblnBmO7PrG8eGIUy5wKS9m/Uv9SP5kvwDsx6lTdhtCYibxB/k+goKjoGF7V/kvHEcwHFIO1l7IfkBBVkeYnYktgHpAP1MLjx0UugMpJMwTTDTMgnaGdsXv5DsHGDwD0qqPIaJy+xoUT42EifKzQSdnxwnyG8pC+BVbmaa0uF7HWveLOAu5436M/G6+XiVhZ2u+dvLJM+SrnkOQMKJmDbjk3wExTzWrgdpr3Iukh9YWORmJe1Tfksaci4Sv5yHV3naYYDkTQSF/LbrTNztMk2F3EzNG7GY5MMrbsm/nV6YsH7ucINJ+vJbBILgF87PwpBuuQBbRMtxdv5FSEk4aTtS1+Z1GOb6CoocJ+cDEIcdl5ku8mH+eUI49Csm6bZFkn0oyPIQuWBgXshF6WfoKHBhevmJCX5peV34QMKm6vTMeL3M7vySkSqP8A+K2Xl54Ren13HyO0yHKWG9jrXD+ZUtsI/3yrcdn/nby0SE+qUr4SQNO/4wmMfa2Oci+fEKmwozHVwTcjMWgS0jB/huiiiv8sRvmHmudt7kt112fu6p8BIcYpIPr7gl/0Hy4RfWzx1uMEnf/i34hfMyW8CmIt1yAfZ52ccJ9vGmAMN3aUMi9nG8+HuZ/ScvFXKcmS8778BMV8Kao2Yw5BHtH6TbFkn2oSDLQ+SCgXkhF2WyIXCJAx2XXIjmhSv4pSUXtH3RSli58P0ubjP9TDFHjkykHJBWUMwysEdmgMSZrREyCet1rHmeyJuUoX2zMm8McrxXWdjpmr+lTXjhV6dyrKRhxx8G81i7HmT0StKR/MDSQeKTOsR5yR8W+En85vXkVZ74DTPP1c6b/LbrTNztMk2GX/nabl5xS/7t9MKEDdoHyG/7T5ZfOBmVTJdMygWIKJdrXI6z82+PkAERgghrintg5ivZ9RUUics8R6+68isP5AFlIO1f2nc6bZH0DRRkeYhcMDAvzMcucgFiCB8dnQggiQO/4SdD/Ha8ZloIIx2JuCMN3DDhJx0UTNL1u7hNcWF2dHBHRyY3YaSHNCQdL8wORzpzM9/4HhSchxyH85GyQZkin+Y/SzlH5FE6YlOo2eFA0A5Tfss/V+RBOniUBTDLUM7RrAcJByRdsyzsdM1zR1vAb4ByR5xST351KsdKGmb8yCvikzhTYeZF6gFI2jBpD6ZbOkh5SR1K/uU3yhGfplDwKk/8hkkdAjtvXu0ebpKGXabJMMtXygJxipvkQ/Jgxi35R5sC9nUdJqxcm6gjKUuYpC/t1uxrkoXDp6SBT6SD8EEJWy4wqQvzcZ64mfmXfJlpSDgg/YO0HZgcY7bpZNdXUCQuOR8gdWXWn5lXfMd5wyR96fflGCkXMw6SH1CQ5SFmR+IFOgDp4G2Tm7R5kXqZgM7Cy8/veElXOolkF7d0dF4mHandmXiBjsXrfMUNeQiDKbpMQ4eZqmwl30DczXwH6TCB/PYy83ySlWEqAeGVrt+5w+SG4VenEs5Mwy4rsyxSkSwv8m8eSH5g6WDehGF+ZSE3MOBVnhLOPEevvJk3azEpJ7tMk+HX7sUkH171hbo0w4pfmLCI33QXkzxJ+nb5itnhTLFqWxhBFrZcvMLCTeo7Wb5w/Zntwu4v4W8S5PoKihwn5wNQN3Az68+sJ3w326Rp0l94tQGSH1CQ5SHSQaLT8AMdA25a0tngE7/l3xrABSg3B3wiXglvgk5E3M2LFMeLOzoedFzyz1eEiVzcXh0qOjLEbd6gEM7sYEQAwUyxYwM/81yQrpRT2I4O4HiJD2kjn9LxIk922SLfdv7gBzPdpXzM8oC/HVZ+I+/SySId5MsEeTLzCkN41IWJxGGWhVe6AGFMoYfv0lkDvzqV8jDTsNuY2f6CgOPtvNhlgPTgh/TTAWVo1qVg3sjsm5NXeUpYuyzhZsYLf7NO0bYkLTudVCAuKR+JS+KW68irvnDO0hZhUqZhwoIgfQDAMWY45E3Cmde7eTxMrmW59oISplxQR/CX/CGcmXdg5heGfKEP8MqXpCtx28DNDIPv5vUVFDnezKvkEZ8C/M2wyLPZf0kZC1IuZhwkP6AgIyQHSAdq3qwIIYT0XyjISFEg//qSGf4x5guSp2ITZH1RDygzr3hts0dB8oVCz3+26A/lIqN4yYwjV/0XCjJSFBSaIJPHCcV206UgSw0FmTcUZD1GQdZ/oSAjhBBCCMkxFGSEEEIIITmGgowQQgghJMdQkBFCCCGE5BgKMkIIIYSQHENBRgghhBCSYyjICCGEEEJyDAUZIYQQQkiOoSAjhBBCCMkxFGSEEEIIITmGgowQQgghJMdQkBFCCCGE5BgKMkIIIYSQHENBRgghhBCSY0ILspqaGlVSUuLYgAEDXFel6urqtDuMEEIIIYQEI7Qgi8Vi7ree79XV1b2+V1VVqcrKSuc7IYQQQghJTkaPLDFChhEzGR0T8NscPSOEEEIIIf5k9MgSI2HAS4DxsSUhhBBCSDAyGiHDY0qIsigEWX19vaqtraXRaDQajUYraIOmCUtGggyjZRBlfGRJCCGEEJI+GU3qx8R9mbwPAcZJ/YQQQggh4QktyCC0ZA6ZKc78lsMghBBCCCHJyeiRJSGEEEIIyRwKMkIIIYSQHENBRgghhBCSYyjICCGEEEJyDAUZIYQQQkiOoSAjhBBCCMkxFGSEEEIIITmGgowQQgghJMdQkBFCCCGE5BgKMkIIIYSQHENBRgghhBCSYyjICCGEEEJyDAUZIYQQQkiOCS3IampqVElJiWMDBgxwXZWKxWLaHYZwhBBCCCEkNaEFGYSXgO9VVVX6O0UYIYQQQkh4MnpkCTFGQUYIIYQQkhkZCTI8shQRZj+yJIQQQgghwUhbkGFkrLKy0v2VCNz9/AghhJAg3D64RjVO/526VnGX84nfhBQraQmy6urqhLlkNhg1S+bvRX19vaqtraXRaDQaTZ1es0BdG3m3ujbif+JWcbfj7hWeRssng6YJS2hBBjFmvl3pBUbHZG4ZIYQQEhZnZMwUY67BnZBiJLQgM+eJwUScmW5hR8cIIYQQEy8xJkZIMZLRpH5CCCEkG/iOkE171A1BSHFBQUYIISTvwAR+TOa3BRkn9pNihYKMEEJI3rFx71V1PPZLLcTqu+1k7F7HnZBihIKMEEJI3vHKhDUJI2Nigyd86YYgpLigICOEEJJ3jCudokXYt7Gf6e+vlk5yQxBSXFCQEUIIyTtWjx6gRdia4X/R39eN+ocbgpDigoKMEEJI3nFpwgNahI0sna6/Xx53rxuCkOKCgowQQkhe0XmpTgswPK68b+BmVRf7hXbrOH/IDUlI8UBBRgghJK9o3bpYi68vhj2vfjt8m1o27AXt1rrpHTckIcUDBRkhhJC8onHhf7X4mlQ6UY1+96gaUzpVuzW9+6IbkpDigYKMEEJIXnFl9E+0+BpVuVptPXRN/XHQF9oNm4yrznY3NCHFAQUZIYSQvKHj9B4tvI6U36+Wb7qoOjq71INDtqg95b/Rfu11W9wjCCkOKMgIIYTkDec/fUOLrg+HvaSamjsc9xHzj6j3ygZqv+ZV0xx3QoqF0IKspqZGlZSUODZgwADXVam6ujrtDiOEEELCcuy1+JpjH86KT97/bMt3qnTIW9rvxuynXR9CioPQgiwWi7nfer5XVVXp79XV1c53uFVWVjrfCSGEkCB0tbdqwQXbteeE66PUd9da1cMDv0rwv3OT+1qS4iGjR5YQXjAZHRPw2xw9I4QQQlKxZ1V84v7+kb91XeO8ULlXbRr+Rx3m9v6Vrg8hhU9GggyiC48wvQQYH1sSQggJw9op5Vps7Zw9ynWNM3/FaTVr6Agd5tby3mEIKVTSFmTmY0kKMkIIIZlwpbFN7Sp/TIutizvXuT5xDpy8oQYMXqjDNFY+4voQUvikJcgwV8ycSxbFI8v6+npVW1tLo9FotH5o8xZv1EILVnf4oGe435VvVWdj8XXKTuxc7xmORsulQdOEJbQggxjzEltw46R+Qggh6fDG+NlaZJ2e8azr2ptJS+rUymHP6bCt2953fQgpbEILMoyEmSbizG85DEIIISQZ+05cd9YcE5HV/NUc16c3X+25ol4b+qoOe3PJQNeHkMImo0n9hcbtg2tU4/TfqWsVdzmf+E0IISS3VC49rg6X369FFlbr9+NmS4d6ZvAnOuy1sT91fQgpbPqNIIP4ujby7vhFDKu4m6KMEEJyyO32O+r5suW6X64f8xPXx5/Bsw52C7gH9DHJBBwhhUK/EWTOyJh78ZoGd0IIIblh5fZLalLpRN0n31z8iuvjz9Kvzqulw/6lj2n56i3Xh5DCpd8IMrlwvYwQQkhuGPTmQfXFsOd1f9y6dbHr48+pi82qovR1fcyNec+7PoQULv1+hAzWum2JG4oQQkhf8e13zeq+gZvVt7Gf6f6481Kd65uclyYkLpPR1XbL9SGkMOlfc8gq7kq4gK+N+J7+fvOj4bygCSGkD5n3xWn1z8GLdT/cOPlB1yc1byw/qXaUx/9o3z6ywfUhpDDpN4IMYFVnuXgbJv5KNU6PrwoNuz79t6r95A43NCGEkGxSMuYb9frQUboPvvVJfMHxVOw42qDml5XqY5tXTHJ9CClM+o0gu3OzXl+4sK72Nse9eeWUBHdYy9fzHT9CCCHZYcvBa87jyq+GP6373rY9n7q+qenqtqHD3tHHXp32RI8HIQVKP3pkuVpfuDfmJ04AvX3gS9UwPj6HAYbFBiHiCCGERM+YBcfUAwNr1OUR9+h+907jRdc3GKOrDlvHX3B9CCk8+o0gu/XZBH3Rtqyf7brGuXPtnGp690UdBtb42gOq/ehGNwQhhJAoaGhqd0bHXhkyX/e312c+6foGZ8W2S2r98D/pONp292zfR0gh0m8E2fXXn9AXbfupna5rb1rWvaHDicGNEEJINHy08bwjyOaVlel+tvmLia5vcK40tqkZQ0frOK4vHeb6EFJ49AtBduf6d/qCvTbqB0p13XF9vMGoGEbH9DHdhtEzjKIRQgjJjBcq9zqCbFt5/I/y7cPrXd9wjJ/yqY7j8vhfua6EFB79QpC17f1cX7AQVkHA/DHMI5PjYJhnhvlmhBBC0uPQqRuOGPvtoDUJ/WtXyw03RDjeXXVGnYz9XMfTceGI60NIYdEvBNmt5fHXqlu+nue6BgNvXMqxYngzkxBCSHhmfHzCEWTDS9/UfeqNt591fcNz+Nsm9emw/6fjat28wPUhpLBIS5BVV1erkpISVVNT47ooFYvFHDcx0y/XNE59VF+sHWf2ua7BwdpkWKNM4oBdn/G4aqx82FlsFrsAcJNyQghJzp07XeqxYdscQbak7D+6P21Zm9k83Rkjp+m4Ls55wXUlpLAILciqqqpUZWWlI8BsQZZPIkzorD+jL9SGcT91XcODVfyxmr/E1csq7qYoI4SQJKz55rIjxmBHRsYX6m4/ud0NkR5vLdyq47pacU/KecKE5CNpP7IsFEHWtmuZvlCbFv3HdU0f7HtZ78ZnG0bKCCGEeDP0rUOOGCsZ9Fm87wzwolUqvt53Ve0rf1jH2X4iM4FHSC6IVJCZjyzzBXNUq3Xze65rZkh8XkYIIaQ356606NGxcaXxHVKaFrzkhkiflrZOtbjsvzrO+s+nuT6EFA6RCTITPNKEhaG+vl7V1tZGbldf/ZW+SE9u/dIzTFi7/Fr8n5hpV8f+1DN8X9iGhfPUkREPqvoRdzmf+O0VLpWdXrOg5/wq7nI+8dsrHI1Go4WxysV7tCDbOD6+CPfZZZM9w4e1Nye9reM8MfFJzzA0Wl8ZNE1YsiLI4A7/XNN56bi+QBte/YXrmjmYK3Zt5N067h77nvPZuukdN1TfsffzT9TVbiFm5qd+xN2Oexg8z4tz4wghEfCX8bu0ILs0Ib7OY8f5Q26IzFi+7kRC39V1q8H1IaQwyNoIGSb/55rW7R/oi/PmB4Nd12iASGmc5r69OfanOh0Y5pn1JUcrHkpIX8xvrpufcW4cISQb7DjSoMXYf0fG5/U2TPylGyJzzlxqUZuH/17H3XZwtetDSGEQWpBBbJlzxWQkzMst10CEycUJcZZN8MKApOV0BrvCjU5lgplutowQQtJlwsJaLcjWz52p+5WbS0vdENHw/tj4mpPfLhjhuhJSGKQ9QlYI4DGlXJx4fJlV7nSqpqp/6vRgt/evcD2zx84Nm9Xl2D0J6aZrviNk0x51UyOEkHA0NXdoMQb7bv7Lum9p2/mRGyoaPl78hY77zLiHXFdCCoOiFWQdFw7rC7Nx8q8dt417r6q/vrpb3T9oi/OJ31GCtcqw4rSkC0t3f7YgrFm0RF0e8f2E9MQwuX9E6cyEjnDce8fUsTM33aN74z2HrPscOIeMEJImyzdd1H3Qf2YecNaDlP6l8+ppN1Q07KptVOdjPzLi/9b1IST/KVpBhiUu5KLE0hcQXw8M3pIgUCDMohZl2APz+pt/1GnD2uu2uL7RgH+cn017LSGNK93C7NsRPfu5YU7Z+gXvOALMPF+xZMIM4gvzOhAPRsxuzP6T60MIKWacebHTf+e8YR3l7iP/mr5f9z0bv1in+6zGaY+5IaJlbeyvOo3zaxe5roTkP0UryMw5XZjPhRExU5SIwT1q7jRcSNhqqWH0D1XHt9Gks//EDfX56Ph6O7ATYx5WN07sd0MkAuEVVpiZb6c2TnnQdSWEFCveo+OZv2F9tLt/kf4Gf4hvrH1Lx3+reowbKlqqX5+h0zj6+r9cV0Lyn6IVZAnD4vVnEoSIbR9tvKButXS4R0ZD5+WTzqNSyUPDhJ+rjvOHXd/0WPHVMfXV8Kd1nLCjlX9TXc2Nbgh/wgqzhnE/02ncabzouhJCihFnZMzoV8QyfcP6zeUndT8zaUmdanonvgn47f1fuqGi5et1O3UaF0amv10eIX1NUQoybCAuFyQ2Fgd+I2RiDw7ZoqZ/dEIdO+s/xyosEGAQYjov3QINQi0dFi5YrQ6WxwUe7HhV+LeIUgmzJWvP9cyvG16i07l9aK17NCGkGDH7FdvSQebr3jcwPk1k9+FLCfHeuRntdBHh2o3b6lgsviB40/Hon4IQkg2KUpC1fD1PX4y3lo9y3NBB3DeotwjxsiGzD6r1u684x2UKHlXikaXkB48y8UgzKGcvt6g5k+f2mrz/3Zdz3RDpkUyYweaVDdVpHXtvgnsUIaTYwB+u+gp7oWvXxocfYfKarwthtvvL+BuQN2Zld3u9dePjU1YOvz/ddSUkvylKQdb0bnxbjra9nztux8/fSuggnp2wW325/Tv1xbbvEiadmvb0mG/Ue6vPqqvXbztxpAsm9Ut+YJj0j8n/qcCGuW+Vj0449mrFPZFNtgV+wqxsyCyd5s6Rf3RDE0KKiZb18eu8t/XsPnLr83B/yPyeRnwyJv4nr3nVVDd0dli/aKFO6+Crf3ZdCclvik+Qdd1R10b9QF+MMv9p7hendccwYVGt42ay9/h19Wq3u9mBmAY/hEkXLH8heYJheQwsk+HHu1+eUR8Mi6/XAzs/8TeRbTNiA2Fmnu8fBq3Q6X434gduKEJIMdDVckPdXDIwoX9pnHS/apjsbmk0/v8m+IURZWY/Ytqu8sd0fO21m9zQ2aHu2OmE/He1t7o+hOQvRSfI2k/FJ3Ref/0J11WpP4+L76O29dA117U3GA3DqBhGx8zORAyjaRhV27DnivNPMMyaZlgo1uwksJAsFpQ1wZIWE+ft7DV5/8Jb/ww0eT8T7H+2R2L36/Q7LhxxQxFCChlMo7g+LS6OYBBnEGkmEGFmmFSi7NTFZjV87uGEPkTs94NWJsTVFwJpX0X8Tfe6jatcV0Lyl6ITZC3rZ+uL8NZnPR3I7tpG3TH8YeQOxy0ImEeG+WRmx+JnQdc0wxIckj/Y6Tdf1MKupFsEDhm5tNfk/cZlY92js4s99+PzYX/XeYh6RW1CSN9j7u8rhseWfgQVZYvWnE3oD20bNTS+FAXetOwLNr0+Uqe5c9ZI15WQ/KXoBNmN+c/ri/C2u7ls5dLjumN4Y3n4txzx5iXewMSbmGYnY9tDQ7aql2fsV0PnHFJjFhxTUz44rmZXn1ILVp1xltZYuf2S2rjvqqqrjr90AMPK0lhZ/0TsXnVlROLk2taavt2kHaLsf0fudM5n2tCxOh+3lrFDI6SQufXZuIS+BUsDBXmDOpko21PXqAZM29erL6yYf0Q9M77nqQT+cB6bO0wf37Lhbffo7LJvzZc6zcOjolmENluL5xICikqQdbW36QsQhonzXV1KPTZsm+4oDp1KHJYPA9Yqg7CSuDKxN4bG/715Wf3IaCfvh6GlrdPJ40tD4rsdXJ/5pOtLCCkksH3Qjbnx1eth+B1mWyFblN2onqBmLouvMSY28M2D6sjpJveoOFiVX47tOL3Hdc0ut1vb1FXjD+6lM2dcn/RAf5yNxXMJEdISZNXV1aqkpETV1NS4LkrV1dU5bmK5oL1us75Qrs96ynHbsOeq7iyenxRNR2DPtUrXTsfii9eadil2T9Ym7wflP6/vVw8O/DohX10tvTtaQkj+ghEwc5FsGEbK0sEWZe+X/Vv3ZQ8N2aI+/tp7OR/sVynHIC99yZ7xf9Fpb/8ws22UsrV4LiFCaEFWVVWlKisrVSwWSxBk+A2hBiRMX9O8Jj5PoXnlZMdtZNVR3Wlgsn4UeK2zc/+gzeqjDefV4W+b1DdHG5xHk3hEiRE1PLLEo0s8wsSjTDzSxKNN86K2LdfMX9HzVuqO8ngn1H58q+tLCMl3vJa0wByydPnuWqv6ekppQnwQZWO7+zT4+YH5pxL+5uJXXNe+Ydd78XtCzaR/u67h6Th/UMfjZYREQdqPLE1BJqNjAn4PGDDA/dV33JjzjL5Abh/ZoK7fak8QTVhkNSogysw5Eqt2XHJ9gnNx0qMJF7XYxYkPuyFyB0Qlzm1xWXyBxZaNfTP3gxASHPRF8mIQPmt2nOq1pAXeqsxkP12MfmEUDH0CRJgZt99Ef+Hm0riIa9262HXtGy4e2qXTPh77hero7HJ9ggEhdnNpz/pp9W48tjVW5r6/JsVBZILMFmB9/diyq7Up4SLB7+rNF7UYe+WNA27I/AFzD+wVsusr7sqLOQm32+845TamdKrOGzZsJ4TkDxBjFUPfcN7MxotBteW/VCdj8e3aYF5LWgQF88EwL0z6UbHtM8oS0kgmyhom/lKH67xU57r2HWcr4muq7d78jeuaHFOIJbfvqRtznnWPIiQzikaQYURMLhKMlAGIMOlAIM7yEYivxmk9I2WYi9C251PXJ/eg/J4ZHF+mo3HSr1wfQkg+MH3sHHW1W4jJNWrbrvmT1IZu0Xbw1A11qaFN3bnjP0Jkj7ThTUlThMHwRiXerAT2nDIvUYa5sOLfOPlB17Vv2T8tvsD2mtnJt1FKJsTg3vL1fN1fmyNmrTuWujEQkj5588iyvr5e1dbWpm3nF5fri+P8ByPVph2Jncnu/Uc9j6P5W+WiPU7ZnY79TJftiV1fe4al0Wh9b0fK44s3m3Y1dpeKlc5M6APFfj9iq/r7xB1qyBvfqPFVu9WbH+51PjEPNjFs4jzZGe/v7ZX+hYWJ4gW/Tf+zyyZrv4tV/07w6yvbuXimzsPGimc8w5zculJdnDdAhzMN7vC3jzn/fkU8XMXd6sTO9b3C0PqvQdOEJRJBBiDAcjmpH29VysWBty0xgV86EkzsJ+GRBXXXDf+zLtvb+1e6voSQXHCn6Ypq3b5UHZ+auJSFbdL/ZWpYfR+r8PuRbKSsacFL2j1Xo/+d187qPFwc8UP1p7HfOKOBINWIGPyTcWP2n3T4pndfcF0JSY/QggxCC6NhYhBmAOJM3Pp6Qj/WGzMvJKxHhiUupEPB0hckPJgA+8CgLeqtofHRx+YVr7m+hJC+QkQYtlsz+zq/ieZnxj2oPtxwXs2qPqVGv3vU2fLtqdE9Cz6HtSB4ijKffYX7GoivE7Ff6LKqK/+lerNspDo15xWdN9OCCDGh48y+hGNbNs5zfQgJT9ojZPkEVuSXCwIr9WPxV+lMsCgsFocl6TF41kE1ZMjb8fJ9mxNYCekL/ESYbfUV9m//F4MwhwxzyTCnDHPLRLQ9aiyebRreJA+KLcqujf1JT366LVfzxwDm2fkJV9PCCDETiDAzHog0QtKhKAQZ9qyUiwF7WWJ7JOlQsG0SSR88+n18UFzwYqVq/PPtzzgvYnD7FJIhXu0oqAjbOPwpNXnoBPV+9d6eeDJ8MQijSF5rK8qjvaD0EmXacvf2uL03sJiItM+HPa9eGLxYPTdxtzO95Z2Vp519jI+fv6XaOxL7OvvFBykfPK6UePEYk5B0KApBdv31J/TF0H5qp7OBuHQqmAdF0mfv8etOOR4sf1CXccfZ/a5v/wM3FW6fQjLFsx0leVsSJiLsfwetUH8a943acaTBjS0aIC4yXVsRXBsfX2bCNIjFXOCVFzEIMblX+JkItVFVRxwhZvrhN8qt88rJ7vr8vo63edU0N/Xs4ycSSeFR8ILszvWL8Qts1A/U1oPxrZL+PC74cDvx5k5Xl7Op+vJh8X/srduWuL79D2dEQ9qbYbm62ZDCxK8d2Y/WTk57Tosw6dfGL6xVN1s63JjyDzP/tuUCvwW4T435tRr5zlFHxEjZpmN43IvHvlsXzkmIv+3YJjcHvYlKROG43iObPSKRFB4FL8ja9n6uL4Cmd19UExbV6oY594vTbiiSCaVvHXJuClLONz8a7vr0P6QMvIyQoHi1HzE8rrxes0RNr9qecKOFVdfk53qKJr5/WqY96oboWzAamWoBbjyaxCPKdbuvOI8s0xVqK4Y9p9M4XH6/emb0ZueFCrxYAdHmzNlbfkrd7yGiPt54QR0906R21zWqzQfq1ZpvLjv1vWTdOTWv+172+icn1cQldd15O6KGzD6oXpq2r/vP8taEeMSQd1J4FLwgu7V8lL4Amr6aq35tNHRcYCRzFq05q/45eLEu5+vTf+v69C+wUbPdsYs1vHafG4qQ1PiKlsqHnUeReCRp3mBxU687d9M9Or/xfqyf2x1IkHY68+xMofZErLdAtu0Pg1ao07H4Zu4fDXvJM1xfGCk8Cl6QNU6ND0dvXvWVboxYUZpEw4GTPW+tXh4RnyNx59Y117d/4LVRc9y+pxpG/0h1Xj7hhiYkObeqx/RuR92iZfXCJboPE8PISqGRrgDKZ7wfD3bXz3J35MtdYuTtSW8l1Ouo0ukJx/SFPdktHknhUdCCrLP+jG70DeN+qsrmHNINculX591QJAoeHrpVbRn+pC7v9mNfuz7FDfYAtDdqbpx0v2qY/EDP75H3aPcbs59WXR233SMJ8abrdrNqnPgr3W5gRyseVHPGJ66s/+SIHWrjPs4FyicgyoK8+GAK7qvj7lWbtx1LucQI5uriMSQeR+KxJB5P4jElHlfisSUeX+IxJh5n4rEmHm/iMactEsXW7rri5oYUCgUtyNp2xfdZrK96OaExXm5oc0ORKCh7+7B6rywuTDBiVOx0fLtbXZ/2mD5nmL1Rc8fpPYn+S4e6PoR407wqvmH/4fIHjH4rfmMdMf+Iunqd4r5Q6WpvTeg7bn4wxPXxH2lLdyK+KRIfKYPYq9HxRv0mLskuBS3IMLlcGvz2BfF/lxgpI9GCf2gVpa/r8m5a0Le7MfQ1rds/0Ocq5idC8TgmIdzaN1wfQhLp/O5YQlvxepz1/vpzbmhSyLTX1iTUtbkBedCRtrA0NXeoFyr36rb0m7KtzkgaKQwKWpA1To4v+DducrVuhFE1bhLn0LdN6ulBceHRMOHnrk/xceuzcfo8nXMd91NnQn8yIMLMY4phzgyJnpuL49v1rB/+J91nmUaKB6xHpvuFkd931iuLGme+Hl4ScRcY/m7rCvX0mPhLIfh+4WqrG5rkMwUryDovHdcNvX78vbrx4S3L1tudbigSJZj7cCJ2ry73zsvRdy65pPPqt+rG3MQNm/Eb7ib4d4t/tfYaQvYmxXicSYhw+/C6hPbxwuBFut8SC7NVESkMsrkBufcbrXerU1996oyOSbvCqFlTHq9dR3ooWEFmPlI6MG2AbnhYh4xkh/K5h9Wq4c/qci+mUSCMgGEkTM4NhpEyG4gvv4UYMaEfE/vl+MbKR9SdJk7KJj1cn/l73Ta+eaM8oQ31tKP05xGR/CWbG5A3Vv4mIW4xjJRtP9KQ0L5KZ3MqT74TmSCLxWKqpKREW01NjeuTHW5+MFg3vjmjX9ONbuuh/rUcQ1+CN1ffHFqhyx371hUDXktaQPALshYR9rfzW4tIFmLE0hcN436m44n6HzEpTFo3Vek20TD+Z6p8ZuKbdlHOIyL5R5QbkHdePKpaNsxRN96Kj7x5GVj7zeWEdjb2vWOOO8lPIhVk2RZhJg2v/kI3vL8N/tBpbNjDkmQPTA59Zch8Xe4YDSok7EeNNTtO9VrS4sqU36gdq9f3rNZdddTZx87s0JKZcPvohoQ4b3023vXJHvY8Evwm+QE2DG8Y/X90ezj08dsJ7ea7a5zf0x/IZANyvPGN+WjXZzyu44DZW21pw0K87rzXT76+kNDeZnzM9RLzlYIUZB0XDuuGd27Mr3RDe2N5cc1pykeeKt+YcOF3dRTG8iIQYxVD31AHy3/d3YndpWrLf6lOxn6ecC6rhv9V/W7QmoTOK6jZCzG2bn4vIW78zhZ+80goyvIDc00q/Il5ceo+3W7mfJY4P5EUL2E3IMdbmvgzZ7681su6hde1Ctv9e/q7vBle9eWZhP4Kv0n+kbVHltnEvNmtiP1DN7JDp+LrQ5HsgPWR9pY/ossf/9wKgelj56ir3UJM8m3b22XDEjos2zCihv3tRlUdcUbYvMLYCzGiMzXTwMhZNnBGxox0xOBOcguuD7NOtn32qW4vj5RtVddvtbshSX8AS19IW3BGt4wR7a72FnX7wCpnOSd7Pqtp8EMYhHWOwei4uytCw+SHVOOkxEWHZe1EjIyZ/RVGzkh+kZVJ/ZWVlY5li6ZF/9GN7dXSnvljz0/iG219wUcbL6iPhg3Q5d+6eYHrk78cO3PTGRmTPJt2NXaXipXG17AT4YVHlti/DnPHMIfMBKNtQRdiNB9TYG5Z1NsrtddtTjgf20huaXrnH7ou8BbusxPij8AXrj7rhiL9iRtz4i9GxQ0jXf5/GDFKhj94GDVLhdfuIlikFn8OMIdM2h+Mq/nnF1kRZHh0iRGzMNTX16va2tpAVj/mx7qh/WXwMqdhTVuy1zMsLVpbu/lwtwiepMv/4rwXPcPlg62pOayGvtmzHo/k18sWfXFArdtyWB0+cswznmS298Ax9dcJ8Yn+D5duUatrDmn/4/u2qyuT3G2Wuu3ytCdU3ZG4f7p2es0CdWlmiROn3zwSpOt1LK1v7MzKOQn1sXjxl7qdPFG+Na32Rit8uzLpwYR2IWZfx1cmP6LOv1+hTtVUe8aTys59mLieIuzMZ6+rlyp36nYIW7b2oOfxtMwMmiYsWRshq6qqcn9Fi/kK8ZHY/bpRnb3c4oYg2eY/sY91HVx97SHXNX/AiNg495/gv4e86+zB6SdaLk582D0qfc5fbU26EGOU2yu17a5W12c9lRCft33PGZ0hOaKzQzVOeUjXx601M52XjqSNfMzHRf2WxOs00fDmJN6gxJuUUeC1nE/jsrFczT9PiUyQmfPHwo6OhaHl6/jrwx8Pe9FpUK+8ccD1JX3BqKqj6sKI+Ftjd27kx+v6phB7bNA6tbgs/mjby+rxJlJEE9+PnG5KuhBjZtsrdanWbUt67asp1rxikmrdukTPIzHFJyf29+DMs+nDt1DNnRsgzBatPqXbxrMTuPhrf8Z3zmdldv7cei14fXXWM+qV0Z/rNsnV/PODrIyQZZOmd1/UjWpM6VSnMVVvvuj6kr5g2aYLatPwP+p6uH14veuTG0whBhtfOrnXG5Swm0vLtGhBpxj1wrapFmIMu71SV1tz9x+Q+apx0n0Jx8Eaxv6kO76ZngvP3lo+WodrnPyg6mq96fr0TyC++vIt1M4rpxLSaty2LEGsr9zO9cb6M97tMbo/h37YW8LVj/mJGjP8Dd0uuZp/auylk6JeyLmwBFnXHXVt1A90g3pq0GdOQ+KbSn3LifO31DtlpboemtfMcH36FluI/WPw+2r98PhK+WJYRLjz6mn3qOySaiHGINsrQWRBbEF0mWFhEGcQaRBrJmZH8cKEzerShPv1MRBo/RnfEYlu92xg1jEeG2NpC2kPWPKCEIivbP459MPc4UbMfMP8N2XbsiY2Ch2Uh98uLVFRUIKs/dRO3YgOlD/kFAgW7yR9z/iKt3RdXH7redc1O5hiA59L1p5LEGIPDKxR8w2BKIYO7/bB1W4sfUeyhRjt7ZWujbxHP0bDY0c8fjTPQQyPK/HYEo8vbbw6ihGlMxOOz/a/73zGLAfboqb92NcJ8V86kLgiP29yJNfgbUt7+oPXGoxRi41CB/ces3zE4B4VBSXIWtbP1g3og7KXncLYsIcNJhdMnb9V18WVUT9yXaPHS2zcNzD+u6L0dXW4PP4Wo1jzmtedEdVckWwhRmd7pdE/6pVnL8MEfkzkT4ZfR7Fy9Ms6nv786PLS+N6PfMWaFr6sOi8dd0Nmjim2sSDs9I/iaz8NfPOgG4qQ3OK1NAameWDBbCycjWWC8KcOW8Vhyziv5X/6E3gaY/attkVFQQmyG/Of140HN+LHhm1TXb0HDEgfUF1zUR2LxRcg7PwuO3uk+YmNZwYvUyuGPafTF2ta8JLquHDEPTq3JFuIseG13iISJhPyb8z7W6BRLax7ZqZh2mOD1jpCTOLuj48uIX4vVfxQl0Hc4quZw1o2znWPSJ/WrYt1fNgq6dvjpxPqY8fRxDXqCMk1Xvv4itWPuNsRZWYbxlZyeCqFdRr9hJr9RKOQR9nsaTEoD9ntRUQr1qSMioIRZF3tbQmN5clBX6rKpdH9syXhOHWxuVsQ/U3XR9s3n7g+0eJ1EXw+7O8JbQGGuVVtu5e7R+UPfgsx2vk3DYu9pmLjvqvqvzMPJMRtG7ZzciYQG3H3p0eX9uPhK903GHyiDa0b/mftLob9BdtPprcfbldzo7NpuMSFzcQnLKrVdRGblx9/EgixwdIYfruY4Fox+xQ/E6HmtZNJIT76tIUYDPchu5wgWvd+Ht29ryAEGW4iDa/1rLSOEYRjsfucAtpd2+iGILlg7sj4WzsXF49wXaMl1ZZHsOYvJqqutvx9HDdk9qGECxujWr4Tzac96h7lzZc7Lql/GmsIpTIIwP761qX9AsXLQxYklM1/hy1RN+b3FvdOe2oPt64hjpHjr8/8vbONm5nWQW7rRvIYs/3btm5KTJWPr05oz2EtynlW2cQWYg8N3Kj+PvgDNap0urowMv6Hy7QoXw7Ke0Hm/MO3XhHGSEnl6LfcECRXvPP2cl0nZyc97rpGy9kJv0moezEI8xtzn1Md3+b/mk5NzR29FmI8teHTwK++46n8sk0XnfWrJA7T3lh2Ui37+oLvdk7f7D/X7x5d2kuMjCudosvDNIxcNW9emPD2NqxxyoPq9v6VbmzJ6Th3MOHY24fXOfFKGhgpIySf8fuDaNr1uX9TZ9e+r77aed55ZIkt5vymlHjZW5+ect7QDwr6QidfEawdmCqu2lMN6q25qxzhNa+sTH05/K/O6KBXOXhZVOS9IPNrKKfHP+KGILnii5ozCXXS1Rb8YguKGb9thYTXav6Xtq5I+up7S1unWrLunPrfkYlbncAeHbZNzV9xWl27cdsNHcdXABpll0nnlu/Yi/DOKxvqlMMjQ3vWAnsihhXz44J1yOyD6tals+rm0t5v6sLtTsN5N2Zvmt57KR5+8Stq59HEeX0nL0R/XRASJegPev1B9DHMj7z16Vi9ZA/mkGEuGfb+xUsAZtv3MizkvmLbJXW73f8lAc/8pLl2oHdcd6mbSwapi/P/rU6OfTjRz8f8dntJ9VQjDHkvyLwKQIzkltOXmtU35b/V9dF+YrvrEw0YPvb7lxLlRdBXpFrNX4DIgtiC6DI7MhjEGUQaxFoyvATglQ8q4uVXpI8u7W2qvhj2vHP+Tll3C1Xh/e4ylLKB/XvGftXQ1O6MimF0zIwDo2etWxe5RyaC8GZYvNyCtykl3mkfRbuZPCHZAsLF/oOIzcxvfliW0MZNw1vgrVsWqju3el5Y8X4r3tse7v6DNPXD4+rASfdxfsdt1XnxmHNNNUy41zM9P1GUzPyOCRrX9e6ywBupEHAQcgn+Pk810iXvBdnFST0NxLYo9iAkmVM98r+6Ts59Nst1jQY8y58xdExCvTsW8UXQl9ir+ZsLMeKxIx4/mv5ieFyJx5ZhXiq2BeB/p2zVczFhxfboEgvqNlY+os9vV/ljzhwQiFEIVBvsOGGW8T+m7FXfXWt15o+Zc8LEMN+s42ziNm3Xp8f/kDSvmuq8bGHGifgIKXQguCC8ku2jC+EGAYdJ7kcrHuwWPHd1fz6kdi7/yOn3xi/smZtlzsuSx4O1I+P7vubKMFJ2cd6/nekOEIXOfqLdItHES7RGSd4LMq9J3ahouJPc88mbb+t6Ofb6i65r5si6Lx8PG5BQ99m4CPoaezX/uPX+Z4kJ/JjIny62AHxn6vyE8ixUYetF07sv6PM6H/uRem7wR44ghTD1A49OzPJ5tlsYY+QX4I1LvHlplhesafHAnqkU3X8M5F9248Rfqa7bzc5K/BIXVugnpNjAKDQeWeLRpX1tOFZh/+55POisezYt9Vw1WDojYX7mF9eZ2I8dUYi5Y5hDlg/kvSBDx4bXTQ/HeraCwSOscUN7JuiS3PPV2vjuCWdH/dJ1zRyMjmGtMYkbVggT+IPy+wrMY4oLAduwpAVGW6LAFoCbJ8cXhCyWR5e3Phuf0FbKhsxyzhWCNBWY+2KWz1Ojd6ras/EywRplZtxehkVgv9weF3ePdAtBbulGipmujjbVtmuZujEvvj5opo8HD3Xf31cNf9YZNcPAi+mH3yuqFqnlmy6quV98q15dXKsGzzroPF3AkwbzGjbNe7mKuxx3/PHPJ/JekPm9xRHlYmwkfc5daVFnY/E9F1NNgA6CjI5hNwaJFyuqFxN2ezYNy2JEjbmdExaMPTcmvnp9oT+6bN38nj4X2PShY5zzhBANSs2Bel0+sMdj2+NzW7rBav5og343FoyYPTsh3lctXH3WPZKQ4gdzJ5u/rPS8NrxM5mXh8WDz7i/UxtVb1OCZexKuQb+BmHTMK6581BCRCbK6ujpVUlKiLSq8JgneP4h7wuUTm0fGF9n8dkPybX6CgNGxpwclvinXfnyb61sc5OKPhrmdEzoos3wL9dHl7aMbEs7jg2E9W6qZuyIEBW9HPlwan3OH73AzMdOyTY77w8gd/XqbGdJ/cR7le1wbDRP+r++8LBMsi4HlMeRaCmMYJUO/ilEzjJ5hFA2jaYivUDREZIIsFoup6uqem3FVVZWqrKx0vkcBCk7WWEKBr8pgTg2JnnUz4gvE7n07s9EWGR1bWPaKjrOp6gXXt3jI1R8Nczsnc35eIT66dPYEHRdfrHHj8Kec8zL3DQ0LRsUwOiZlBMPomeB3wzk24gEd/uM0xCAhxQD+2AVdXzEZfn9YMScUfdjitefUqp2X1a7aRnX6u2Z10+NtdZNC0RCRCDIZHRPwe8CAAe4vUuxs+yw+mnVg3B9d1/TA6NgfBq1IuKDbj21yfYuLXHUSsp0THl0ej/1Cl3MhPbq0t0XC7h1oN+isMwXzxzCPTG4CMMwzA143HJmPgnB4G5aQ/gyukUzfRMzVH9ZcE5kgswVYlI8tSX5z8exFfXO6POL7rmt4ZHSsqmyIjg8TRkn0yHZOhfro0mtbJAjNqMCblnjj0rwh4I1MYN5wrk39rZo0fKoOs3I7R+8JiYJCGdWKkrwRZPX19aq2tpZWoHZsRHx9q02ff+YZJpUNffMb9figVY6ok7i+XbvQMywtM9t74Jj664SeR3Pmo8urr/5K1R3a63lMvti5pfG9OWHYFumlyp2eYTOxrbuOqGfHJz6+fOvjfQlhJi6IT0R+rrs8TT8ajdZ/DZomLHxkSSJh55R/6xvk5gXh9xmV0bG5ZfFVoW/Mecb1JdlAVvMvpEeXXtsi2avwRwlW78cq/iK6YFjlH2DRV9O92B+nEEKyS2ST+iHAsjWpn+Q/+5bO0TfJTRPDL1GBuWOPDlqnLsTiiw3ePrDK9SXZQlbzNx9dOks7VGS+oW/UeG2L5LcKf5Q0t3U6+12a4gv7YWKHBfmN7ZIIISQTIhNkNTU1zigZjKNj/Y/Lh3bpG+Wh2IOuazBkdGz20JiOA1t0kL5BVvPfNPyPuvy1pbmhb1Q487XwZiNWxR95j84XtkV6vGxT0lX4o6TzTpeKzTuiBViibVHzvjjthiSEkPSITJCRfk7XHXV1RPzts0NHgi+MidEx7G92xlhgtm3v564v6QuwiOrh8gd0+Zt2eczP3TWEjiVdQyhqIMbquwWhnZ/vYj90tkUKsgp/1DzmseE7DJOOCSEkEyjISGQcezU+wrL6g+Wua3JkdGzm0JH62Oszn3R9SV8i5Z/Krs+Ir7LtJ9TMka2Ujz7vdKo7jRdVx5m9zmNqrLzfvHKyujL6x57pHy+/N9Qq/FFiCzHTCCEkEyjISGQcezf+9tunk8e6rsnB6Nj9A2vUydjP9bHYH430PdhSROogHROhho2EIcQS/CvuVs0rJmuxdfP9wc5LG42T00szV/gtWJnNHRYIIf0DCjISGVe3xd+AW1/+Z3Wnq8v18UZGx6YNHauPuz79MdeX9DXTx87x2IT3f5yNfo/OeNHZf87087NMNxgW8wuPPelyRX9dsJIQkn0oyEhkdNaf0TfN07Gfqb3Hr7s+3mB0DDe02tgv9XGtO5a6vqSvgaioGPqG74a+I+YfUVev3XL2o8OjSjyyxIhYUKGWzDBShhEzjJxd/+w1terNqWpR2UAPgXiXmjom/LIqUdIfF6wkhGQfCjISKRdG36tvnsuq/TcEl9GxyUMn6PCNUx5yfUmuMMXGn8ftUn+blPiI7skRO9TGfR6jQR23tVBrmBBvAwk29sfO40o8tsRcMcwZw9wxzCETELe5bRGW4zAF4vhugcjRKEJIMUJBRiLl2zdf1Dfgdyv9RzJkdMx8s691yyLXl+QTVSvPaIEkNqv6lOvbG0zgD7vB8K3WTjVl6fFe6QyeddARhvjO0ShCSDFTUIIMa5xluuBsLBbT66XJQrbpgMVvJZ4oFsGVfKWLmR8YfqeLuaYcLAz1q2Y7N+DZLzyUEAdM8iSjY6+WvqZv2I2T7lPKZ84Z6smMBztBpEOma+XJjhR22SIuiRdpBAHHoM5t/Nz9QFj7XNAeJT9B24G0H7NszXJ/ZVCZ+tO4bxLE0r+m71d15266oXuQMv5kzpReGwxLvuz6s0fFYE+VfeqENfMveRQLem5eZSp1KRakTUn+BbtdwoL0BXKc2Vbsay5IO5JjzH7MPq8g+LVf6ZNgZhp+JGt3Xu3UD7NczXrzc/fDLFMzbT/3ZEh4mN1WpF0GqTOvMjXzAwtybua1YLc5sz5T1ZtXmdr5Mf388Gt3tnuq/AC/9ujn7odXWYNk7dSLZGUtcUVFQQgyqVQUql0gYUAlSgVIo0sXs4EiHvsiDQPykmnF4ryCNK5USFmnez7tx7dqkbWj/Hdqd22j447ykotIRsf2lz+sw7Zu8s+7mR+cY7ptABe05AFxhIkHx+F4u5zN/EiYVOB80JbtTs7P3Q+khfBmmignyU/QukR4nAfiMcOa+YDf2vUb1fiFtQnCCVZdc9EJI3lHfPhugnjhZ6bhNyo2bMrH6sUXvcva/B2EZGUtbSEIOB7x4Dg/vM7bBvlHOMRnpo9ykd9e+bWRMHaakk+AtILEI+HxKW0J3+VYaUfJQBjkRb6b7Q5xmnGnwsyzHAv83P0ww+O7tB3bPVU8OE7qxqtMkRe4pWpP8Jc84LuUKb7bcabCDG+2ZbhLGkEw4/ErU8SXKk7UvdQ3vktbQPwSJ/KYqg0grITHp4RH+hKnWXZ+IIxZ1hKPnQeznfphl7WEl3NLlZcwFNQIGU5eKiUKoihI6XwyQRpIJvEEuWiCkGk8XS1NWmTBFqw8mXARyOjY2NKpOkzDhJ+rriQLjuJYuQjSzR+ONy9E5Mm80IJip484EZdg5jUZfumHzZd9XjZB8wOShTU7IggwW0RBqN1s6dlPEteodKoCzknyik/PUbHu3+b8NLus7d9Bscs0k37E7xrFOYW5fpEfs92Yv8Pkzy5rMw9S3kExwyM/Zrx2flNht6WweRG82hLwc/fDr+0gT2HOy25LyIe4hYkHSF3ZcYZF0k63jAW/MjWv/SAgDsQFzDjDnqfdHs3ytdtXMpC+pIv8mO3A75y9QHpe13iY6z4V/VaQmZWUDqhUVAQs7IVoYnYUmVSsmZ9M8oTyNeNJp4wuTo6/dTd12icJjV5Gx3aX9zzOgrVseNvx8wPnIvnJpP5xvFzE5sUeBrO+gN0x2B2HHwjjVbZ+7n4kOw/EFeYc7XNBnUm5250WHlXikaUpqPBIc8eRhl6dHL5LvWHka/TsrxKOg2GkDCNmJnZZp9vG7TK14wlTRgjvhZ3XVNjtBOUu+QlT/3ZZ41zMeP3y6wXyL/Vk589OJxk4zi7TdK835N9sk4Kfux9mueBTyjpMnQGUgxyDeKSu7PJKBcpSjjXzEzZP0m6AHQ8sTBl5hTev3aDgvMy2InkJW/9mezTLHQQpb4SRtAW7HSPOVOUNf4nHK00z/kzpt4IMhRimsSbDr6JSgfTNRhpVxaKc0un8gN1gzY4sKA0fjtBia+x/h+vzktGxkaUztH/D2B+rrrZbjr8fuCCQD+Qr3bIG5oUFS6eM7AsYcZjtKGjHjDDSIZv4ufthtyGTsHVnn4sJ/Mx2IWByvy2u/lM6LiGs1D9Gv0r+8v/UQy9/rMPao2ImdlmbhGnjdpna8cLP69y8kHOxgXuY/sRuJ2bbxmfQ/NjXq8RjWhCQdzOsnT87nWR4tbtk7dQPpOlV/37ufiAsjvEC5xk0LpyTeQ74LnVul1cqUNZe7UXqwcvPC7PtIn38FpKdt41fmXrVZTKQFzMPOBbnI59By1rKQZDfpgXNl6QNcJ5mO0Z+guYJeKUrcUdBvxRkYRtZKsJWqoDzkcZlWtCLMRmIJx3sc7EbcBDadn6kBdfIfz6jRoya6LjL6Nj28se1f8u6Nx0/P1AWZieIejN/pwviSact2eVjtyX8DlJ/OMbsuAQ/dz/s8hHMjjooyfKe7NqDoMJyGCKynniuXP110Czn7cjfDHinV/uGPfLifM9RMRO7rG0QTxDsMrXPJVU6Jl5pIr4wdQYQ3mw3Zryog6Dnluz6RDxB84X0zLq324+dXz/s4wS/duoH6sOrvfm5+5GqboJeb3adyG/b/OrCxO4zbIKWNcrBbLd2GQc9N78yxfFh6swrPH6b7QplZP72I1W4MPkCUqZ2meF3kDoTcKx5PEBeoyIyQYaTkkYJC3PRBAVpZBovKjJMBXiBijUrJYo4QVQVi7wEuRC9sI9FnoJcQCYdF45owYXjpy7YpEfHhpe+qf2ujbpHdTX3TPr3A2Vtlksm52aCOkPcYbEvSLRHaZNeHZIfCOt1Hn7uftidMMDxdqcRBMQjdY1P81pLFefV67edhWNFkD32jxlaoPVYz+r2f/zz3503KP1GxUzssjYJ0w7sMrXrKUxb8LpGwxwvID9yDMravM7s/CUDdeTX9yANPz8TpG3nH+UuZSb5SwXC+9UX4gh6TmbaJn7ufuDcvdI04zCvXz/s+vECcQZpA8hPsjoJWtZ++TbzCX+/+hCSlSncg7QfgHP3yjfc7HaeCvMYL5DnVHWGMHa6+LTbhLj7gTjMMvSqvyDnFJRIBVmqQkoXqWzTkhWiHyhYO56gDc4GFSNxmBWWCYgrXXDxSH4yiQeYcaVbPvWjfqje/9f9quyvv1Pl07/Wo2Obh/9eC7Lm1dPd0MlBuzLPLZ26B2adhT0vhDfzABNMt1R5g78ZHob27eeeDDs88uiVz1Tt0247ch2b7kGubfv6gvhKFGWb1TPP/UPtP3TMPcIbv7IO28aTlanZpoJcv2bbMY9BfGYHnwq7LeOcgF12qereDi95MMsoyHnZ+YFJ2uY5p8pPsnZnuyOsH151hnPyc0+GHV7KyDznVHEAs0zFkB8ThAlbZzCURdi6h78ZHibXp1kPqc4tWZmKX1Ds60OOtdtFsroHfu3RLKMgdWafm1mmZn2myg8wz828puxzNv3SpSAEGSk8Gt5+XguvsiGznJtx6ZC3tBvsTtMVNzQpJmwRZhohhBBvsvbIMojyJMVL86ppWnjNKytzbsYbhz+l3bCFDilOsKK+LcRg2JKJEEKIN2kLMnO4zh6+lSFV0n/Z+8UnWnxBiA0cMk//ht1puOCGJMUG9pp8YHDPnDGx+wd1C3LuQUkIIb5ENkJmA8FmC7Vk1NfXq9raWlqR2L9Gr9Di61zsR2rd8D/p3xcWlnoeQysee3/lAVUyapsjxp4evU29++k+z3A0Go1WjAZNE5asCDKMkEGQkf4LbsR1sV9oEWZa59XTbihCCCGEgMgEmf12RJjRMVJ8YB7RjuGPq3pXhMnnylEvuyEIIYQQImTtkSXp3+z9/JNuEXZXwsgY7PCHs90QhBBCCBEoyEhWaJwe38/SNLgTQgghJBEKMpIVvMSYGCGEEEISoSAjWcF3hGzao24IQgghhAgUZCQr3D64Rl0beXeiIKu4y3EnhBBCSCIUZCRrQHxhRMwZGZv+O9W251PXhxBCCCEmFGSEEEIIITmGgowQQgghJMdQkBFCCCGE5BgKMkIIIYSQHENBRgghhBCSYyjICCGEEEJyDAUZIYQQQkiOSUuQ1dXVqZKSElVVVeW69DBgwADHHVZTU+O6EkIIIYSQZIQWZBBaEF4QY6Ygw/fKykrnu4QhhBBCCCGpSfuRpS3IIMDMUTH8xkgaIYQQQghJTqSCzBRgsViMjy0JIYQQQgKQUpDJnDCYCQUZIYQQQkg05M0jy/r6elVbW0uj0Wg0Go1W0AZNE5bIBBkm9HNSPyGEEEJIeEILsurq6oTHmOajTNONE/oJIYQQQoKR9ggZIYQQQgiJBgoyQgghhJAcQ0FGCCGEEJJjKMgIIYQQQnIMBRkhhBBCSI6hICOEEEIIyTEUZIQQQgghOYaCjBBCCCEkx1CQEUIIIYTkGAoyQgghhJAcQ0FGCCGEEJJjKMgIIYQQQnIMBRkhhBBCSI6hICOEEEIIyTEUZIQQQgghOYaCjBBCCCEkx1CQEUIIIYTkGAoyQgghhJAcQ0FGSD+jsrJSVVdXO9/r6upUSUmJNsHP3QTxmGEkziAETdcrzmTpwq+mpsb9VVhUVVU5JpjniHIRBgwYoN3DnqtfnKa75AGfYeqUEJIZFGSE9DNwQxcgYOTGjO8wEIvF9M0YN2b8tkHYdG/YfvGb7hAbZl6FZOniGK+8FgLmuaJMRGzhu9SL+d2vfPzAcTgeoPzkWNNdBLF8DxM/ISQzKMgIKQBOXWxWn235zjF8TxfciOWGbmP6yU0Z+N2YEdZPGKXCL34zTj9xlSrdvhYRnZfqVNvOjxzD93RIJiRxviKYcG4i1AB+i6BOhR1W6gBxm4LMLD/kKWj8hJDMoCAjJM9ZtOas+vXgLeqJ2Hb16LBtzvcPN5x3fcOBG6+fmDFHp+wbvymgBAgFuIuFuXEni1/iM4WBSap0cR5m3NmkZeNcdW3k91XDhHtVw9ifON9bN7/n+gYH5S6iSJDzM4WaLarCnKtZv8CMC98lPZNU4pcQEh0UZITkgCkfHFf3DdzcJ4a0BL8bLNzMGz9+yw3a60ZtAzFhHm+DG7/Eg5u/X/wQF/gun7ZIsfFKNxMRcWv5KHVtxP/0iSEtAefhd67mOYYVZGb5SpmahrikbsQf5SckyxchJFooyAjJAV7CKZsmeIkV3Ihxo/cDN+xkYgsgTLI4kmHGbwsOEQ1+eKWbiSDzEk7ZNCGV8EE5AJyrKcDs8gqDlJst6sw0KMgI6TsoyAjJAbkcITNvsDIqkgzcsFMJHMRpjqyEwYxfRmoAhEaqvHmlawuMMORyhMw8DxGoAGUj4glhJBzOUdzDgjikHSAOs35NEYxw6YpbQkg4KMgIyWMwgR9zxmyRBbd0Jvfj5mre+HEzxg3YNABBIL9NAWcKAvNYUxgEEWd+8SN/4g4TMRAkXQHufQEm8GPOWC+x1e0WdnI/xJUtwuQc7fMx3UU4Ibx5vBcoPznOrB8R5WJmfaB80x2BI4SEg4KMkDwHE/hFlD1UutX5jon+6eIlYqIEN3sRUn2NLWyyDSbwa1E26gfOd0z0T4dM6gUiyhRSUQAhlu22QgiJQ0FGSAEQ1bIXIJuCCTfxvhRENji3dB9XpksUy16ATERVNoQT8pIrYU1If4SCjBBCCCEkx1CQEUIIIYTkGAoyQgghhJAcQ0FGCCGEEJJjKMgIIYQQQnIMBRkhhBBCSI6hICOEEEIIyTEUZIQQQgghOYaCjBBCCCEkx1CQEUIIIYTkFKX+f5IU4ojKC6vbAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Image(\"spatialVStimewise_imputation.PNG\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chitwan\n",
    "- impact_score has 132409 (34.0%) missing values\n",
    "- Land_sea_coverage_nearest_neighbor_land1sea0_surface has 122793 (31.5%) missing values\n",
    "- Snow_depth_surface has 5544 (1.4%) missing values\n",
    "- Snow_depth_surface has 356611 (91.6%) zeros\n",
    "- Water_equivalent_of_accumulated_snow_depth_surface has 356781 (91.6%) zeros\n",
    "- Categorical_Snow_surface has 316016 (81.2%) missing values\n",
    "- Composite_reflectivity_entire_atmosphere has 316016 (81.2%) missing values\n",
    "- Graupel_snow_pellets_hybrid has 316016 (81.2%) missing values\n",
    "- Graupel_snow_pellets_hybrid is highly skewed (γ1 = 73.26876449)\n",
    "- Graupel_snow_pellets_hybrid has 73043 (18.8%) zeros\n",
    "- Graupel_snow_pellets_isobaric has 316016 (81.2%) missing values\n",
    "- Snow_mixing_ratio_hybrid has 316016 (81.2%) missing values\n",
    "- Snow_mixing_ratio_hybrid is highly skewed (γ1 = 31.41715949)\n",
    "- Snow_mixing_ratio_hybrid has 72934 (18.7%) zeros\n",
    "- Snow_mixing_ratio_isobaric has 316016 (81.2%) missing values!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import libraries\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the merged data:\n",
    "data = pd.read_hdf('data.h5')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['date_key', 'zip5', 'impact_score', 'grid_lat', 'grid_lon', 'Date',\n",
       "       'Time', 'ForecastRange', 'x', 'y',\n",
       "       ...\n",
       "       'Snow_mixing_ratio_hybrid', 'Snow_mixing_ratio_isobaric',\n",
       "       'Total_cloud_cover_isobaric', 'Vertical_velocity_geometric_isobaric',\n",
       "       'Ice_growth_rate_altitude_above_msl', 'density', 'state', 'Region',\n",
       "       'USR', 'Weekday'],\n",
       "      dtype='object', length=128)"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [],
   "source": [
    "missing = [\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\",\n",
    "\"Snow_depth_surface\",\n",
    "\"Water_equivalent_of_accumulated_snow_depth_surface\",\n",
    "\"Categorical_Snow_surface\",\n",
    "\"Composite_reflectivity_entire_atmosphere\",\n",
    "\"Graupel_snow_pellets_hybrid\",\n",
    "\"Graupel_snow_pellets_isobaric\",\n",
    "\"Snow_mixing_ratio_hybrid\",\n",
    "\"Snow_mixing_ratio_isobaric\"]\n",
    "identifiers = [\"date_key\", \"zip5\", \"impact_score\", \"grid_lat\", \"grid_lon\", \"Time\", \"state\", \"Region\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [],
   "source": [
    "dirty = data[identifiers + missing]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_key</th>\n",
       "      <th>zip5</th>\n",
       "      <th>impact_score</th>\n",
       "      <th>grid_lat</th>\n",
       "      <th>grid_lon</th>\n",
       "      <th>Time</th>\n",
       "      <th>state</th>\n",
       "      <th>Region</th>\n",
       "      <th>Land_sea_coverage_nearest_neighbor_land1sea0_surface</th>\n",
       "      <th>Snow_depth_surface</th>\n",
       "      <th>Water_equivalent_of_accumulated_snow_depth_surface</th>\n",
       "      <th>Categorical_Snow_surface</th>\n",
       "      <th>Composite_reflectivity_entire_atmosphere</th>\n",
       "      <th>Graupel_snow_pellets_hybrid</th>\n",
       "      <th>Graupel_snow_pellets_isobaric</th>\n",
       "      <th>Snow_mixing_ratio_hybrid</th>\n",
       "      <th>Snow_mixing_ratio_isobaric</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>12</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>18</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>2722</td>\n",
       "      <td>16.868994</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389326</th>\n",
       "      <td>2019-12-30</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>18</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-20.000004</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.990000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.890000e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389327</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>0</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-18.260004</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.151257e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.051257e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389328</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>6</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.899997</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.120000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.020000e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389329</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>12</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.509996</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.390000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.270000e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389330</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>18</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>19.489996</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.610000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.470000e-09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>389331 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         date_key   zip5  impact_score  grid_lat  grid_lon  Time state  \\\n",
       "0      2017-01-01   2722     20.268081      41.5     -71.0     0    MA   \n",
       "1      2017-01-01   2722     20.268081      41.5     -71.0     6    MA   \n",
       "2      2017-01-01   2722     20.268081      41.5     -71.0    12    MA   \n",
       "3      2017-01-01   2722     20.268081      41.5     -71.0    18    MA   \n",
       "4      2017-01-02   2722     16.868994      41.5     -71.0     0    MA   \n",
       "...           ...    ...           ...       ...       ...   ...   ...   \n",
       "389326 2019-12-30  98421           NaN      47.5    -122.5    18    WA   \n",
       "389327 2019-12-31  98421           NaN      47.5    -122.5     0    WA   \n",
       "389328 2019-12-31  98421           NaN      47.5    -122.5     6    WA   \n",
       "389329 2019-12-31  98421           NaN      47.5    -122.5    12    WA   \n",
       "389330 2019-12-31  98421           NaN      47.5    -122.5    18    WA   \n",
       "\n",
       "                       Region  \\\n",
       "0       New England Northeast   \n",
       "1       New England Northeast   \n",
       "2       New England Northeast   \n",
       "3       New England Northeast   \n",
       "4       New England Northeast   \n",
       "...                       ...   \n",
       "389326           Pacific West   \n",
       "389327           Pacific West   \n",
       "389328           Pacific West   \n",
       "389329           Pacific West   \n",
       "389330           Pacific West   \n",
       "\n",
       "        Land_sea_coverage_nearest_neighbor_land1sea0_surface  \\\n",
       "0                                                     NaN      \n",
       "1                                                     NaN      \n",
       "2                                                     NaN      \n",
       "3                                                     NaN      \n",
       "4                                                     NaN      \n",
       "...                                                   ...      \n",
       "389326                                                1.0      \n",
       "389327                                                1.0      \n",
       "389328                                                1.0      \n",
       "389329                                                1.0      \n",
       "389330                                                1.0      \n",
       "\n",
       "        Snow_depth_surface  \\\n",
       "0                      0.0   \n",
       "1                      0.0   \n",
       "2                      0.0   \n",
       "3                      0.0   \n",
       "4                      0.0   \n",
       "...                    ...   \n",
       "389326                 0.0   \n",
       "389327                 0.0   \n",
       "389328                 0.0   \n",
       "389329                 0.0   \n",
       "389330                 0.0   \n",
       "\n",
       "        Water_equivalent_of_accumulated_snow_depth_surface  \\\n",
       "0                                                     0.0    \n",
       "1                                                     0.0    \n",
       "2                                                     0.0    \n",
       "3                                                     0.0    \n",
       "4                                                     0.0    \n",
       "...                                                   ...    \n",
       "389326                                                0.0    \n",
       "389327                                                0.0    \n",
       "389328                                                0.0    \n",
       "389329                                                0.0    \n",
       "389330                                                0.0    \n",
       "\n",
       "        Categorical_Snow_surface  Composite_reflectivity_entire_atmosphere  \\\n",
       "0                            NaN                                       NaN   \n",
       "1                            NaN                                       NaN   \n",
       "2                            NaN                                       NaN   \n",
       "3                            NaN                                       NaN   \n",
       "4                            NaN                                       NaN   \n",
       "...                          ...                                       ...   \n",
       "389326                       0.0                                -20.000004   \n",
       "389327                       0.0                                -18.260004   \n",
       "389328                       0.0                                 13.899997   \n",
       "389329                       0.0                                 14.509996   \n",
       "389330                       0.0                                 19.489996   \n",
       "\n",
       "        Graupel_snow_pellets_hybrid  Graupel_snow_pellets_isobaric  \\\n",
       "0                               NaN                            NaN   \n",
       "1                               NaN                            NaN   \n",
       "2                               NaN                            NaN   \n",
       "3                               NaN                            NaN   \n",
       "4                               NaN                            NaN   \n",
       "...                             ...                            ...   \n",
       "389326                          0.0                   2.990000e-09   \n",
       "389327                          0.0                   3.151257e-09   \n",
       "389328                          0.0                   3.120000e-09   \n",
       "389329                          0.0                   3.390000e-09   \n",
       "389330                          0.0                   3.610000e-09   \n",
       "\n",
       "        Snow_mixing_ratio_hybrid  Snow_mixing_ratio_isobaric  \n",
       "0                            NaN                         NaN  \n",
       "1                            NaN                         NaN  \n",
       "2                            NaN                         NaN  \n",
       "3                            NaN                         NaN  \n",
       "4                            NaN                         NaN  \n",
       "...                          ...                         ...  \n",
       "389326                       0.0                2.890000e-09  \n",
       "389327                       0.0                3.051257e-09  \n",
       "389328                       0.0                3.020000e-09  \n",
       "389329                       0.0                3.270000e-09  \n",
       "389330                       0.0                3.470000e-09  \n",
       "\n",
       "[389331 rows x 17 columns]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "dirty.to_csv(\"dirty.csv\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2722,  3063,  8085,  8512,  8518,  8691,  8817, 17013, 17015,\n",
       "       17339, 18031, 18045, 18202, 18640, 18707, 19709, 20166, 21224,\n",
       "       23803, 23836, 28214, 29172, 29303, 29418, 30144, 30291, 30517,\n",
       "       32218, 32221, 33182, 33572, 33811, 37090, 37127, 37321, 37421,\n",
       "       40165, 40218, 40511, 41048, 42718, 43062, 43125, 43207, 46075,\n",
       "       46168, 46231, 47130, 48150, 53144, 53703, 53718, 55379, 60433,\n",
       "       60436, 60446, 60449, 62025, 66030, 66102, 66219, 67337, 75019,\n",
       "       75241, 76177, 77090, 78154, 78666, 80216, 85043, 85338, 85353,\n",
       "       89030, 89115, 89434, 89506, 91752, 92374, 92376, 92408, 92551,\n",
       "       95304, 95363, 95391, 95837, 97124, 98032, 98327, 98390, 98421])"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty[dirty[\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"] ==  1][\"zip5\"].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([2722, 7008])"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty[dirty[\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"] ==  0][\"zip5\"].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2722,  3063,  7008,  8085,  8512,  8518,  8691,  8817, 17013,\n",
       "       17015, 17339, 18031, 18045, 18202, 18640, 18707, 19709, 20166,\n",
       "       21224, 23803, 23836, 28214, 29172, 29303, 29418, 30144, 30291,\n",
       "       30517, 32218, 32221, 33182, 33572, 33811, 37090, 37127, 37321,\n",
       "       37421, 40165, 40218, 40511, 41048, 42718, 43062, 43125, 43207,\n",
       "       46075, 46168, 46231, 47130, 48150, 53144, 53703, 53718, 55379,\n",
       "       60433, 60436, 60446, 60449, 62025, 66030, 66102, 66219, 67337,\n",
       "       75019, 75241, 76177, 77090, 78154, 78666, 80216, 85043, 85338,\n",
       "       85353, 89030, 89115, 89434, 89506, 91752, 92374, 92376, 92408,\n",
       "       92551, 95304, 95363, 95391, 95837, 97124, 98032, 98327, 98390,\n",
       "       98421])"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty[pd.isna(dirty[\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"])][\"zip5\"].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>zip5</th>\n",
       "      <th>date_key</th>\n",
       "      <th>Land_sea_coverage_nearest_neighbor_land1sea0_surface</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4321</th>\n",
       "      <td>3063</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4322</th>\n",
       "      <td>3063</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4323</th>\n",
       "      <td>3063</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4324</th>\n",
       "      <td>3063</td>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4325</th>\n",
       "      <td>3063</td>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8329</th>\n",
       "      <td>3063</td>\n",
       "      <td>2019-10-15</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8330</th>\n",
       "      <td>3063</td>\n",
       "      <td>2019-10-16</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8331</th>\n",
       "      <td>3063</td>\n",
       "      <td>2019-10-16</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8332</th>\n",
       "      <td>3063</td>\n",
       "      <td>2019-10-16</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8333</th>\n",
       "      <td>3063</td>\n",
       "      <td>2019-10-16</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4013 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      zip5   date_key  Land_sea_coverage_nearest_neighbor_land1sea0_surface\n",
       "4321  3063 2017-01-01                                                NaN   \n",
       "4322  3063 2017-01-01                                                NaN   \n",
       "4323  3063 2017-01-01                                                NaN   \n",
       "4324  3063 2017-01-01                                                NaN   \n",
       "4325  3063 2017-01-02                                                NaN   \n",
       "...    ...        ...                                                ...   \n",
       "8329  3063 2019-10-15                                                NaN   \n",
       "8330  3063 2019-10-16                                                NaN   \n",
       "8331  3063 2019-10-16                                                NaN   \n",
       "8332  3063 2019-10-16                                                NaN   \n",
       "8333  3063 2019-10-16                                                NaN   \n",
       "\n",
       "[4013 rows x 3 columns]"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty[dirty[\"zip5\"] == 3063][[\"zip5\", \"date_key\", \"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Features with 80-100% missing values are difficult to impute. Let's drop those columns: \n",
    "\n",
    "[\"Categorical_Snow_surface\",\n",
    "\"Composite_reflectivity_entire_atmosphere\",\n",
    "\"Graupel_snow_pellets_hybrid\",\n",
    "\"Graupel_snow_pellets_isobaric\",\n",
    "\"Snow_mixing_ratio_hybrid\",\n",
    "\"Snow_mixing_ratio_isobaric\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Land_sea_coverage_nearest_neighbor_land1sea0_surface is 1 for all land neighbors and 0 for a sea border\n",
    "\n",
    "So I'm not 100% certain what this column means, but I am assuming that it is static over time - either a zipcode has all land borders or it has a sea border? I tried to look up a few zipcodes and it didn't follow this pattern.\n",
    "\n",
    "All zipcodes have NA values for 1/1/17-7/18/17 and 6/13/19-12/31/19. All zipcodes except 02722 have only 1 non-NA value. I backfilled for missing data 1/1/17-7/18/17 for all zipcodes. 02722 will be a special case where the value switches from 1 to 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chitwan/anaconda/envs/weather/lib/python3.7/site-packages/ipykernel_launcher.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n"
     ]
    }
   ],
   "source": [
    "## Filling in missing data for Land_sea_coverage_nearest_neighbor_land1sea0_surface\n",
    "clean0 = dirty\n",
    "clean0[\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"] = clean0.groupby(\"zip5\").\\\n",
    "    transform(lambda x: x.fillna(x.mean()))[\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"]\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/chitwan/anaconda/envs/weather/lib/python3.7/site-packages/ipykernel_launcher.py:3: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  This is separate from the ipykernel package so we can avoid doing imports until\n",
      "/Users/chitwan/anaconda/envs/weather/lib/python3.7/site-packages/ipykernel_launcher.py:6: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame.\n",
      "Try using .loc[row_indexer,col_indexer] = value instead\n",
      "\n",
      "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>date_key</th>\n",
       "      <th>zip5</th>\n",
       "      <th>impact_score</th>\n",
       "      <th>grid_lat</th>\n",
       "      <th>grid_lon</th>\n",
       "      <th>Time</th>\n",
       "      <th>state</th>\n",
       "      <th>Region</th>\n",
       "      <th>Land_sea_coverage_nearest_neighbor_land1sea0_surface</th>\n",
       "      <th>Snow_depth_surface</th>\n",
       "      <th>Water_equivalent_of_accumulated_snow_depth_surface</th>\n",
       "      <th>Categorical_Snow_surface</th>\n",
       "      <th>Composite_reflectivity_entire_atmosphere</th>\n",
       "      <th>Graupel_snow_pellets_hybrid</th>\n",
       "      <th>Graupel_snow_pellets_isobaric</th>\n",
       "      <th>Snow_mixing_ratio_hybrid</th>\n",
       "      <th>Snow_mixing_ratio_isobaric</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>0.925648</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>6</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>0.925648</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>12</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>0.925648</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2017-01-01</td>\n",
       "      <td>2722</td>\n",
       "      <td>20.268081</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>18</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>0.925648</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2017-01-02</td>\n",
       "      <td>2722</td>\n",
       "      <td>16.868994</td>\n",
       "      <td>41.5</td>\n",
       "      <td>-71.0</td>\n",
       "      <td>0</td>\n",
       "      <td>MA</td>\n",
       "      <td>New England Northeast</td>\n",
       "      <td>0.925648</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389326</th>\n",
       "      <td>2019-12-30</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>18</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-20.000004</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.990000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.890000e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389327</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>0</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-18.260004</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.151257e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.051257e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389328</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>6</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>13.899997</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.120000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.020000e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389329</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>12</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>14.509996</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.390000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.270000e-09</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389330</th>\n",
       "      <td>2019-12-31</td>\n",
       "      <td>98421</td>\n",
       "      <td>NaN</td>\n",
       "      <td>47.5</td>\n",
       "      <td>-122.5</td>\n",
       "      <td>18</td>\n",
       "      <td>WA</td>\n",
       "      <td>Pacific West</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>19.489996</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.610000e-09</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.470000e-09</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>389331 rows × 17 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         date_key   zip5  impact_score  grid_lat  grid_lon  Time state  \\\n",
       "0      2017-01-01   2722     20.268081      41.5     -71.0     0    MA   \n",
       "1      2017-01-01   2722     20.268081      41.5     -71.0     6    MA   \n",
       "2      2017-01-01   2722     20.268081      41.5     -71.0    12    MA   \n",
       "3      2017-01-01   2722     20.268081      41.5     -71.0    18    MA   \n",
       "4      2017-01-02   2722     16.868994      41.5     -71.0     0    MA   \n",
       "...           ...    ...           ...       ...       ...   ...   ...   \n",
       "389326 2019-12-30  98421           NaN      47.5    -122.5    18    WA   \n",
       "389327 2019-12-31  98421           NaN      47.5    -122.5     0    WA   \n",
       "389328 2019-12-31  98421           NaN      47.5    -122.5     6    WA   \n",
       "389329 2019-12-31  98421           NaN      47.5    -122.5    12    WA   \n",
       "389330 2019-12-31  98421           NaN      47.5    -122.5    18    WA   \n",
       "\n",
       "                       Region  \\\n",
       "0       New England Northeast   \n",
       "1       New England Northeast   \n",
       "2       New England Northeast   \n",
       "3       New England Northeast   \n",
       "4       New England Northeast   \n",
       "...                       ...   \n",
       "389326           Pacific West   \n",
       "389327           Pacific West   \n",
       "389328           Pacific West   \n",
       "389329           Pacific West   \n",
       "389330           Pacific West   \n",
       "\n",
       "        Land_sea_coverage_nearest_neighbor_land1sea0_surface  \\\n",
       "0                                                0.925648      \n",
       "1                                                0.925648      \n",
       "2                                                0.925648      \n",
       "3                                                0.925648      \n",
       "4                                                0.925648      \n",
       "...                                                   ...      \n",
       "389326                                           1.000000      \n",
       "389327                                           1.000000      \n",
       "389328                                           1.000000      \n",
       "389329                                           1.000000      \n",
       "389330                                           1.000000      \n",
       "\n",
       "        Snow_depth_surface  \\\n",
       "0                      0.0   \n",
       "1                      0.0   \n",
       "2                      0.0   \n",
       "3                      0.0   \n",
       "4                      0.0   \n",
       "...                    ...   \n",
       "389326                 0.0   \n",
       "389327                 0.0   \n",
       "389328                 0.0   \n",
       "389329                 0.0   \n",
       "389330                 0.0   \n",
       "\n",
       "        Water_equivalent_of_accumulated_snow_depth_surface  \\\n",
       "0                                                     0.0    \n",
       "1                                                     0.0    \n",
       "2                                                     0.0    \n",
       "3                                                     0.0    \n",
       "4                                                     0.0    \n",
       "...                                                   ...    \n",
       "389326                                                0.0    \n",
       "389327                                                0.0    \n",
       "389328                                                0.0    \n",
       "389329                                                0.0    \n",
       "389330                                                0.0    \n",
       "\n",
       "        Categorical_Snow_surface  Composite_reflectivity_entire_atmosphere  \\\n",
       "0                            NaN                                       NaN   \n",
       "1                            NaN                                       NaN   \n",
       "2                            NaN                                       NaN   \n",
       "3                            NaN                                       NaN   \n",
       "4                            NaN                                       NaN   \n",
       "...                          ...                                       ...   \n",
       "389326                       0.0                                -20.000004   \n",
       "389327                       0.0                                -18.260004   \n",
       "389328                       0.0                                 13.899997   \n",
       "389329                       0.0                                 14.509996   \n",
       "389330                       0.0                                 19.489996   \n",
       "\n",
       "        Graupel_snow_pellets_hybrid  Graupel_snow_pellets_isobaric  \\\n",
       "0                               NaN                            NaN   \n",
       "1                               NaN                            NaN   \n",
       "2                               NaN                            NaN   \n",
       "3                               NaN                            NaN   \n",
       "4                               NaN                            NaN   \n",
       "...                             ...                            ...   \n",
       "389326                          0.0                   2.990000e-09   \n",
       "389327                          0.0                   3.151257e-09   \n",
       "389328                          0.0                   3.120000e-09   \n",
       "389329                          0.0                   3.390000e-09   \n",
       "389330                          0.0                   3.610000e-09   \n",
       "\n",
       "        Snow_mixing_ratio_hybrid  Snow_mixing_ratio_isobaric  \n",
       "0                            NaN                         NaN  \n",
       "1                            NaN                         NaN  \n",
       "2                            NaN                         NaN  \n",
       "3                            NaN                         NaN  \n",
       "4                            NaN                         NaN  \n",
       "...                          ...                         ...  \n",
       "389326                       0.0                2.890000e-09  \n",
       "389327                       0.0                3.051257e-09  \n",
       "389328                       0.0                3.020000e-09  \n",
       "389329                       0.0                3.270000e-09  \n",
       "389330                       0.0                3.470000e-09  \n",
       "\n",
       "[389331 rows x 17 columns]"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Maually change values for zip5 = 02722\n",
    "# Somthing funky is happeneing, return to this...\n",
    "clean0.loc[:, (clean0[\"date_key\"] < \"2017-07-19\") & (clean0['zip5'] == 2722)]\\\n",
    "    [\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"] = 1\n",
    "\n",
    "clean0.loc[(clean0[\"date_key\"] >= \"2019-06-13\") & (clean0['zip5'] == 2722)]\\\n",
    "    [\"Land_sea_coverage_nearest_neighbor_land1sea0_surface\"] = 0\n",
    "\n",
    "clean0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Snow_depth_surface"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Snow_depth_surface = Snow depth @ Ground or water surface. Most values are 0 which is not unexpected but let's investigate the 1.4% missing values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 2722,  7008, 98032, 98421])"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty[pd.isna(dirty[\"Snow_depth_surface\"])][\"zip5\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These 4 zipcodes are missing every single datapoint for snow_depth_surface. I "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAEICAYAAAC3Y/QeAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAX5UlEQVR4nO3dfZQsdX3n8fdHEDELiAoqctlcEzHykCMeR8TNUXf1RjEkgri6cDBoYpaYYNQc1yiSdTWre3B3fX5KbvAZE9Qo4orGhFXW6PFprqJyRRfU63IXhHGPCNcHIvjdP6oG+g7dM/dOT013T71f5/SZ7l9VV/361zXf/tavflWVqkKS1C93mXQFJEnrz+AvST1k8JekHjL4S1IPGfwlqYcM/pLUQwZ/dSrJZ5M8dJ3XeUaSf9iD+V6T5NnrUadJSPL4JB+ewHq3J/nXK8xz3yRXJrnbOlVLS8Rx/v2WZAfwB1V16V6+7zLggqo6f5l5fgc4u6pOHKuSHUlyGPBF4Fer6p/Xcb0FHFlVV3e8nnngOVX1+S7Xs1pJ3gJcWVVvnHRd+sjMX116NvCeSVdilKq6Dvgm8KRJ12WtJXk4cI9pDfyt9wJ/OOlK9JXBX0MluWeSjyZZSPLD9vmmdtorgUcBb0qyK8mbhrx/P+CxwP8aKPtYklcPvH5fkrcvU4fXJ7kmyU1JtiV51J4sK8kzk3ymfZ4kr01yQ5IfJflakmMHVnMZcNKI9f99kucsKftqklP3YLl7Lcldkvx5ku+1y313knu00zYnqSTPSPJ/kvwgybnLLO6J7N72bx5sr7bsfyR5/oi6HJ/kc0luTHJdkje13ylJ/lW7/iPa1w9p53tw+3pHki0Dy5lvv8Prk7xmYDVfAH4lyS/vfWtpbFXlo8cPYAewZUj5vYGnAL8EHAh8APjwwPTLaLqLRi33GODHS8ruB9xA86NwBvAd4MBllvH0th77Ai8Avg/sv9KygGcCn2mfPwHYBhwMBDgKOGxgHacCXx6x/jOBzw68Phq4EbjbSstdoc0LeOCQ8t8HrgZ+BTgA+BDwnnba5vZ9fw3cHXgIcAtw1Ih1fAB44cDr44Frgbu0rw8BfgLcd8T7Hwac0Lb9ZuBK4PkD018JfLKty9doupfutE0BnwN+t31+AHDCkvV8DXjSpP8P+vgw89dQVfX/quqDVfWTqrqZ5p/9MXuxiIOBm5cs8/s0XUHvAl4PnNkue1QdLmjrcWtVvZom6P7aXi7r5zQ/Xg+mOcZ1ZTXdPYtubus6zEXAcQOZ6RnAh6rqlj1Y7mqcAbymqr5TVbuAc4DTkuw7MM/Lq+qnVfVV4Ks0PwLD7Nb+VfVF4EfA49qi04DLqur6YW+uqm1V9fm27XcAf8Xu3//LgHvQHDO5FnjziHr8HHhgkkOqalfduRtqufZXhwz+GirJLyX5q7YL4ibg08DBSfbZw0X8kCY4LvVRYB/gW1X1mYH1bW+7kHYtdu8keUE7IuRHSW6kCTaHrLSsQVX1SeBNNMHp+iRbkxw0MMuBNNn8sPfeDFxCEyhp/753D5e7GvcHvjfw+ns0mfd9B8q+P/D8JzTZ9DDD2v9dNHtTtH/fA7ePjlps+4+3ZQ9qu/q+337//4WBtq+qnwPvBI4FXl1Vo0aOPAt4EPDNJF9K8ttLpo9sf3XL4K9RXkCTZT+iqg4CHt2Wp/270jCxq2i63A9fUv5Kmi6Ew5KcvlhYVcdU1QHt45/aH4AXAU8D7llVB9NkrllpWUtV1Ruq6mE0XVEPAl44MPkomgx6lL8FTk/ySJoujk/t4XJX41pgsP/7XwK3AkOz8xV8ra3ToAuAk5M8hOZzfxigqt470PZPbOd9K83B8CPb7/8lDLR9+73+J+AdwKszYshmVV1VVacD9wFeBfxdkn/RLmNf4IEs3/7qiMFfAHdNsv/AY1+ajOynwI1J7kXzjz7oepq+6aHazPBSBroKkjwa+D2avvQzgTcO+XFYdCBN4FsA9k3yUuD2zHpPl5Xk4UkekeSuwI+BnwG3DczyGODjoz4H8DGagPwXwPuq6hd7uNyV7Lekzfeh+aH50yQPSHIATbb9vqq6dS+WO1jv3brpqmon8CWajP+DVfXTZd5/IHATsKs9kPtHixOShCbrfxtNZn8d8J+HLSTJ05Mc2rbbYoa/2E7HAzuq6nvD3quOTfqgg4/JPmgOztWSxytouiAuA3YB/5tmSF4B+7bve2Rb/kPgDSOWfRLw8fb5Qe26ThuY/irgH2jPN1ny3n1ogstNNMHlz9r3b1lpWex+wPdxNFnwLuAHNN02B7TTDgN2Avut0EZvaz/7wwfKllvuSxY/94jlLW3vAv6AJhl7KXANzY/eBTR7PXDHAd99B5ZzGcsfdP8SzZ7bYNnT2+X8mxU+86NpMv9dwD/R/Pgttunz2s++X/v6/m19HzWwTS0e8L2A5sD8LmA7cMrAOt4MPHfS/wN9fXiSlzqVZsjln1TVVyZdl6XaoY/frqq3TLouXUjyeOCPq+qUgbJH0wTkzdXuxUyobvehGYr60Kr62aTq0WcGf6kn2i6qC4GvVtVfTLo+miz7/KUeSHIUTZ/7YcDrJlwdTQEzf0nqITN/SeqhfVeeZToccsghtXnz5klXQ5JmxrZt235QVYcOmzYzwX/z5s3Mz89PuhqSNDOSjDyHwm4fSeqhzoJ/mkvsXt4+diS5fGDaOUmuTvKtJE/oqg6SpOE66/apqn+3+Lw9meZH7fOjaS6QdQzNmYGXJnlQVe3NqfGSpDF03u3TXgfkaTTXLQE4Gbiwqm6pqu/SXL/8+K7rIUm6w3r0+T8KuL6qrmpfH05z7ZJFO9uyO0lyVnsXoPmFhYWOqylJ/TFWt0+SS2nuqLTUuVV1cfv8dO7I+mH3S/IuGnqmWVVtBbYCzM3NeTaaNKU2v/iSO5XtOG/o3TE1JcYK/lW1Zbnp7aWBT6W5JdyincARA6830VzHXNIMGhb4F8v9AZheXY/z3wJ8s5rriC/6CPA37Y2c7w8cSXMrOEm6nXsT3eq6z/80du/yoaq2A+8HvgH8PXC2I30kDVpub0Jro9PMv6qeOaL8lTS34JMkTYBn+EpSDxn8JY1lVD+8/fPTbWYu7CZpehnoZ4+Zv6Sp495E98z8JU0lA323zPwlqYcM/pLUQ3b7SLqdZ9X2h5m/JMCzavvGzF/qGbN7gZm/1Ctm91pk8JekHrLbR9qA7NrRSsz8pQ3Grh3tCTN/SXtkub0J9zRmT6pm49a4c3NzNT8/P+lqSFNvmjJ8fwAmK8m2qpobNs1uH6lHvGCaFtntI/WMgV5g8Jd6Z1T//I7zTho5bZq6krQ2DP5Sjyw3EmjxB0D9YPCX1sh6j4ZZzxE2y+0VrHddtDYc7SOtgdV2i6w2QHbRDWOw3ngc7SP1iCN6tCfs9pE2IAO9VmLmL/WIewVaZOYv9YyBXmDmL62J5TLqLrJtM3iNy9E+krRBOdpHkrSbzvr8kxwH/CWwP3Ar8MdV9cUkAV4P/BbwE+CZVfXlruqh6eBJQNJ06TLz/6/Ay6vqOOCl7WuAJwJHto+zgLd2WAdNAW8uIk2fLkf7FHBQ+/wewLXt85OBd1dzsOHzSQ5OclhVXbfWFTDblKThusz8nw/8tyTXAP8dOKctPxy4ZmC+nW3ZnSQ5K8l8kvmFhYW9WrnZpiSNNlbmn+RS4H5DJp0LPA7406r6YJKnAW8DtgAZMv/QIUdVtRXYCs1on3Hqqtk06xdEk6bVWJl/VW2pqmOHPC4GngF8qJ31A8Dx7fOdwBEDi9nEHV1C0u262Htzj1BqdNnnfy3wGOAy4LHAVW35R4DnJLkQeATwoy76+zU9Ge5KlwOWtP66DP7/Hnh9kn2Bn9GM7AH4GM0wz6tphnr+Xod16K2Vbtqx3gz00nTpLPhX1WeAhw0pL+Dsrta7aBLZptnt9PC7kJa3oS/stp7/7NOWafeZ/frSyjZ08Nds62LvzeMP7hWpYfDXVOsiKPU50LmHqkVe2E2SesjgL0k9ZLfPBtXnvu0+f/ZZ4/c0OQb/DazP/0R9/uyzwuMPk2XwXyNmm92wTaVuGPzXkEFpbZkZSt0x+Gvi+p7dr/bzr+Z97qFqkcFfE9XV2bjLBbhpCn6r3bsZZ6/IQC9wqKc2oOUCo5d+mB6jfoT8cVofZv7SOljPrp1ZspE+y6wx85c6ttq9DfdS1CWDvyT1kMFfE7Vcv699wqPZNhpXmnurTL+5ubman5+fdDU0I1Y72me9bhg/rpXqNCujndStJNuqam7oNIO/1FguSI8THNc7+K92ff4AbDzLBX+7faSOrbZry64ddcmhntI6WC5gr3aaNA4zf0nqIYO/JPWQB3ylARtlJMw0jGjyDObJc7SPpM6sdpRUV6OrdAdH+0iSdmPwl6QeMvhLUg8Z/CWphzzgK2lsXYz2cSTQ+CYy2ifJQ4C/BA4AdgBnVNVN7bRzgGcBtwHPrapPrLQ8g7+0u40cHB0JtDYmNdrnfODFVfXrwEXAC9vKHA2cBhwDnAi8Jck+HdZD2nC80YvG1WXw/zXg0+3zfwSe0j4/Gbiwqm6pqu8CVwPHd1gPSdISXQb/K4Antc+fChzRPj8cuGZgvp1t2Z0kOSvJfJL5hYWFzioqSX0zVvBPcmmSK4Y8TgZ+Hzg7yTbgQOCfF982ZFFDDzxU1daqmququUMPPXScqkqSBox1Seeq2rLCLI8HSPIgYPEozU7u2AsA2ARcO049JG0sO847aUMf0J4GXY72uU9V3ZDkLsA7gcuq6u1JjgH+hqaf//7A/wSOrKrbllueo32k3RkctZLlRvt0eTOX05Oc3T7/EPAOgKranuT9wDeAW4GzVwr8ku7MQK9xeJKXJG1QXtVTkrQbg78k9ZDBX5J6yOAvST1k8JekHupyqKckzZQ+nTth5i9J9O9KqQZ/Seohg78k9ZB9/tKM6lP/tNaemb80g/rWP621Z/CXJEbvNW3UvSm7fWaYu/3Txe9j9vXp+zLzn1Hu9k8Xvw/NGoO/JPWQwV+SesjgL82gvh2c1NrzgK80owz0Goe3cZxh0za6ZLn6TFtdu9CHz6jZstxtHA3+WhOrHdVicJS64z18JUm7MfhLUg95wHeG2ccsabXM/GeUZ5RKGoeZ/5Sblex+x3knjayrP0jS9DH4T7FZy+6n8UdJ0nB2+6hzno0qTR8zf60LA700Xcz8Z5TZtKRxjJX5J3kq8DLgKOD4qpofmHYO8CzgNuC5VfWJtvxE4PXAPsD5VXXeOHXoMwO9pNUaN/O/AjgV+PRgYZKjgdOAY4ATgbck2SfJPsCbgScCRwOnt/NqCLN7SV0ZK/OvqisBkiyddDJwYVXdAnw3ydXA8e20q6vqO+37Lmzn/cY49djIDPSSutBVn//hwDUDr3e2ZaPKh0pyVpL5JPMLCwudVFSS+mjFzD/JpcD9hkw6t6ouHvW2IWXF8B+bkZcVraqtwFZoruq5QlUlSXtoxeBfVVtWsdydwBEDrzcB17bPR5VLktZJV90+HwFOS3K3JA8AjgS+CHwJODLJA5LsR3NQ+CMd1UGSNMK4Qz2fDLwROBS4JMnlVfWEqtqe5P00B3JvBc6uqtva9zwH+ATNUM+3V9X2sT6BJGmveScvSdqgvJOXJGk3Bn9J6iGDvyT1kMFfknrI4C9JPWTwl6QeMvhLUg8Z/CWphwz+ktRDBn9J6iFv4C71zOYXX3KnMm8a1D9m/lKPDAv8y5Vr4zL4S1IPGfwlqYcM/pLUQwZ/Seohg7/UI6NG9Tjap38c6in1jIFeYOYvSb1k8JekHjL4S1IPGfwlqYcM/pLUQwZ/Seohg78k9ZDBX5J6yOAvST1k8JekHjL4S1IPjRX8kzw1yfYkv0gyN1B+7ySfSrIryZuWvOdhSb6e5Ookb0iSceogSdp742b+VwCnAp9eUv4z4D8C/2HIe94KnAUc2T5OHLMOkqS9NFbwr6orq+pbQ8p/XFWfofkRuF2Sw4CDqupzVVXAu4FTxqmDJGnvrXef/+HAzoHXO9uyoZKclWQ+yfzCwkLnlZOkvljxev5JLgXuN2TSuVV18V6ub1j/fo2auaq2AlsB5ubmRs4nSdo7Kwb/qtqyhuvbCWwaeL0JuHYNly9J2gPr2u1TVdcBNyc5oR3lcyawt3sPkqQxjTvU88lJdgKPBC5J8omBaTuA1wDPTLIzydHtpD8CzgeuBr4NfHycOkiS9t5Y9/CtqouAi0ZM2zyifB44dpz1amPZ/OJL7lTmfWalbnmGryZqWOBfrlzS2jD4S1IPGfwlqYcM/pLUQwZ/Seohg78k9ZDBX5J6yOCviRo1nt9x/lK3xjrJS1oLBnpp/Zn5S1IPmflLA7zUhPrCzF9qeakJ9YnBX5J6yOAvST1k8JekHjL4S1IPGfylliecqU8c6ikNMNCrLwz+mmqOu5e6YbePppbj7qXuGPwlqYcM/pLUQwZ/Seohg78k9ZDBX5J6yOAvST1k8NfU8oxbqTue5KWpZqCXumHwlzYgz4zWSsbq9kny1CTbk/wiydxA+W8m2Zbk6+3fxw5Me1hbfnWSNyTJOHWQtDvPjNaeGLfP/wrgVODTS8p/APxOVf068AzgPQPT3gqcBRzZPk4csw6SpL00VrdPVV0JsDR5r6qvDLzcDuyf5G7AvYCDqupz7fveDZwCfHycekiS9s56jPZ5CvCVqroFOBzYOTBtZ1s2VJKzkswnmV9YWOi4mpLUHytm/kkuBe43ZNK5VXXxCu89BngV8PjFoiGz1aj3V9VWYCvA3NzcyPmkaecBWE2bFYN/VW1ZzYKTbAIuAs6sqm+3xTuBTQOzbQKuXc3ypVmx3AFYfwA0KZ0M9UxyMHAJcE5VfXaxvKquS3JzkhOALwBnAm/sog7SrFhur2A1eww7zjvJPQ2tKFWr701J8mSa4H0ocCNweVU9IcmfA+cAVw3M/viquqEdEvpO4O40B3r/pPagEnNzczU/P7/qukqT0sUQSwO59kSSbVU1N2zauKN9LqLp2lla/grgFSPeMw8cO856JWkjmOQemtf2kaQJmPTJeF7eQZKmUNd7BWb+kjRl1mOvwMxf6thKo2/WerSPurHRvouxRvusJ0f7SJqU5TLu1f4ArDaL35v1LTfax24fSeohg78k9ZDBX5ImYLnblK7HLUw94CtJE7JcMO/6YLKZvyStYD0y8fVm5i9Je2CWA/0wZv6S1EMGf0nqIYO/JPWQwV+SesjgL0k9NDPX9kmyAPwY+MGk6zKFDsF2GcW2Gc22GW2jtM0vV9WhwybMTPAHSDI/6iJFfWa7jGbbjGbbjNaHtrHbR5J6yOAvST00a8F/66QrMKVsl9Fsm9Fsm9E2fNvMVJ+/JGltzFrmL0laAwZ/SeqhqQz+SY5I8qkkVybZnuR5bfnLkvzfJJe3j9+adF3XW5L9k3wxyVfbtnl5W/6AJF9IclWS9yXZb9J1XW/LtM07k3x3YLs5btJ1nYQk+yT5SpKPtq97v80sGtI2G36bmcrgD9wKvKCqjgJOAM5OcnQ77bVVdVz7+NjkqjgxtwCPraqHAMcBJyY5AXgVTdscCfwQeNYE6zgpo9oG4IUD283lk6viRD0PuHLgtdvMHZa2DWzwbWYqg39VXVdVX26f30zzpRw+2VpNh2rsal/etX0U8Fjg79rydwGnTKB6E7VM2/Rekk3AScD57evgNgPcuW36YiqD/6Akm4GHAl9oi56T5GtJ3p7knhOr2AS1u6iXAzcA/wh8G7ixqm5tZ9lJT38sl7ZNVS1uN69st5vXJrnbBKs4Ka8D/gz4Rfv63rjNLFraNos29DYz1cE/yQHAB4HnV9VNwFuBX6XZpb8OePUEqzcxVXVbVR0HbAKOB44aNtv61mo6LG2bJMcC5wAPBh4O3At40QSruO6S/DZwQ1VtGyweMmvvtpkRbQM92GamNvgnuStN4H9vVX0IoKqub/+5fwH8NU3g662quhG4jOa4yMFJFm/LuQm4dlL1mgYDbXNi241YVXUL8A76t938BvCkJDuAC2m6e16H2wwMaZskF/Rhm5nK4N/2R74NuLKqXjNQftjAbE8Grljvuk1akkOTHNw+vzuwheaYyKeAf9vO9gzg4snUcHJGtM03F7ebdrs6hZ5tN1V1TlVtqqrNwGnAJ6vqDNxmRrXN0/uwzUzrDdx/A/hd4Ott/y3AS4DT2yFXBewA/nAy1Zuow4B3JdmH5sf7/VX10STfAC5M8grgKzQ/nn0zqm0+meRQmq6Oy4FnT7KSU+RFuM2M8t6Nvs14eQdJ6qGp7PaRJHXL4C9JPWTwl6QeMvhLUg8Z/CWphwz+ktRDBn9J6qH/D6Q4obhLtdK8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.scatter(dirty[\"grid_lat\"], dirty[\"grid_lon\"])\n",
    "plt.title(\"Lat (x-axis) vs. Lon (y-axis)\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Impact Scores"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "date_key                                                0.073965\n",
       "zip5                                                    0.073965\n",
       "impact_score                                            0.112085\n",
       "grid_lat                                                0.073965\n",
       "grid_lon                                                0.073965\n",
       "Time                                                    0.073965\n",
       "state                                                   0.073965\n",
       "Region                                                  0.073965\n",
       "Land_sea_coverage_nearest_neighbor_land1sea0_surface    0.071296\n",
       "Snow_depth_surface                                      0.073142\n",
       "Water_equivalent_of_accumulated_snow_depth_surface      0.074569\n",
       "Categorical_Snow_surface                                0.000000\n",
       "Composite_reflectivity_entire_atmosphere                0.000000\n",
       "Graupel_snow_pellets_hybrid                             0.000000\n",
       "Graupel_snow_pellets_isobaric                           0.000000\n",
       "Snow_mixing_ratio_hybrid                                0.000000\n",
       "Snow_mixing_ratio_isobaric                              0.000000\n",
       "dtype: float64"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty.loc[dirty[\"impact_score\"] == -1].count()/dirty.count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 3063,  8085,  8512,  8817, 18707, 20166, 29418, 30144, 32218,\n",
       "       32221, 33182, 40218, 43207, 48150, 53703, 53718, 60449, 66102,\n",
       "       66219, 67337, 77090, 80216, 89115, 89434, 95837, 97124, 98421])"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dirty.loc[dirty[\"impact_score\"] == -1][\"zip5\"].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "These 27 zips have atlesat one \"-1\" value. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}