SensiML Basics - Building Models Directly from Feature Vectors.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "#os.environ['http_proxy'] = ''\n",
    "#os.environ['https_proxy'] = ''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sensiml import SensiML\n",
    "\n",
    "client = SensiML()\n",
    "client.project='Model Building Demo'\n",
    "client.pipeline='feature_vector_to_model'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare the feature vectors and upload the data to SensiML server\n",
    "\n",
    "This data set consists of many reps which have 128 feature vectors each as well as a label column containing the ground truth and the rep number. We drop the rep number because we don't want that as a feature vector. Note: All feature vectors must be integers scaled between 0 and 254. \n",
    "\n",
    "The model generator requires that the ground truth column has the name \"label\", so we rename our ground truth column."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Uploading file \"feature_vectors_128.csv\" to KB Cloud.\n",
      "Upload of file \"feature_vectors_128.csv\"  to KB Cloud completed.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<sensiml.client.SensiML at 0x253670453c8>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df= pd.read_csv('Support/scaled_reducedFeatures_128.csv')\n",
    "# Rename the 0 to label and drop the columns we don't need\n",
    "df = df.rename(columns={'0':'label'}).drop(['1','130'], axis=1)\n",
    "\n",
    "\n",
    "client.upload_dataframe('feature_vectors_128.csv', df, )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reset the pipeline and set the input data to be the file we just uploaded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Group columns must be a list of strings.\n"
     ]
    }
   ],
   "source": [
    "client.pipeline.reset()\n",
    "client.pipeline.set_input_data('feature_vectors_128.csv', label_column='label')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model TVO description\n",
    "\n",
    "* train_validate_optimze (tvo) : This step defines the model validation methodology, the classification method to use and the training algorithm to train the classifier with. \n",
    "\n",
    "A tvo step is composed of a \n",
    "* Classifier\n",
    "* Training algorithm \n",
    "* Validation method\n",
    "\n",
    "For this pipeline we use the validation method \"Stratified K-Fold Cross-Validation\". Which splits the data into 5 folds and iteratively trains on 4 folds, test on 1 fold.  The training algorithm Hierarchical Clustering with Neuron Optimization uses a clustering algorithm to optimize neurons placement in feature space. After the model has been trained, neurons are loaded into the pvp classifier and model validation is performed using the selected validation method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "client.pipeline.set_validation_method('Stratified K-Fold Cross-Validation', params={'number_of_folds':5})\n",
    "\n",
    "client.pipeline.set_classifier('PME', params={\"classification_mode\":'RBF','distance_mode':'L1'})\n",
    "\n",
    "client.pipeline.set_training_algorithm('Hierarchical Clustering with Neuron Optimization',\n",
    "                                    params = {'number_of_neurons':7,\n",
    "                                             'cluster_method':\"DLHC\"})\n",
    "\n",
    "client.pipeline.set_tvo({'validation_seed':0})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Execute the pipeline on kb cloud. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Executing Pipeline with Steps:\n",
      "\n",
      "------------------------------------------------------------------------\n",
      " 0.     Name: feature_vectors_128.csv   \t\tType: featurefile              \n",
      "------------------------------------------------------------------------\n",
      "------------------------------------------------------------------------\n",
      " 1.     Name: tvo                       \t\tType: tvo                      \n",
      "------------------------------------------------------------------------\n",
      "\tClassifier: PVP\n",
      "\t\tParam: classification_mode: RBF\n",
      "\t\tParam: distance_mode: L1\n",
      "\t\tParam: reinforcement_learning: False\n",
      "\t\tParam: reserved_patterns: 0\n",
      "\n",
      "\tTraining Algo: Hierarchical Clustering with Neuron Optimization\n",
      "\t\tParam: cluster_method: DLHC\n",
      "\t\tParam: number_of_neurons: 7\n",
      "\n",
      "\tValidation Method: Stratified K-Fold Cross-Validation\n",
      "\t\tParam: number_of_folds: 5\n",
      "\n",
      "------------------------------------------------------------------------\n",
      "\n",
      "\n",
      "Checking Pipeline Status:\n",
      "\n",
      "\n",
      "Status: Running  Time:   5 sec.  Step:  1  Name: tvo              Type: tvo .\n",
      "Retrieving page 1 of 1.\n",
      "\n",
      "\n",
      "Results Retrieved... Execution Time: 0 min. 31 sec.\n"
     ]
    }
   ],
   "source": [
    "results, stats = client.pipeline.execute()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TRAINING ALGORITHM: Hierarchical Clustering with Neuron Optimization\n",
      "VALIDATION METHOD:  Stratified K-Fold Cross-Validation\n",
      "CLASSIFIER:         PVP\n",
      "\n",
      "AVERAGE METRICS:\n",
      "F1-SCORE:    70.3   sigma 3.18\n",
      "SENSITIVITY: 70.6   sigma 3.51\n",
      "PRECISION:   87.1   sigma 2.08\n",
      "\n",
      "--------------------------------------\n",
      "\n",
      "STRATIFIED K-FOLD CROSS-VALIDATION MODEL RESULTS\n",
      "\n",
      "MODEL INDEX: Fold 4\n",
      "ACCURACY: 54.55\n",
      "NEURONS: 7\n",
      "\n",
      "MODEL INDEX: Fold 2\n",
      "ACCURACY: 49.09\n",
      "NEURONS: 7\n",
      "\n",
      "MODEL INDEX: Fold 3\n",
      "ACCURACY: 40.00\n",
      "NEURONS: 7\n",
      "\n",
      "MODEL INDEX: Fold 0\n",
      "ACCURACY: 40.00\n",
      "NEURONS: 7\n",
      "\n",
      "MODEL INDEX: Fold 1\n",
      "ACCURACY: 38.18\n",
      "NEURONS: 7\n",
      "\n"
     ]
    }
   ],
   "source": [
    "results.summarize()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}