linear_regression_training_flic_bgd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using linear regression to predict the joints\n",
    "***\n",
    "\n",
    "### Datasets: [subset](http://cims.nyu.edu/~tompson/flic_plus.htm) of full [FLIC](http://vision.grasp.upenn.edu/cgi-bin/index.php?n=VideoLearning.FLIC)\n",
    "- Number of all the training images is 17380\n",
    "    - Divided into validation sample 17380 * 20% = 3476\n",
    "    - Divided into training sample 17380 - (17380 * 20%) = 13904\n",
    "- Number of test images is 3548\n",
    "- Number of joints is 7 (x, y)\n",
    "    \n",
    "### Data preprocessing\n",
    "- Using the ground truth coordinates of joints to decide a bounding box (human detector)\n",
    "- Add some noise (padding, shift) to the bounding box by uniform distribution\n",
    "- Crop the original image acoording to the bounding box\n",
    "- Resize the cropped image to 100x100\n",
    "\n",
    "### Algorithm: Linear regression, **y** = **XW** + **b**, and y is a vector coordinates of joints\n",
    "- Model\n",
    "    - Build a linear model for every coordinate (x, y) of joints, i.e., 7*2=14 linear model in total\n",
    "    - Use L2-norm loss (square error) $\\sum_i^n (f(x_i) - y_i)^2$ for each model\n",
    "    - The total loss is the sum of that for every model\n",
    "    - We can regard all the models as a single Neural Network (no hidden layer) model with 14 output for regression\n",
    "- Optimization\n",
    "    - Use Stochastic gradient descent (SGD) to update weights and biases with batch size 32\n",
    "    - Use num_epochs=100 to repeat the training and optimization.\n",
    "- Test\n",
    "    - Compute the loss of validation data at the end of each epoch\n",
    "    - Plot the loss decay based on each epoch\n",
    "    - Plot original, transformed and predicted joints on the image\n",
    "\n",
    "### Some implementation details\n",
    "- Use CPU on Mac OS X 10.10.3\n",
    "\n",
    "- Memeory analysis\n",
    ">If using 220 x 220 size, the number of input features (raw pixels) will be `220*220*3`, which will take ~0.00108 GB in python. So with 17380 training sets, it will take about 17380 * 0.00108 ~= 18GB memory. And if we resize image to 110x110, it will take about ~4.5GB. \n",
    "      "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# A bit of setup\n",
    "import numpy as np\n",
    "import cv2 as cv\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.core.display import clear_output\n",
    "%matplotlib inline\n",
    "plt.rcParams['figure.figsize'] = (15.0, 8.0) # set default size of plots\n",
    "plt.rcParams['image.interpolation'] = 'nearest'\n",
    "plt.rcParams['image.cmap'] = 'gray'\n",
    "\n",
    "# for auto-reloading extenrnal modules\n",
    "# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython\n",
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare for training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# set up fransform parameters to find the bounding box, crop and resize the images\n",
    "# transfrom parameters\n",
    "from my_scripts.transform import Transform\n",
    "pad_inf = 1 # infimum scale\n",
    "pad_sup = 1.5 # supremum scale\n",
    "size = 100 # resize size\n",
    "shift = 3 # radomly shift the bounding box\n",
    "lcn = True # local contrast normalization\n",
    "trans = Transform(pad_inf, pad_sup, size=size, shift=shift, lcn=lcn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# set up the model and trainer for training\n",
    "from my_scripts.regressor_trainer import NetworkTrainer\n",
    "from models.LinearNet_flic import LinearNet_flic\n",
    "\n",
    "data_dir = 'data/FLIC-full' # store image \n",
    "data_info_file = '%s/train_joints.csv' % data_dir # store corrdinates of joints for training data\n",
    "model = LinearNet_flic(size)\n",
    "trainer = NetworkTrainer(data_dir=data_dir, data_info_file=data_info_file,\n",
    "                         model=model, trans=trans, sample_batches=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:42 Time: 0:01:42\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:25 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "17380\n",
      "Number of training data 13904\n",
      "Number of validation data 3476\n",
      "{'weight_decay': 0.0001, 'data_dir': 'data/FLIC-full', 'verbose': True, 'data_info_file': 'data/FLIC-full/train_joints.csv', 'sample_batches': False, 'learning_rate': 0.0001, 'update': 'SGD', 'batch_size': 32, 'reg': 0.0, 'self': <my_scripts.regressor_trainer.NetworkTrainer object at 0x10ddb1b10>, 'snapshot': 5, 'val_rate': 0.2, 'num_epochs': 100, 'gpu': -1, 'model': <models.LinearNet_flic.LinearNet_flic object at 0x10722ea50>, 'trans': <my_scripts.transform.Transform object at 0x1042bf250>, 'dropout': 1.0, 'momentum': 0, 'num_iters': 50}\n",
      "epoch01\ttraining mean loss =0.797548132653\t elapsed time=321.025662184 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:29 Time: 0:03:29\n",
      " 99% (3468 of 3476) |#################### | Elapsed Time: 0:00:27 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch01\tvalidation mean loss =0.668794333338\t elapsed time=39.0921449661 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:13 ETA:  0:00:08"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch02\ttraining mean loss =0.614660926029\t elapsed time=308.228049994 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:17 Time: 0:03:17\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch02\tvalidation mean loss =0.574336771872\t elapsed time=39.0559251308 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:09 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch03\ttraining mean loss =0.543887273833\t elapsed time=303.805325031 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:13 Time: 0:03:13\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch03\tvalidation mean loss =0.519228755229\t elapsed time=39.2202398777 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:45 Time: 0:01:45\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:13 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch04\ttraining mean loss =0.499167099234\t elapsed time=308.109660149 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:17 Time: 0:03:17\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch04\tvalidation mean loss =0.482527149598\t elapsed time=39.1056890488 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:05 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch05\ttraining mean loss =0.470447381495\t elapsed time=298.556098938 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:09 Time: 0:03:09\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch05\tvalidation mean loss =0.452868167511\t elapsed time=38.7169289589 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:09 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch06\ttraining mean loss =0.444984172591\t elapsed time=302.974040985 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:13 Time: 0:03:13\n",
      " 99% (3461 of 3476) |#################### | Elapsed Time: 0:00:27 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch06\tvalidation mean loss =0.43469514717\t elapsed time=39.0021319389 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:06 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch07\ttraining mean loss =0.426281294014\t elapsed time=299.626476049 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:10 Time: 0:03:10\n",
      " 99% (3463 of 3476) |#################### | Elapsed Time: 0:00:27 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch07\tvalidation mean loss =0.413219984269\t elapsed time=39.1858799458 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:04 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch08\ttraining mean loss =0.40755275833\t elapsed time=298.068621874 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:08 Time: 0:03:08\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch08\tvalidation mean loss =0.400583902576\t elapsed time=39.0812380314 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:05 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch09\ttraining mean loss =0.392244993591\t elapsed time=298.957993984 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:09 Time: 0:03:09\n",
      " 99% (3462 of 3476) |#################### | Elapsed Time: 0:00:27 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch09\tvalidation mean loss =0.388702346055\t elapsed time=38.9343230724 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:08 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch10\ttraining mean loss =0.382021874166\t elapsed time=301.851245165 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:12 Time: 0:03:12\n",
      " 99% (3467 of 3476) |#################### | Elapsed Time: 0:00:27 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch10\tvalidation mean loss =0.37855781594\t elapsed time=39.1570339203 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:06 ETA:  0:00:08"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch11\ttraining mean loss =0.374664832276\t elapsed time=299.02402401 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:10 Time: 0:03:10\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:29 Time: 0:00:29"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch11\tvalidation mean loss =0.368013220266\t elapsed time=41.3890359402 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:53 Time: 0:01:53\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:09 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch12\ttraining mean loss =0.36339354373\t elapsed time=311.511734009 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:13 Time: 0:03:13\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch12\tvalidation mean loss =0.360386237435\t elapsed time=40.4307000637 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:07 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch13\ttraining mean loss =0.357398540877\t elapsed time=305.533961773 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:11 Time: 0:03:11\n",
      " 99% (3462 of 3476) |#################### | Elapsed Time: 0:00:28 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch13\tvalidation mean loss =0.351903504343\t elapsed time=40.0493500233 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:09 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch14\ttraining mean loss =0.34908116007\t elapsed time=307.855800867 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:13 Time: 0:03:13\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch14\tvalidation mean loss =0.342815160247\t elapsed time=40.4054939747 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:05 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch15\ttraining mean loss =0.342643401815\t elapsed time=304.39023304 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:10 Time: 0:03:10\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch15\tvalidation mean loss =0.338225573989\t elapsed time=40.220854044 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:48 Time: 0:01:48\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:14 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch16\ttraining mean loss =0.337519430015\t elapsed time=314.786942005 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:18 Time: 0:03:18\n",
      " 99% (3475 of 3476) |#################### | Elapsed Time: 0:00:28 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch16\tvalidation mean loss =0.333744359014\t elapsed time=40.2661201954 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:44 Time: 0:01:44\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:09 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch17\ttraining mean loss =0.332114938204\t elapsed time=304.723583937 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:13 Time: 0:03:13\n",
      " 99% (3475 of 3476) |#################### | Elapsed Time: 0:00:27 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch17\tvalidation mean loss =0.32619060226\t elapsed time=39.8819289207 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:27 Time: 0:00:27\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:06 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch18\ttraining mean loss =0.32432940726\t elapsed time=304.67246604 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:10 Time: 0:03:10\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch18\tvalidation mean loss =0.323448662726\t elapsed time=40.2000379562 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:09 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch19\ttraining mean loss =0.321253278259\t elapsed time=307.653508902 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:13 Time: 0:03:13\n",
      " 99% (3462 of 3476) |#################### | Elapsed Time: 0:00:28 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch19\tvalidation mean loss =0.320392296985\t elapsed time=40.2780919075 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:03:07 ETA:  0:00:07"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch20\ttraining mean loss =0.312935478359\t elapsed time=305.568632126 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:03:10 Time: 0:03:10\n",
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch20\tvalidation mean loss =0.316463581997\t elapsed time=40.2727351189 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Saving 11.0 x 8.0 in image.\n",
      "100% (13904 of 13904) |###################| Elapsed Time: 0:01:49 Time: 0:01:49\n",
      " 96% (48 of 50) |######################## | Elapsed Time: 0:04:38 ETA:  0:00:17"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch21\ttraining mean loss =0.308958769191\t elapsed time=406.739904165 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (50 of 50) |#########################| Elapsed Time: 0:04:53 Time: 0:04:53\n",
      " 99% (3475 of 3476) |#################### | Elapsed Time: 0:00:28 ETA:  0:00:00"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "epoch21\tvalidation mean loss =0.310793039234\t elapsed time=40.5321040154 sec."
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (3476 of 3476) |#####################| Elapsed Time: 0:00:28 Time: 0:00:28\n",
      "Saving 11.0 x 8.0 in image.\n",
      " 36% (5066 of 13904) |#######             | Elapsed Time: 0:00:45 ETA:  0:01:13"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\r",
      " 36% (5079 of 13904) |#######             | Elapsed Time: 0:00:45 ETA:  0:01:12"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-4-364b34aa3864>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mbest_model\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtrain_loss_history\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mval_loss_history\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrainer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrain_bgd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m/Users/houxianxu/Documents/PhD/Implementation/deeppose/my_scripts/regressor_trainer.pyc\u001b[0m in \u001b[0;36mtrain_bgd\u001b[0;34m(self)\u001b[0m\n\u001b[1;32m    371\u001b[0m         \u001b[0;32mfor\u001b[0m \u001b[0mepoch\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mxrange\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnum_epochs\u001b[0m \u001b[0;34m+\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    372\u001b[0m             \u001b[0mst\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtime\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtime\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 373\u001b[0;31m             \u001b[0minput_data\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mall_data_load\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mtrain_info\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mresult_dir\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    374\u001b[0m             \u001b[0mmean_loss\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcompute_loss_model_update_bgd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_data\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mlabel\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mnum_iters\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    375\u001b[0m             \u001b[0mtrain_loss_history\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mappend\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mmean_loss\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/Users/houxianxu/Documents/PhD/Implementation/deeppose/my_scripts/regressor_trainer.pyc\u001b[0m in \u001b[0;36mall_data_load\u001b[0;34m(self, data_info, result_dir)\u001b[0m\n\u001b[1;32m    432\u001b[0m             \u001b[0mimg_file_name\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'%s/images/%s'\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mdata_dir\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    433\u001b[0m             \u001b[0mjoints\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0marray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfloat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mp\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0mp\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mx\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 434\u001b[0;31m             \u001b[0mx_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcv\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mimread\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mimg_file_name\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    435\u001b[0m             x_data_trans, joints_trans = trans.transform(img_file_name, x_data,\n\u001b[1;32m    436\u001b[0m                                                          joints, result_dir)\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "best_model, train_loss_history, val_loss_history = trainer.train_bgd()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}