digit_recognition-mnist-sequence.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep Learning Project\n",
    "## Build a Digit Recognition Program"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this project, we will design and implement a deep learning model that learns to recognize sequences of digits. We will train the model using synthetic data generated by concatenating character images from [MNIST](http://yann.lecun.com/exdb/mnist/). \n",
    "\n",
    "To produce a synthetic sequence of digits for testing, we will limit the to sequences to up to five digits, and use five classifiers on top of your deep network. We will incorporate an additional ‘blank’ character to account for shorter number sequences.\n",
    "\n",
    "We will use ** Keras ** to implement the model. You can read more about Keras at [keras.io](https://keras.io/)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementation\n",
    "\n",
    "Let's start by importing the modules we'll require fot this project."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n",
      "/Users/sajal/anaconda/envs/py27/lib/python2.7/site-packages/matplotlib/font_manager.py:273: UserWarning: Matplotlib is building the font cache using fc-list. This may take a moment.\n",
      "  warnings.warn('Matplotlib is building the font cache using fc-list. This may take a moment.')\n"
     ]
    }
   ],
   "source": [
    "#Module Imports\n",
    "from __future__ import print_function\n",
    "import random\n",
    "from os import listdir\n",
    "import glob\n",
    "\n",
    "import numpy as np\n",
    "from scipy import misc\n",
    "import tensorflow as tf\n",
    "import h5py\n",
    "\n",
    "from keras.datasets import mnist\n",
    "from keras.utils import np_utils\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Setting the random seed so that the results are reproducible. \n",
    "random.seed(101)\n",
    "\n",
    "#Setting variables for MNIST image dimensions\n",
    "mnist_image_height = 28\n",
    "mnist_image_width = 28"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Import MNIST data from keras\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Shape of training dataset: (60000, 28, 28)\n",
      "Shape of test dataset: (10000, 28, 28)\n",
      "Label for image: 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFfCAYAAACfj30KAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJztnX2MvVtV3797Zs7L/Ob3+92bQnMx0lT02ihpNBZbSpRy\nW5po+QPxH4w1QWpMgy+NMakSElIQTIwaDI1KYxoLmlYTEm1RA1wr4gsioigqGiXQi6hwryAv9/cy\nZ86cmd0/Zta566xZa+99zpwzzzlnvp9k59nPPm/P85yZ71nP2mutnXLOIIQQ0g07XR8AIYRcZyjC\nhBDSIRRhQgjpEIowIYR0CEWYEEI6hCJMCCEdQhEmhJAOoQgTQkiHUIQJIaRD9ro+gJTS0wB8HYCP\nARh1ezSEELIUhgC+CMCjOee/Lz1xZSKcUvouAP8ZwDMA/DGA/5Rz/n3nqV8H4H+t6jgIIaRDvgXA\nz5WesBJ3RErpmwC8AcBrAHwVzkT40ZTS052nf2wVx0AIIWvAx2pPWJVP+HsB/FTO+Wdzzn8B4BUA\n7gP4Nue5dEEQQraVqr4tXYRTSj0AzwHwLhnLZ6Xafg3A85b9eYQQssmswhJ+OoBdAE+Y8Sdw5h8m\nhBByDkPUCCGkQ1Yhwp8GcALgITP+EIDHV/B5hBCysSxdhHPOxwA+AOCFMpZSSuf771325xFCyCaz\nqjjhHwPwlpTSBwC8H2fREjcAvGVFn0cIIRvJSkQ45/zW85jg1+HMDfFBAF+Xc/7UKj6PEEI2ldT1\nQp8ppX+GM/cFIYRsG8/JOf9h6QmMjiCEkA6hCBNCSIdQhAkhpEMowoQQ0iEUYUII6RCKMCGEdAhF\nmBBCOoQiTAghHUIRJoSQDqEIE0JIh1CECSGkQyjChBDSIRRhQgjpEIowIYR0CEWYEEI6hCJMCCEd\nQhEmhJAOoQgTQkiHUIQJIaRDKMKEENIhFGFCCOkQijAhhHQIRZgQQjqEIkwIIR1CESaEkA6hCBNC\nSIdQhAkhpEMowoQQ0iEUYUII6RCKMCGEdAhFmBBCOoQiTAghHUIRJoSQDqEIE0JIh1CECSGkQyjC\nhBDSIRRhQgjpEIowIYR0yF7XB0DIJpJScrfR2Lzv2/JYzjnc2rHSMeutdx6l/rx4x1U6j5btpkMR\nJhtLJAaXEYlWdnZ2pqKl+95+6zHNI+wpJZyens60nHO4L8e0s7NT7XvnELUW7I+DHFdpGz3mjW06\nFGGycZSssmVZbDV2d3enwmWbfax0/HbMWqVWyPX+yclJU5PjlSbHGI3pY9cC7e17RBaqiOjJycnM\n1htr2Z6cnCCltPEWMUWYbBStVuKqj8ETsr29PVfcvGOK9ksWtW2TyQTHx8eYTCYXmozL+8rxtTQt\nyrWtxoqh3dc/DJPJpNj3tpPJBDs7O5hMJtP3166XTYUiTDaCFtGNfJurOJaaqGlRrvlW9Zi2Nq0Q\n27HxeIzj4+PpVvf1NTg9PcXe3h56vV5Ts8df+qGRz/B8vHZfxFT/cNT6cl4i/FqAd3Z2ppb+JkMR\nJmtNze/r3a6XXrcMxAoUYbMCp/f39vYuHE/tx6PFHSAifHR0hKOjo2lfW9/AmQBrAe33+xfaYDCY\n2bdCXGrWHVDqn5ycuD8Yti/74/F4ehzywyLvJS6Jq/D/r5qli3BK6TUAXmOG/yLn/OxlfxbZbiLf\n6Tx+01WgXREibL1eb0bEpK9FuMWCj/zMXhuNRhiNRuj1ehcEWG7TxfrUPwyDwWDahsPhhX70Y+L1\ntQjXIh4mk8lUXOWHw9s/Ojqafs54PJ7+6Mj7yA9L5JPeNFZlCX8IwAsByH/CZEWfQ7aUmoiWZutX\nLcTWEhZh01albEWoSi4TvdUTY7X+4eHhjNXqWYraf6t/HIbDIYbDIfb39y9s5bjl+Xpr+/oa10LJ\njo+PMRqNpta7NDtmf1S88xJ/Ny3hmEnO+VMrem9yDbG38y1tVYgYyi25vq0Xa1K2/X5/5pijvnZF\nRBELNnpBC7AVK+1/tVa7HN/+/j5u3LiBGzduTPv7+/sz7gnrqrDN+oRLMcvj8XhqvY9GIxweHs7s\ny5hcV+0P1yFqk8mElnADX5pS+lsAIwC/C+BVOee/XtFnkS2jFElQclFclRB7lrC2LrWVKUIVWen2\neO1EWNREhMUC1pNwJycnU9+qiJm12rUIHxwc4ODgYNrXPyDadeGNaRGuJY2Mx2McHh7i/v37ODw8\nvNDkR6Vk2YtrRfvIN51ViPD7ALwcwF8C+AIArwXwWymlf5pzvreCzyPXkJLwrvqf0/MJa7/q/v7+\ntIlQ1dwmWoR1BIKNRoiiEwDMCJX4V+1rtSUsVvDBwQFu3rw5bfIj4ln29jHrEy71j46OcP/+/QtN\n/NAtAiyTdvbHZ5NZugjnnB9Vux9KKb0fwF8BeCmANy/788jyWOQPOnrNPOOtlq8ntFH21yr/Qff2\n9qYia2/l7Zi2FmsWuxX3Un9nZ2cmO07H0npRBvpHQv9QaEtYRFj8w/r5JREGZmN2vQy5nDP6/f6F\nH0gtsjpWWI7di19e9fd71aw8RC3n/PmU0ocBPLzqzyKzlMSs9vzS4zX/5qJ973PsWGuW2iqt4d3d\nXXdCy9sOBoOZcy25JvS5RNlt+lytL1oSG3R6r2CtXRFecUPI8WpXhA1Xs5Z3KR3ZttPT05kfB/0D\nYUPV9JhOQLHntulJGsLKRTildBNnAvyzq/4s8hQlofNEzr4ues+WmgLRc2qvLR2jZy3W+qsU4dpt\nuhazSHS9fulHxY6JL3owGOD4+HhqSUYiLEJsxVc3HbJmRdj+uMn71+pAyNaKcCTINmZYZ9TJ+W1L\nthywmjjhHwXwyzhzQXwhgB8AcAzg55f9WcQnuv2NxqLX27FWN4CXaFDqRyLs9b1JqshvukoRthNU\n0b4XolY619Zrl1KaWqvaUjw5OZkRJ3lv7XYQAdaREdYS1jHBNgLDuiBqxYOklUTYE2VrBesfmG0o\n3COswhJ+JoCfA/A0AJ8C8B4A/zLn/Pcr+CwSULNWS0Jsx2qugFb3QPRY6Rbd9nUWl83osvurFGEd\nc+s1HU+rr2HN6p/n7kE+w96uawGW672/vz8TAaG3VoglycQTYR0Wpt0RthCP1zyfdc094bkjtskK\nBlYzMffNy35PshjRP7MnfPZ13v48hV0890DkPihZ6nZcBFDEodRflQiLK6C1DoN3nVvcL6UxANNw\nMyu++rlahLX1W7KEJfRNC7D3XWlLuKU6Wk18o326I8hG4gmwt9XPr/VbY1hr7gIv5rXmNpEW1Wvw\n2ipFWFvcWvy9Vrqu0bVuIXJByDHq669D0qKoDhFh/X15SSIppQuTblp0bblJ2dei2iLGWoDtxJz9\n0dlkKMJbiCdm1rdYsoYjsbCiOs+2NNYy2adFuJRWqx/b3Z0ttbjM69saxyvHULrbsI/V0n91yJcV\nJH3HIK3X612IX9bNCnHJlaSP1Qpxra6xJ76lSnCeJczoCLJReP7F0j9V7bY5Ktc4b7OWY0mE7WNa\nfG0lMDu+ShG2/m0tXJ7P23sPbz8K8YrCvjwB1pa6zebTYmu30uwdU+TGinzBXt3gki+4JMrWH+y5\nXjYdivCWUhJgO8lS809KX4uu54u1Y3Y8qsZV+4fXfR2WZbd2bFUiDMR1f70ohuj78Whd0scTYOuC\n0CI8GAwuiK3XBoPBzHGXfqRL7gidPFITYSu8tYk5+oTJ2tPiipB/WPv80tYKrFfacJFm3SOeJSb9\nUilG21YpwvYal/q19xC8sC4rcF6Ilr5G2gXR7/engtYS0yxb63LQWztWEmAtxF6h9kiISxNz9gdp\nG6AIbxFaAPQ/ZMuEmH29bHW/JKTaF1sa9/y5kVXpjdl4XCsgenyVItxKznk6kQVg2tdjmmhiy76n\nvJf+QdXWob6rkLhmG0Jns+G8iTf5vCgbLhJab0yXq7QF6UuJGqXoiG2AIrxBtFis4rttCeOy4VNR\nX7aRiyEaK1nJejxyPXiCrP2++lxqPtiusGLrWZYaETt7G+6tIzfPUkFW2G1EwtHR0fQHTo6jJR25\ndGy2jUYj3Lt3b6ZJBTUpZSnCHAkx3RGkMzyRjCayrFBFKz/oMK7abbWIe2toVkn4rWjPMzFnoyOs\nJdfiBrhK5hEJG2VQ8qOWFvn0WlTkp9frTQVYWjQJ6O17roeoPxqN3CpqIsLWMmbGHFlLalbjzs6O\nWwc2qgsLxFlqdsyLipgngy0aK/2o2HOVW2trVVshXhcit0OEtoS9mgradxpZy17fCvB4PA7dSPo4\nvNTkKCzN+zy9lVKWYv3ausKj0Sgs3uP5hbfBCgYowhuDJ0h2Jl58g6XZcFsjIBJcr9XiYUsxs15f\nttHneccik07WurY+zXUR4nmFQoubFky7JNDR0dGMMNX68l6liBXZ1oTXWsLe53pjspSRFl27uoa2\nhL33ZJww6RTP6rUhZ3K7LkIbZUjduHFjWg+2VYi9LKqW/dp49LnecdhstXV3R8xjCZcE2C4DNBqN\nqskRusmP3vHxcTXOuyVEzlrCLckadnkjK77aHaGrwtnP2DYhpghvCCXx1VsRYbGEpXCLraAl2xZX\ngP1sm4xQ2q8lM4gI63OUvjcmIuxZ2Os4MTePAMtW11rQImyXA7Li56UKWxH2fgTtuBf/6wlyKVXZ\nO5bIoo9EOKpHsU2TcgBFeKPQYhhZljo4XyxhXdD71q1b074W4ZbUYRuB4GXfeVl5tcfk3Fq2+py9\n67CJlrAnwNZ/KyKsJ7SiYjle37te0daLV47GrEh6xyDb4+PjmfC0aLl78XeXYqa3RYABivBGEAmi\nZ9HYDCkR4Vu3buHWrVu4ffv2tH/z5k3X11yaGJu33zIm56jPtzQWifkm+oTt45ElLBbj/fv3p+Fd\nJZGyrfUHcWdnJxS9eRJJvP3JZOJmx3n9yWQSukCsn3rToQhvEN4/j01T9XzCYvnevn0bDzzwwHR7\n69atqgiXhFmOqdRanyPPs+cbXYPaD8Qm4AmIZwlbd4QnwpG7QPql79Tue+6GeQS/JNQt9SNkMs4L\nlbNtG6AIbwiRb9YKcMkSvn37Nm7fvo0HH3wQDz74IG7fvl0NebMWqxyL3raO1Z7vnXPpWtj3aXnP\ndcETEO2LtSFq1h1x9+7dYtSCZy22XLOUUtXKLbkISvst2XUyfnp6OnOd5kl42TQowhuEZ6V6VrH1\nC3ti/MADD+CBBx4oug2sC2HTWMY/afQelxH6SIBlq8O7tDtCRPjevXu4e/du0UK8zARWiwh7dRys\n+HqP1WKJpb8tAtsCRXiD8P7RvH8ML67Ss1rkPcQPKEJrLadNx1pOeuv5ZEvvIUTWZGT9t3yO/k69\nyTkbrtZyu74MEW6xfktWuPRL8b7b5mKYB4rwBhFZPFaI7Qy1FWP9Oi3AWpzENyj9TWUekfJueaP9\nFp+5vqYt7zuPCB8eHl74MWk5rxZKLoWS1VuyxEt/j9Ztct2gCG8I9h/Oiqj9A2/14wGYisU2ia9Q\nssw88ZDX6NfbMWC2nnDU19fTHlPUt3cyesLKxtbaY4725xW3kjuhdP0iy1Y/Xku6uI5CTBHeICIh\nTinh5ORkOqlSitn0rA/btkF8gfh6RRadfb59D9lG/nh5nRd2p98j6luxanVHeMfs9ee5bpEQtwpu\nJMI199h1hCK8YXh/2HL761nCNYtYo63gy1hS64QnAN6PU2RJen0RYZ00knOeqV8s19JawqW+tYR1\ntTMbL3x4eOgem90u8t2VhLYmvvr1djzyJW/T39siUIQ3gJJlYf2QrS4J/Xr7ftv0j2AtuSittlVQ\nhL29vem19IrHaxePvI8+Jm9M/1i0WsKl97Nj81yzeZt+nX0Pe352S0uYbBRWMLUQA7ggNCUxlvcB\nLlrB8lmbTmQJe8VhWgVHrrsUu9HXSb4L8dfLMXhbOzavNTyPwC963Wo/SN7fS2lb8iMveqybDkV4\ng4gsYe1WqEVGeO6IyA0hn7mp2H9+66bRMaott8hWhO21slln8wiUPs553BHee3jXYNHrZrfzno/3\n+pqwXzcowhtCJAjWr1uLEbZWoU5T1f7LbfmnsNfKipxeeaLVB+qJrJ6sq8361yzFVneEfV973pe5\nZtGxRmO1/VZB34a/uXmhCG8gnvgKJavXExrtzthGi6QkwFrorEui1LchfV4G4zxWoxWhFmtYW8It\n12AZ13Ge9235cViG1b4NUIQ3CO9WTotxztn1d5Ya8JQVV7sNXxXzhMTNcxw1C1jH4MrknDdxZMVY\n3zHozDgdrlayhqOtd5yRO+Lo6Kj5OpD1hiK8IdjbYBmTrVho+lbWW5lB6g8MBoMLywyVliDSEQAt\nfsB5qnYtej2i45G+1F7wml1qXeoVtAjxzo6/mKrXtytal7YnJyczxxWtNEy2C4rwBmKtsZolJaJz\neHg4XeRTlpovLcJp9z3LOBrTVqHe2rFFRLjl83POoQXp7UeTc5El7K1k7W31atK12sqnp6euCHsR\nHGR7oAhvENrK1P+I4tcVl8Lp6alrCR8eHl4Qh9py9Z4Il8KMpNkym9qqzjlPLcRFq7N57hJ7HPYH\nSN8N2EUmpXxizScsIiyLjerVir2+XYS01BcRPjo6mqmt600cku2BIryBRP+EdmbdirC9TdYiXFuF\nVy8AWQq4l619j16vN42tBbCwAEfC6x2DXhpIVqXQS63rJdfH43H1B8azhEut3+9fWIy0tCI1gKol\nTHfE9kER3jBqVlAkwiLEvV5vaoEBmFk+XguIt1+LuND72jdqi7XYWgvzuiTshFt0PNYSllUp9BJB\n0hcRbrH0tSUcNX39vLsLbx94SoS9lSZoCW8nFOENwc7G239EEQcrwtoa1LfG8j41S05uh3u9XnER\nRzs2GAwwGAwuCEhKaWr5XcaqsyLpHYu9E9CrUty9e3daHP3u3bs4Ojqqujf0xFzLj5Z3PaOxyWSC\nlNLMisN6Uo4ivL1QhDcMzy+srUkb6C8iZH2T8no7kWSFQYtxVHfBG7Nrhckx7+7uYjKZTN0bi14D\nawl7x6HLP4oPWIT3zp070+2dO3dwdHTkiq4nyGIJey4Xb0zuCGRSVLfj4+Npf3d3N4yOsFY+2R4o\nwhtI9E8oAqFFaDwez/gdtQCfnp5eCKvSwjCZTKbuBC3CNuHBG/MEWPugF/VveuLo/Sjo89eheeKG\nuHv3Lp588slp06UhS2Is19i6FDy3g4iw3BVIGw6H7p2CFmHritCWMNkuKMIbhLZ4S+4IGx2hBVie\nm/NZYodYaNoHqd0QVmC9tcGi9cJEZHXpRxF8LdCLXovIGpbP96JDxB1x584dPPnkk/j85z+Pz33u\nc8XlgiKfcC2qRERYVr6WNh6PMRwOL/h79/b2muKEKcTbBUV4wyj9A8pj2iesXRDyHO0zHQwGGI/H\n01tlEWQrrrVmnyciqwVYhGkwGFzKv1mygvXn24k5vUimFuDPfe5z0+WCalawtoSjZq3h/f39mWYj\nH+Qze73ejE9YZ/PRJ7y9UIQ3HPsPaSfmrA/YC2MT4ZXbY1tdTARAi7P0ozEdu6wF2IrPoudcsoRF\ngG2ImvUJixB/9rOfnYpwSxMRtskstu3u7qLf7+PGjRszwmrFF8DUjeS5I/QdCQV4+6AIbyGe2NpJ\nOXle6ZZeRKDf78/M1lvR9VqUCOJNXEmMrE1ltgXr9TF7Vca0S+X4+HgaB6wz42q3+brpz7PjEiVh\nr50OtxPBti4h7/qfnJyg1+vhzp0709C5w8PDmUgJCvF2QhHeQrzbdRFGKxLe82xxG5m8i1wT3riI\nsGct2kks+wNR2kbWvFcfQmKDxRK2mWj2Ft/zs5du/+31E2tWk1LC8fHx1N9rf1DEfy8RIzp+WY5b\n/3AwYWP7oAhvIS0CHD1PJuX07bwOWatNxsm21RLWt+5ROi+Amey6SIBtXQhrCddqMsg1iQTZ7lu3\nSJR0Ir55uwKK/eHb29ubxjJbS1j7hmkJbxcU4S3E3ubu7OxgMpm4j1shkNjgXq83FeCjo6MZS7gU\npraICItLIkrlBRDewosQ60pxNk1ZW8KRX9aKsO3b62uPxRNgeZ6eGAXi9PK9vb2p9W5/PLQlTBHe\nLijCW4oWqsgHLMK5t7c344bQFqqIw97enhuLGyVttLghZCvHYGtU6EkrXe84ygrUoWi2NkQUf+uJ\nsN62jMkxRc/3XBD6+HU9D/2DEbkjKMLbBUV4y7BCZf2UkQiLEEcz/pJm7GWmef15LGGZlJIiP6en\nZ7UnrPjacxAhs0v/iABHE3M24sC6I/TntF5r7zE5dj1mJz7ljkOusa51rLd0R2wvc4twSun5AL4P\nwHMAfAGAl+Scf8k853UAvh3AgwB+B8B35Jw/cvnDJS1YIfbGxFoVX6TnCtATa7u7uzORBFpwvTEr\n5qX0XpuSK1EF2i+sRdKbRIxE2E5w1dwR8hm2H7kldJSEvs7e4qnWAtY/cHt7Z1XtbISHtd45Mbd9\nLGIJHwD4IICfBvCL9sGU0isBfDeAlwH4GIAfBPBoSunLc87jxQ+VtOLdHutVN2xhdZk4sk0e1wV3\nbEiWHZMWWdSeCFv3A/CUH1WO27v91/HA3uohniXsuSO0gOprVrvGgvYJi/jqZe+9cEHvGovvvpYU\nQ0t4u5hbhHPO7wTwTgBI/nTw9wB4fc75V86f8zIATwB4CYC3Ln6opBUtVLqvQ70mk8m0oE7LKhha\nDK3gemNahGu1iuWYBVtpzVqrNp655o7QPmGvME4pFK0kePYxcf/YWOednZ3pj1+piesocvVY1wnZ\nDpbqE04pPQvAMwC8S8Zyzk+mlH4PwPNAEb4S9Iy9WGRe4oNdIViLtDeuhb221VlyVoyl2LlYxXbh\nTG0dirvCuiMkvtZOzOlJuZboiJJPuHaNdd+uF6f70dp6pWSU2vUl28OyJ+aeASDjzPLVPHH+GLki\nIssOuCgU3tpnXt9ajdba9kQ4qqegRViw4muz2fR52QkuW7LS+oNboyMWuc56G1EqXO/Fb0efQ7YP\nRkdcQ+w/tLaUxX+sfZtiTVsBLrXITWF9yp5f2b6XRv8o6GZ9zrpam5581C4WbYFe9TVvfYxsP8sW\n4ccBJAAPYdYafgjAHy35s8iKiERhHhHW7zPP66L30niFgUR0pU7v/v7+hR8CPcElFvdVCjEhHksV\n4ZzzYymlxwG8EMCfAEBK6TaA5wL4yWV+Flke8/hBS+JYeu9IkFusXz2mLWEtwnr1iuFwODPppl0X\nMpFnS3wS0hWLxAkfAHgYZxYvAHxxSukrAXwm5/zXAN4I4NUppY/gLETt9QD+BsDblnLEZKnoSSU7\n7o1ZS7clqsB7fuRLts/10JEb2g2hi9PbVGovNveqXRKEeCxiCX81gHfjbAIuA3jD+fjPAPi2nPOP\npJRuAPgpnCVr/DaAf5cZI7z2aNHTEQv6sUgoIwtW92uTeiWXhj6uyBLWNZFtGJut0UARJuvCInHC\nvwmgeA+Xc34tgNcudkikC7RFXJtE8lwMtefXJvBqPmXBC2Gzq0Pr+FodxhZVbZP35QQZ6QJGR5AZ\nItdCyTL2trXPmFeINZ4QSySEDjuTjDpJ0tDZet7EHIWYdAFFmFwQ2UiMPBG2ffv8RSIhSr7myB3h\nLRmkRfjo6GimWlk0MUchJlcNRZgA8N0RWpBK/t7W/VZ3RHR88pidmJNax7YA0MnJyVSAB4PBBSs4\n8glTiMlVQhEmU0oTc7Xnl57jWbgtE3OeMHuWsJTC1M+T+hiSRSchbNYSjibmKMTkqqAIE5dlCFDk\nQ/bEuNUiBi6mN9t6CrpI0Wg0wmAwmAqwZw0T0iUUYbIStMDaxUPFRSCRCtq9YKMeZCv1HgBc8P1a\ny1jGxuPxVIC1Jey1yWRy4YdC+naMkGVCESYrQQTYrqM2Go2mwmtdAbqimLV2RYxTSu4knJS/1EKp\nRVcLsde8lZettS7nRcgyoQiTlRCJsA0N89wI2qq1scBSm1eLpl4GSRYHTSldsIBLIiyWsFcjWY6R\nQkxWAUWYrIRIhL36xF5lNG81jn6/P7NShV49RF6v+y0WsBZhW+HNVpDjZB1ZBRRhshI8EbYuCBG8\nmgtCxFQsac9nq+sfixhLllyLNWwz7STETZ8PBZisAoowWQmeCFsBlscAXJiYsxNng8FgWvdBiPzJ\nQqsVrEVYln2yYWu25jIhy4IiTFaGFlodCqaL64g4lyzgwWAwTbjQPmC7PJAd8yzgyBqWCI5oqSG9\nojIhy4QiTFaCtoS1BSk1HUSAJeLB+oC1QEqh9vH4rBCfXYDUW2lD4oTnmZizFrCNY2a1NbIKKMJk\nJWgRtvt7e3vT2r4SKWEF2FrBsj6cPFfQxXe8NepaJuek/KW1gkV8rYVNyDKhCJOVoEXXCpq4J7Q1\na9eG06tkyCKe4/G4ukip+JNFhFt9wmJlW+tXzoGZdWRVUITJStAJDnIrL64JK5xahLX7YTAYYDQa\nYTgcTstR2tfa6AhtUYsIW+s6anLcWoRFgGkFk1VBESYrQSc4aJ+wxN3q23udyqyXrR8MBtOtiKie\n1PPSlqNUZqk5rC3s4XCIGzduYDweI+c8U/Rd15ew+xLRoc+ztt+6JdcPijC5MiKh0VEUWowPDw8v\nVD2TcLfBYDAjxiLoIpQ2nVmvQyerMeu16HZ3d6efLVZ3tC8i7FWI8/o6HVpn4zElmgAUYXIFaEG0\nYwAuxBNLjQkrwACmoWRe2rK4I8Rils/U4/1+H8PhcFoQSER8b2/PFV4rwuPxeGYl51qzvmVvX64B\nRfh6QhEmK0VEUvoeniXs1f0FcKFuBIALk3Keq8KuyGxFPBJhb0yLcMtWzk2vAG2z8hiDfH2hCJMr\nwYqxzjwTodKlLkej0YwAa0HzqqdFyxzZx8US1rUnZGKwJMLWHaEtWtvsxJ5Y77rZUDhO+l1fKMJk\nZWixLaX72prDR0dHzQJsXQ3aStXP2d3dvVCyEsDMYyVfsG7Hx8cX3Aulvvy46PoZ+hpRhK83FGFy\nJZQE2boj9AKcWoBF/LQfWFwQ/X5/xs+rIzDsWnRagGW8JLqeCGu3iHUz2DEpPGRD3SR70AozuV5Q\nhMlKKbn/igOWAAAgAElEQVQhrJUrlqJnAYuVnHOeyYyzk21exISIrRVwHVMsadHzinDk79UFgfQ5\nCXJeEvJGEb6+UITJyqlNztlCP5EAHx0dTQu3awGW1OaaJWxdEHrpJPFFtwjweDy+ILSlvrXsdf0M\nW9yIXD8owuRK0OJrBVmLbSTA4/EYvV4PAGaSL2xtCesT1qFrsi8CrEtYlkTYPiafo/3Y3uSbLuEp\n52sn7Gz0B7l+UITJlWOtYa/Qj0zW6UI/IqTaBTEcDrG/vz8jjjr+VixhaxXbiAbxR9cEWMbks7QA\n2zF5rT4vbQFHa+2R6wVFmHSOjhDwxFj7Tnu9Hg4PD6dtf38fo9HIbTZxQhAx1rUnRBDtis9SSF7X\nOPbE1tvqiUYrslEihz7OlhRnJnhsPhRhshZ4mWaehahdB7rOxP3792dqTOzt7WEwGMyUutTv5/W1\n68KuX2cflx+Emkvi+Ph4Kupa5HXdY1uQyKY4R2Ny3fQ19PpkvaEIk7XBCrAXP6vFTQvx/fv3ZwR4\nd3cXw+HQLfauV3TWReF3d3enIif+Z+DiKtAtE3Lan61FV7tGvJVBSkkgtsk1k60Xi00xXn8owqRz\nvOI32hLVQuIJ8OHh4QUB3tnZmboSxLVgt8BTk4RaJDVWtK3QRqFpevUQK/5a1K1FbmOMo3177bQA\n15JjyHpBESZrg73t9h73RNgTYCmRqX25ugFPxSnbovCCtYBlVZCaQFpRtuKrt/I5AKZV4qw1rd/H\nltH0fN7yGNkMKMJkLbACrBfW1I95IhwV+zk+Pp6GsUnKso0V1skbOolEi7IW2JY0ZRuG5lnCkW/a\n8yfLexwfH4fXq3ZtyfpCESZrgxVgLU6RCEcCnHPG8fExhsPhTMlKeT+bwGFD2XS8soh1S8EeL/TN\niq9tgljv0nZ3d4v1JvT18sSY4rsZUIRJ53hxw1oItaVqRTgSYF0UKBJg8a3q97dRCl6EghXB0tYu\nIOqJr0bOza4mLeel4421D9i7e6BfeDOgCJO1wvNtajGR8DA96eYJsPanagGWELN+v38hs64Wl2sn\nD0tN3lvqBltLODpvXUEuEmAJcxPBtXcPFODNgiJM1oJIfIHZFTl0Ft3R0dGMuNUE2KYr64I+IpQe\nkWhaUfb6IsLW7WDPW7YisPY1+tyk8ppG3z1oAaYYrz8UYbI2tIRY2dhbLVQ6IkEqnQEXBdjWmLCW\nqrymlODhHbe3b90RtXOPSl7alGcrwvrHxBNjsr5QhMlaEomHFll7i299psBTlqVNPe73+zg6Oppu\nbeyu3gK4MLkG+BayHRPfs/inS5N70efI9bATgDpqQ8cY29dRjNcbijDZOKzY6qQILV5S7rLmOz45\nOcFwOHRTi71+yUqOLGn9QyBhczq+WF7jJZZEzYsn1payTuqQc671ydVDESYbhbUIbSyuft7Ozk6T\nCE8mEwyHwxnxk6b3tTBqP7IW2ygJQ68CIkXkrQDrMpsl8ZUWldOUpA59rnbrWccU426gCJONQwuo\nCI4XSyuREJ742nrFw+FwKtg6s87uS9acl4KsXRfyWQBcS1jXf9DWcmuT5A0dU2zdNPp8o8lDfb1I\nN1CEycZhhVRnk+nHrGWqHxcXhsTlDofDafpzv9+faTJ2cnIyFVBdAEhn3wkixNpCFktYi6IWcW0B\nW2vYc43oGseS2BGdqw6fs9fSi9YgVwdFmGwUWkxs+JcVZ2B2FQ95jViMIsCj0QjD4RCDwWCmie9W\nuw50rK6O2RW0K0KLrC4mb+OW9VJL1hViJxb1mHa1eNEi+lpEiSe2hjG5euYW4ZTS8wF8H4DnAPgC\nAC/JOf+SevzNAL7VvOydOecXXeZACRFEPFIqF4HXz5fHo1KYskqHbXryTN/Oa0G1Imb9rTpJJBrX\n7o+SC0I37Rax/nAbU2xdEdoVol9Hrp5FLOEDAB8E8NMAfjF4zjsAvByAfMNHC3wOIRewImLHtIvC\nWnw1Ad7f3y8ulQQ8JVqnp6dTUbXCKhayHrMuC+0nluLw/X7/gouj1KxvWrAxxTq7Tq6Frc9BAe6O\nuUU45/xOAO8EgBRHoB/lnD91mQMjJMIrVCPCIgIsYlgSYO1yGA6HFxYMtRawCKe2LGVchNbe8mtB\nledqF4Qte1mzfksV2bykDnkPfa3kGjK7bj1YlU/4kZTSEwA+C+DXAbw65/yZFX0WuUZoEZF9m7Ir\nTURUC7BO0pAJt16vF1rAnv+2JMLaAteP68gJsUxtsoYnwiVBtv5uK8B6kVQhulYU4O5YhQi/A8Av\nAHgMwJcA+CEAb08pPS/zWyZLQP6MdHaYvimTvl1mqNfrTctf6vCzvb09jEajJgG2omajH7So6iw4\neY71zern6Ym+SHz1vjfpKOdr46Oj69iSUk1Wy9JFOOf8VrX7ZymlPwXwUQCPAHj3sj+PXE9sAoKH\n547Y29ubWog6AeP4+HhGEG0Chs2Y0xN2VnS97DlBJ3DYMW+STz/HHpN+jfWJ61RpqaMh7hJ5XF9L\na92Tq2PlIWo558dSSp8G8DAowuSK8UKybI2FlNK0PKaErHmTXvJ6cWvoJv5lL8bYS+zw+nIsWvDF\nWtfWuWdNC96PhvzIeKtBa6G2vnZyNaxchFNKzwTwNACfXPVnEaLxwrJ0WJtGEj5qheJPTk4wHo+r\nAixjvV6vKQPOs3R10SFb6Ed8uNb6tq4LEWHxd8vEpGx1tAdFuBsWiRM+wJlVK/dYX5xS+koAnzlv\nr8GZT/jx8+f9MIAPA3h0GQdMyDxYK9i6CuQ5Ozs7TYXiRai18FoRtvteIobsW7+xtYS9DDttOQMX\nBdhL8pCIEPmRkcpxcn7ivuAE3dWziCX81ThzK+Tz9obz8Z8B8J0AvgLAywA8COATOBPf/5JzPr74\nVoSsFivAXmysiJstFC+P2TTnkgjrsDcv9Vm20aoesi+WsLVQtUjrscgNIZOQdkFUe35ejWJyNSwS\nJ/ybAErf1tcvfjiELJ9SgodYwSkljMfjqgCPx+MLQtuyHQwGmEwmGAwGF4rJi9CKOOpIC7GWPSu5\nJL62frIIsD0/HU/MKIluYO0IstVYd0T0mIhZTYDnEV2dCCIRClaARTQ9365k2FmhFYH1hFk/rq3v\nkgBL5TVawt1AESZbiwiYTdXVwiuPyb68zhNgW2EtEl09NhgMLggwgBlBjWKS9b4VbR11Ebkg9PFG\nLgid0MGkjW6gCJOtRguKTdXVPmKdMKEFuNfrTSfsdIKHV2ktsoJLFrCUyLQiLGgr2C5t5FnHdgmn\n0WiEfr8fCrCNBiFXD0WYbD12Ak7HB+sQLW0BizDpiAkdseAJr7c/Ho+LAqwXHJVj0jHD8pi4Jmw5\nTR3KpovG6zYej6fnbwVYojfoE+4OijDZauyttRY7QcROF2sXP6mXNry3txeKrh0fj8cXXBBipeo0\naXtcIro6M0620vd8wGK5y+dLUSItwDoppSW9mawWijC5FkRirBHfq14uyW4llE2ETCxN3SSEbTwe\nT2NvRXxFgCNf8Ty1HLSrIoqIkOPzSnjq8DmvKDy5GijChJxjJ+esr1gsRV1/QS806iWBpJRmLFKd\nPiyvt6nIAJrEMMqSs5N9Oeep2NrJRW85J22Ze2nS+jjJ5aEIE6KwQmwfk60IsRVgnU4sIXEyQadT\nh3W1Nm+JIX0cJWz4mrzf3t7sv3ZJeK04TyYTt+aGrVVBIV4OFGFCDJG46OgKHaEgLgcd8qbXuZN0\nYXFXaCs4EmE5jpIQ21oTOrbYvp8V2pJVbM9DR5LoHxeK8HKgCBMSUBJjvYKHfr4NIwOA0Wg0447Q\nlrBdCdmKbkmIPVeEV9QnpeRWeIuanqiUHwr94+OJPFkcijAhCi0uUeKCFVk7Lm0ymQDAjBVsfcLW\nEl5EiG2GnYzLdmdnJ7SAPWsYwIz46rKfErFBEV4eFGFCzvH8wVaIRYSsCFs3hNRjyDnPWMLREkr2\nM+admJP3sGUxxUouia8VYuAsUkTEVyJGShY7WRyKMCEGGxlhx20lNivAOrY45zzjE/aiI1om5iLh\n02Jrx7SFXBNf3Qcw4+fW56l9wmQ5UIQJUWgBLgmx9LVlqFOJtQiPRqMZK1jig7U7wr63t2/xBBjA\n1FLXqc4t4it9m1loBdhGgpDLQREmxBAJsEZHCthkDi2Op6enRUvYW7Jonlt9/VlaGOUHQN6/1QqW\nIvKRAHvx0ORyUIQJcSgJsBZKb6v7ngjb6IgoRM1+lkXGtU9Y93Wbxx1hkzO0ADOrbvlQhAlpIIqa\naLkdj8TXhqfpz/KEzoq/92PgHVfOeSadOepL0z8M2s+tJ+ooxMuDIky2HluspzRmLVlv33ufaNvv\n9/HAAw/g1q1buHnzJm7cuIH9/X0Mh8OZCmZ2Yq3Wt9aq7nv73urK8qOgayZbl0mUWEJf8PKgCJOt\npkVUrR/X+li9sajZ9+33+7h9+zZu3749FeHhcHhhNWbvx6GG53aImhVeK8ZR01a7l7pMLg9FmFwL\nIgG1YuuVr/RaJNS2ry3hg4ODqSUsIuyt7iy0CrL229q+bFuE10Zw2AnEmv+aLAZFmGw1nuh6ZSq9\n5eJFHL0x77XevljC4o44ODhw3RE1AY4E2RNd7TawCSSREGvxLbkjrIVNLg9FmGwtkQBHTQTRbr0x\n+9porNfruT5h647Qq2l45+FhBdGmTWvr1QpwZBGLNazdEC3FhsjiUITJ1uMJsBVNu0SQLpDujev3\nsFazHuv1elNLuOSOiCIOai4JT4St+0CLsBZgXejdWsO2ZrK1sCnEy4MiTLYazxL2RNML1YrGPEs5\nar1eDzdv3py2aGLOW1poHp+wZwFrMZ7HEhYRtkJu/cxkOVCEydYTuSSsWErCQm3rCXHkwuj1ejg4\nOJi2WojaZaMjrAjr1T/m8QmXfMwU4OVCESZbS018PcHUa8DpymJ6P3JR2KXn5T1FePf392fcEfJ5\nJXdEDc8frC1gT4BLQixF5220hReBQZYDRZhsNVaIvYgHLZi11ZO1L1eLcak/HA4vNC9EbVG0QHoC\nrIXYRkB4Quyte+ftk+VAESZbjyfE0XLx2vIVsbRbsWKtzzja1yJuLWubrCHHG6VJ6/3IHVFzRZQy\n5iRBwxNZCu9qoAiTtSJK09X7LZlqKaUZN4O1Uu02soCt+C5iCXtuDPkR8M7Bil1pP3IVeOFqXoRD\nVFTeQgFeHRRhshbU6jNIvzVJQkc8tBSvsRaqdUPoscgnHPmFIwH2qpLNI3aeu8BL3PDqPpRa9Dlk\nNVCEydpQq8tQCjPz4nS9ELOoRYte2gk67UKoRUjYCTotxPZHY95JuVYBrrXWzyGrgyJMOqdUz8Hu\nRxls1sr0BNgudFkKSyuNiQi3tMha1vUiFomKAOICPq2uCFrA6wFFmKwNXsUy625o8fNaN0Or0HoC\n7Y3rjLmWrLkojngRSzgSxtpEnRfr60U91D6HLB+KMFkLSkV2bHpxi3B6qwhbl4Ldr/mNtdC31o7w\nCgAtYglHlmpr7Yh5hJgCfLVQhMla4Pl/PWHTouj5a0u+3NK4zoSr1Y3Y29trnhz0BNmO1YS45CpY\nhktCvw+5eijCpHNsFETJorRhXzaMzAs18yIf7FZPtnmRDrpv3Qi1fkmcbXSEpUUYl+ETJt1BESZr\nQWsUhM1si7LRbGyvJ9A27MxLZ/bG7MRh1OS8ShON9vktlKxgr5awFxvM2sDrA0WYrBXWSvQm5Gx6\n8XA4nNZmkOI4so0SMGzTImz9uPbHQFuv82xLYx4tEQulULVSJTQvUYNC3A0UYbIW1Cxh7RLQ7ggR\nXCmMo7faMrbWse3rGg61ybaa1bpoyFkNL1NO9+eJD45cEhTiq4ciTC5FJDg1y0/3o8kvrw0GgwsV\nyWxfW8SRn9irERxZ4Z5Pt5Uo7CsKCytNtNmx8XiMo6OjafUzb1/anTt3cPfuXdy/fx+Hh4cYjUbT\n5+p15MjVQxEmc6OF1+uXki9s/G9KKazp4I2JCGuXg+eG0KtXeEVz5D09a9dOvC3it7WUwsEWsWRP\nTk5C4dX70r9z5w6efPJJ3L17F/fu3cPh4SFFeE2gCJO5aPV5tliV0mrpxHo/mozzWpT5VqrjEEU6\n2POfh5pFq8d0+cla0yLrCa/u3717d9ru37+P+/fvYzQaVSunkdVDESZz47kVvDAzLznBG/My2KJ+\nyb/rVTuzoh7VcdAVzawVLOe1KFZsvQgG6UcF2L1SlDUR1mP37t3DvXv3cP/+/QuWsIgwV83oBoow\nWZhSeFZrgZvd3d1iIoXnUogqnUUCXPIxt4adyfkuSktW2+npqbvuW9Ss2Ja2h4eH06b9wuKOmEwm\ndEd0xFwinFJ6FYBvBPBlAA4BvBfAK3POHzbPex2AbwfwIIDfAfAdOeePLOWISWdEAuUJmC2kU8pE\nKwmptz9vtbPSD4BXw6EkxItgBViLrl0Jw5tUs01bwdIi8ZX+aDSaTsZJn+6I9WBeS/j5AH4cwB+c\nv/aHAPxqSunLc86HAJBSeiWA7wbwMgAfA/CDAB49f854WQdOuqM26aYtYc8VoP2y2s/bEkpWKshj\nt7bQTsk1YmN/lyXAgifE3lJEVjy10EYteo3tRxN52h1Brp65RDjn/CK9n1J6OYC/A/AcAO85H/4e\nAK/POf/K+XNeBuAJAC8B8NZLHi/pmFoEhGy9kpKRrzfKeIsm21rrBJeK6nhNn5fXX5RSEoVdgkgL\npbZYbauJsd0vrSunoyNoDV89l/UJPwggA/gMAKSUngXgGQDeJU/IOT+ZUvo9AM8DRXgr8CbhdESE\n9LX7Iarf0O/3p2FlNsTM21o/b9QXd4P3AxFt7bl557soLQIsAikCrP244sPV+yUBtuM64iLq0xLu\nhoVFOJ39Rb4RwHtyzn9+PvwMnInyE+bpT5w/RrYEzxK2FqdX68FrNsmi1Gx4WakfRTnUJt2s2C4z\nPljEOBJgbQXriTTbrAjXWqmmhE1fJlfLZSzhNwF4NoCvWdKxkDWnNnEV1XooFd0ZDAZuyrE3JiJc\nKq6jWy2+dxm+3la8iTkrxFaER6PRNKTMhpjdu3dvLhFuydYj3bCQCKeUfgLAiwA8P+f8SfXQ4wAS\ngIcwaw0/BOCPFj1Ishwi0SlZfrrfuqTP7u7uzGRa5Ov1iu94qcd2qXmvulpUaKeVWnqx7nsZb9G+\nDTvzfLIyLvG82uLV+7qvLeEoNli/N1lf5hbhcwH+BgAvyDl/XD+Wc34spfQ4gBcC+JPz598G8FwA\nP3n5wyXzEIlp5O/0/KK631LbQSzTeSIeonKUOt5XpxhHGW6XnUiL0om9sZbECxv7q4XRa+PxeCaO\n1/a1b1gnWojQWv8urdvNYN444TcB+GYALwZwL6X00PlDn885j877bwTw6pTSR3AWovZ6AH8D4G1L\nOWLSRDTRFAltSytFItiIhcj/K5XP9ASdzXTTj9s6Dzau1wqxPrdFsGnFUZqxDTGLws5OTk5cEfb6\nOp5XxDbqS8SDvFaLsPX1kvVmXkv4FTibePsNM/4fAPwsAOScfySldAPAT+EseuK3Afw7xghfPZ7Q\n6n0AYZyvFwPcul5bbRWLlq0tuDNPcsWieKFkXt9GFuitN6ZFVve9rQ4/08kVdswLO9OWsLXkyfoy\nb5xwk5Mt5/xaAK9d4HjIEqhFA3gTal6Ime3XMtrsY7U13bTI1rZeuUnPEl6GCNdSi0VcrQB6omiF\n1gqxt99aD8LWk9CWOK3gzYG1I7aYeYQ3Ejcd7WCL5cxTwSyymFtXOdY+YS/Gd9n+YOtm0Ftr3Zbc\nC1ZsW1OSo/eyLojIKqcQbw4U4S0mEt4opKzWF+u1JZ7XCqzej6xcG+/rxf9Gbgiv7OS81LLatI93\nXqvV2/fGdbiat/VcD/Y4Gfu7WVCEtxTrjogKq9uIg1Lolw4pk1jeqLWkFev04lrJSx37W2qXIarv\nYC1NPYnm+Wq9tOKW2r8iwlFWW2Tt2sk4WsGbBUV4CylFQHguh3lif7UIHxwchM1bFSNaMaNUz8Fz\nlbSG1S2CJ8SlWr5eBIPdjwrreNuSG8Ru9WSht6UlvBlQhLeUVgG2IqyjEGzfE+GbN2+6zavlEJWz\nbHExlPy+UX8eakV2vPoOOr3YxvPatdxaC+6URDUS2VI8M0V4/aEIbzGlSblIgEs1GSIRvnXr1ky7\nefPmjJVb8/la69UK6TJEtoVSfQfrirA1Hrz6DnoFi9aSlF5Wnj42PWb7pTGyvlCEN4BSyJk3Virb\naMdaJ8X29vamAnvz5s2p20HXedAVz7zi6VFh9XmEtSW9WO9HImYf10kVLStaeKJbKrRTKjOp3RG1\ncybbB0V4TahZf60+Uy+yoTTWUo1Mmli+2vd748aNmbA0O9HmHdey/La635JeXNp6NR2iNOOjo6ML\nqcSeS0L7g6MU49okGgV4+6EIrwElP6cIVs1na/dLEQ+RH9iKrxXhKBJCLy/vLaAZRTUsSiS0UV2H\n0sSWF/vrCbDetwXX7aScTS/Wsb5RUgXdCtcXinDHeLP7spW+iJkXWWC3XllHLwJCl5uMtrav3Q22\n0tm8xXb0uc6DrVBWKp4ThZh5Bc1LiRZRjYdaeJpew81byaJUaIcCfH2gCK8BpbArcUV4ywR5WWhe\n3G2tlYRYb2tlKbUIt4aZLcI86cVekkPUr9V2qKUXe1vrgmhJMaYAXy8owh3iRQVEIixiWKrXIELo\nuSpqYy0TaV6xHTsWre3mhZstinU5RFXMbO2GmquhVtOhVISnJN5RwoV1R1B8rycU4TWglgWm3RF2\ndQq7PttgMAhFtDZWe24t/Vi7S1ZZ4yGK59UWpk0vbkkbLtV5sGJri/V4+9L3UotZ95cIFOE1oiTC\n4pfVIuwtATQcDqv+XTtWm+TzJulKiRh7e3tNiReLEmW1eSUkS3G52n87b5EdW7bSa9bijdKLKcTX\nG4pwx8wT86sLpsukmA4Vu3nz5lSES/G+niiX3Bbec2qTf6VMt2UKcbRqsV25uNZKQmwfE+u2ZOHq\n5k0c2jGK8PWFItwhkT/YWo+eT1gsYZs6fOPGjaKF6glxq9/Yxit78cv6uPU5XsYFoYkm5bQQ24k2\nu3qx16xFXHJdtFq3ns83auT6QhFeAzwr0YqcuAAiEb516xZu376Ng4OD0GXguRDmsXhrwloS2mWm\nG0fVzmrLx5cy2yScrKXa2fHxcTVWORLZKLOPXF8ownPSUtegZasFtmRRSh3fUsUy20qWr1e/t7WS\n2jzMIy5eGnG0jQqqe2OyZLxkspVSi2suCVtuspSpZ8+JkBIU4QZKMbyesNqJKC9SYJ7W7/dnXA62\nhoNMzEm8rucH9ibjagkVy7ReNZ416KUSe2PzrFRRc0F45SZtpIOXXuxZt4zzJYtCEW4kKqsYiasV\nOC10pTHv8V6v59ZtsEV0rAi3TLLZz4x8upellBUWJVt4+6VECTsWpRJ7rSTAUWIFxZcsA4pwhSha\nwbNwW2Jtbb9U40H6/X4/rNtgLeHhcNjsXmhJqpBrcBlKflBt8VrR89KLS2FnUe3elhRjG3oWpReX\nXCelcyYkgiLcgBXgqEkYmed/tf1WkZT39NZx0yUktTuipXiPZ/0uO7MNqJee1FlvpZhbad5S8Nql\nYLfzpBdHsb+RFVzaEtIKRbgBL2bXcyHY+g5eX/bnidPt9XpuhpxtssCm5+LwfjAii34VfuGSWHkh\nZqVavq1uBi8TLupHhX1s5bVSxAMFmCwCRbgRT4g98ZQaCrp5Y7bqWWkrCRq1NhwO0ev1mmN5rdiu\nygpu2Yo7orZUvNTy1ZNuUV/7eSNxt64HrwaFF/Orj7/UJ6QGRbiC54qIrFYtuiKKpUI7LZltYglb\nEffaYDBAr9dritSIRDdqy8Kb1PIsYev71TG/tnh6VFRdLNyWNm/ShXdOtk9ICxThBiIhtuIpYhm5\nC2zpx9ZYXu3msG4NO+bVbaiJa7S/DDzR9fpiCXuJFnZVYy24Nt5X96WCWeRmiNKLa1t9Xt65EjIP\nFOEG7C27N3GmU4r1gphRE6s1msCLJvVa0pCjuObSmD7X6LFFiWJqtQDb4uo6gkH7faOEi3v37l3Y\nPz4+DlfS8Pa9xIt5st8IWQSKcAUtRnZSzrNUdVqxRDB4Tfy3+rXW4tUCPE/ImSecpUy/lvHLEiU4\n6PRjzxLW67lZkZUW7YsIR4kfdquPrdYnZFlQhBuIfMJWNLUVrC1hG9N7cHBQFWHdjxIrvH5JRCMR\nsZEKLa+pYTPfoky4nPOMn9dzK9gxLbRagK11fHx8PD2WVuuWkKuGItyAdUVYy1d8sy0RDLrV3A0t\nacXR5FkpJKx02x2NzytUpWWHbIuiG6Ixr96DuDG0f3dZ50LIKqEIV/B8wZ4LQrshWrZahKOoiFJW\nW+R2AGKR9cZaJ6PmvS2v1djVrbXmr/YN69hgne0W1XnwIhooxmQdoAg34MUHewJcit/1ti1pzSUr\nuJRUEbkBPF9s1Kz16hEJWbTqhDdWSz/2Uo11QoZYw/J+kQBTeMk6QhGu4CVpaEvYiwv2rF6vtdZ1\nEF/vPBltLVaurcdrV4mwj7X4lAVvrbWo35paHBXqidwRkQBzso2sExThBrwCPV5ExLzuiFpNBy3E\nteQK4KnIBk+AvcQDWzTH6+sxoSVGNlp5eJ7VjWv7duXkyB2hj88TYkK6hCLcgGcJz+OOiFpUzyGq\nalbLcLN4AmxdDK0ZZZPJ5MJ7e58neCtSRCtWlGpF1FKNvfKTrPFANgmKcAUvMsJmyM1rBQ+Hw5lC\nO9bVEBXU0cfT4oqwvl9bD0GSIzxRs2OymoT9HO+zATSVmpR9LaD686J9b2VlW/+3xf1AMSbrAEW4\ngZIlbIW4tYkIW1FtcTm0ZLVFk29erV57W6+bHtfv7fX1/jzVziJhjdwiUaEdr9hO7TgJ6RqKcIVa\niFpLaJrX+v1+KKylOg4tfS8SIiqWXqtaph+T99afo9H7Ucyv10oVzKKKZp7P2/YpvGQToAg34Lkk\ntEH2YBgAAAiaSURBVAjXhDiyhPX7e31vPxqz1CzhaGXi0vJB+r1r/VJxHTuul5D3BLVUSrLWJ2Td\noQgvSM0nK9vIJ6ujDZZJztmNyY36tRCwVhG2Y7Ul5nWzBdOtpesljBCyLVCEGxDh1FbjaDSaiWaQ\n52nr0lYBkzoHN27cQK/XW8mxiuuhFnom/daVi8UdIZ9R69u0Y5terF0QnvuEE2jkukARriBWmBbh\no6OjqX9Yx+aKCNtSjIeHhxeWJVq1CHvN+ll1NIQXd2vH5P3t53n90lJDVoBLIWUUYLLtUIQr6Nth\nEa2SAOvb+NFoVKwbsSq8yawWIa4tAyTnaq+Ptx+tiKzTi1stYUK2GYpwBWsJj8fjmbhd/bhYwIPB\nYCrAdp05vbTRqo83StLQfS8GN4rNlfe3n+cdg1wLO9Gn04ujQjsUY3KdoAg3oEVY+4CtAI/HY/T7\nfYxGo2l5S2/V5X6/j93d3ZUcqxe6VWrRpJ23jQTRGy+5NvRn2Uk3uiXIdWMuEU4pvQrANwL4MgCH\nAN4L4JU55w+r57wZwLeal74z5/yiSx5rJ1hLOLKARWC8JYq8rQj5KqjFz+pta7nJRQv4WKu6ZAnL\ne3kRF4RsK/Naws8H8OMA/uD8tT8E4FdTSl+ecz5Uz3sHgJcDkBiuI2woWqw8H7COmPCWq4+Wsl+l\nCNuwLm/yywpxzW1RC6mzgtma/WYt32hLyLYylwhbazal9HIAfwfgOQDeox46yjl/6tJHtwaIQIgI\nedESXglKvfX6VyHCLc1zXURWdMvnClGtiij7zb6eIWrkunBZn/CDADKAz5jxR1JKTwD4LIBfB/Dq\nnLN9zsYgAqQF2FY58yqflfqrWlBTjlO2nmXp+V8jcV5kgiwS+Jq4t0z6EbJtLCzC6UxF3gjgPTnn\nP1cPvQPALwB4DMCX4Mxl8faU0vPyBv5XWTGJqpnVaj9EtSBWfewt+63beT87mmyLJt8IuY5cxhJ+\nE4BnA/gaPZhzfqva/bOU0p8C+CiARwC8+xKf1xmcnSeErIqFHJMppZ8A8CIAj+ScP1l6bs75MQCf\nBvDwIp9FCCHbzNyW8LkAfwOAF+ScP97w/GcCeBqAolgTQsh1ZC5LOKX0JgDfAuDfA7iXUnrovA3P\nHz9IKf1ISum5KaV/nFJ6IYD/A+DDAB5d9sETQsimM6874hUAbgP4DQCfUO2l54+fAPgKAG8D8JcA\n/juA3wfwr3LOx/bNCCHkujNvnHBRtHPOIwBff6kjIoSQa8TqMgYIIYRUoQgTQkiHUIQJIaRDKMKE\nENIhFGFCCOkQijAhhHQIRZgQQjqEIkwIIR1CESaEkA6hCBNCSIdQhAkhpEMowoQQ0iEUYUII6RCK\nMCGEdAhFmBBCOoQiTAghHUIRJoSQDqEIE0JIh1CECSGkQyjChBDSIRRhQgjpkHUQ4WHXB0AIISui\nqm/rIMJf1PUBEELIivii2hNSzvkKjqNwACk9DcDXAfgYgFGnB0MIIcthiDMBfjTn/PelJ3YuwoQQ\ncp1ZB3cEIYRcWyjChBDSIRRhQgjpEIowIYR0yFqKcErpu1JKj6WUDlNK70sp/fOuj2kZpJRek1I6\nNe3Puz6uRUgpPT+l9Esppb89P48XO895XUrpEyml+yml/5tSeriLY12E2vmllN7sfJdv7+p4W0kp\nvSql9P6U0pMppSdSSv87pfRPnOdt5HfXcn7r9t2tnQinlL4JwBsAvAbAVwH4YwCPppSe3umBLY8P\nAXgIwDPO29d2ezgLcwDggwC+E8CFEJuU0isBfDeA/wjgXwC4h7PvsX+VB3kJiud3zjsw+11+89Uc\n2qV4PoAfB/BcAP8WQA/Ar6aU9uUJG/7dVc/vnPX57nLOa9UAvA/Af1X7CcDfAPj+ro9tCef2GgB/\n2PVxrOC8TgG82Ix9AsD3qv3bAA4BvLTr413S+b0ZwC92fWxLOLenn5/f127pd+ed31p9d2tlCaeU\negCeA+BdMpbPrtqvAXheV8e1ZL70/Bb3oyml/5lS+kddH9CySSk9C2fWhf4enwTwe9ie7xEAHjm/\n5f2LlNKbUkr/oOsDWoAHcWbpfwbYyu9u5vwUa/PdrZUI4+xXaxfAE2b8CZz9YWw67wPwcpxlCL4C\nwLMA/FZK6aDLg1oBz8DZH/62fo/A2e3sywD8GwDfD+AFAN6eUkqdHtUcnB/rGwG8J+cscxNb890F\n5wes2Xe318WHXldyzo+q3Q+llN4P4K8AvBRnt0hkQ8g5v1Xt/llK6U8BfBTAIwDe3clBzc+bADwb\nwNd0fSArwj2/dfvu1s0S/jSAE5w5zDUPAXj86g9nteScPw/gwwA2YuZ5Dh7HmS//WnyPAJBzfgxn\nf78b8V2mlH4CwIsAPJJz/qR6aCu+u8L5XaDr726tRDjnfAzgAwBeKGPntwgvBPDero5rVaSUbuLs\niy/+kWwa53/Uj2P2e7yNsxnrrfseASCl9EwAT8MGfJfnAvUNAP51zvnj+rFt+O5K5xc8v9Pvbh3d\nET8G4C0ppQ8AeD+A7wVwA8BbujyoZZBS+lEAv4wzF8QXAvgBAMcAfr7L41qEcz/2wzizmgDgi1NK\nXwngMznnv8aZL+7VKaWP4KxC3utxFuXytg4Od25K53feXgPgF3AmWA8D+GGc3dU8evHd1oeU0ptw\nFo71YgD3Ukpi8X4+5yxVDDf2u6ud3/n3ul7fXdfhGUFYyXfi7Ms/BPC7AL6662Na0nn9PM7+mA8B\nfBzAzwF4VtfHteC5vABnoT8npv0P9ZzX4izc6T7O/sAf7vq4l3F+OCtT+E6c/ROPAPw/AP8NwD/s\n+rgbzss7pxMALzPP28jvrnZ+6/jdsZQlIYR0yFr5hAkh5LpBESaEkA6hCBNCSIdQhAkhpEMowoQQ\n0iEUYUII6RCKMCGEdAhFmBBCOoQiTAghHUIRJoSQDqEIE0JIh1CECSGkQ/4/dWtzaJOhYVoAAAAA\nSUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11aaacbd0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Checking the downloaded data\n",
    "print(\"Shape of training dataset: {}\".format(np.shape(X_train)))\n",
    "print(\"Shape of test dataset: {}\".format(np.shape(X_test)))\n",
    "\n",
    "\n",
    "plt.figure()\n",
    "plt.imshow(X_train[0], cmap='gray')\n",
    "\n",
    "print(\"Label for image: {}\".format(y_train[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Building synthetic data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The MNIST dataset is very popular for beginner Deep Learning projects. So, to add a twist to the tale, we're going to predict images that can contain 1 to 5 digits. We'll have to change the architecture of our deep learning model for this, but before that, we'll need to generate this dataset first.\n",
    "\n",
    "To generate the synthetic training data, we will first start by randomly picking out up to 5 individual digits out from the MNIST training set. The individual images will be then stacked together, and blanks will be used to make up the number of digits if there were less than 5. By this approach, we could increase the size of our training data. We'll build around 60,000 such examples.\n",
    "\n",
    "While concatenating images together, we'll also build the labels for each image. First, labels for single digits will be arranged in tuples of 5. Labels 0-9 will be used for digits 0-9, and a 10 will be used to indicate a blank.\n",
    "\n",
    "The same approach will be used to build the test data, but using the MNIST test set for individual digits, for 10,000 synthetic test images.\n",
    "\n",
    "\n",
    "\n",
    "Let's write a function that does this."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def build_synth_data(data,labels,dataset_size):\n",
    "    \n",
    "    #Define synthetic image dimensions\n",
    "    synth_img_height = 64\n",
    "    synth_img_width = 64\n",
    "    \n",
    "    #Define synthetic data\n",
    "    synth_data = np.ndarray(shape=(dataset_size,synth_img_height,synth_img_width),\n",
    "                           dtype=np.float32)\n",
    "    \n",
    "    #Define synthetic labels\n",
    "    synth_labels = [] \n",
    "    \n",
    "    #For a loop till the size of the synthetic dataset\n",
    "    for i in range(0,dataset_size):\n",
    "        \n",
    "        #Pick a random number of digits to be in the dataset\n",
    "        num_digits = random.randint(1,5)\n",
    "        \n",
    "        #Randomly sampling indices to extract digits + labels afterwards\n",
    "        s_indices = [random.randint(0,len(data)-1) for p in range(0,num_digits)]\n",
    "        \n",
    "        #stitch images together\n",
    "        new_image = np.hstack([X_train[index] for index in s_indices])\n",
    "        #stitch the labels together\n",
    "        new_label =  [y_train[index] for index in s_indices]\n",
    "        \n",
    "        \n",
    "        #Loop till number of digits - 5, to concatenate blanks images, and blank labels together\n",
    "        for j in range(0,5-num_digits):\n",
    "            new_image = np.hstack([new_image,np.zeros(shape=(mnist_image_height,\n",
    "                                                                   mnist_image_width))])\n",
    "            new_label.append(10) #Might need to remove this step\n",
    "        \n",
    "        #Resize image\n",
    "        new_image = misc.imresize(new_image,(64,64))\n",
    "        \n",
    "        #Assign the image to synth_data\n",
    "        synth_data[i,:,:] = new_image\n",
    "        \n",
    "        #Assign the label to synth_data\n",
    "        synth_labels.append(tuple(new_label))\n",
    "        \n",
    "    \n",
    "    #Return the synthetic dataset\n",
    "    return synth_data,synth_labels"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Building the training dataset\n",
    "X_synth_train,y_synth_train = build_synth_data(X_train,y_train,60000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Building the test dataset\n",
    "X_synth_test,y_synth_test = build_synth_data(X_test,y_test,10000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(7, 1, 9, 7, 10)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWEAAAFiCAYAAAAna2l5AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAAPYQAAD2EBqD+naQAAIABJREFUeJztvX+MbNtVHvit6uru6u5777v8mveGgQyPPEIcOTLBYMcT\nDDM8hGMkiJEiQiaaJ4g8liGW3kQjxVhjxBs/hjCOYDwkeGQpGhFgmMj/JA6eDA9jZxJjx7bAxsiD\nITH2wxjz7oDxu7e7flf1nj+q13nrrFr71Knuqjp7n1qfdFTd1dVVe9fZ5zvf/tbaa1MIAQ6Hw+Fo\nBp2mG+BwOBz7DCdhh8PhaBBOwg6Hw9EgnIQdDoejQTgJOxwOR4NwEnY4HI4G4STscDgcDcJJ2OFw\nOBqEk7DD4XA0CCdhh8PhaBBbI2Ei+ntE9BkiGhLRh4jom7f1WQ6Hw5ErtkLCRPS3APwUgB8D8FcA\nfBzAM0T05dv4PIfD4cgVtI0CPkT0IQAfDiE8efU7AfhDAD8TQnireu2XAXgVgGcBjDbeGIfD4dg9\negC+BsAzIYQvVL2wu+lPJqJDAC8F8BP8XAghENGvAXiF8S+vAvB/bLodDofDkQD+DoBfqnrBNuyI\nLwdwAOCeev4egEeM1z+7hTY4HA5HCnh21QtSyI5wC8LhcLQVK/ltGyT8pwDmAB5Wzz8M4LktfJ7D\n4XBki42TcAhhCuA3ATzOz10F5h4H8MFNf57D4XDkjI0H5q7w0wB+joh+E8BHAPx9AKcAfm5Ln+dw\nOBxZYiskHEJ451VO8FuwsCF+C8CrQgh/so3PczgcjlyxlTzhtRpA9I1Y2BcOh8PRNrw0hPDRqhek\nkB3hcDgcewsnYYfD4WgQTsIOh8PRIJyEHQ6Ho0E4CTscDkeDcBJ2OByOBuEk7HA4HA3CSdjhcDga\nhJOww+FwNAgnYYfD4WgQTsIOh8PRIJyEHQ6Ho0E4CTscDkeDcBJ2OByOBuEk7HA4HA3CSdjhcDga\nhJOww+FwNAgnYYfD4WgQTsIOh8PRIJyEHQ6Ho0E4CTscDkeDcBJ2OByOBuEk7HA4HA3CSdjhcDga\nhJOww+FwNAgnYYfD4WgQTsIOh8PRIJyEHQ6Ho0F0m26Ao1kQ0dIRQigdAIpHh8OxWTgJ7zGYdDud\nTum4vLxECAGXl5fFzwwnY4djs3AS3kMQUfEzE+/BwUFxXF5eYj6fF6+7vLwEsCBgVsoOh2MzcBLe\nY7ASZvLtdrvodrsFAc9ms5IlAbgSdjg2DSfhPQOrW+kBdzoddLtdHB4e4vDwELPZrOQNsxIGnIQd\njk3DSXgPweQLAAcHBzg8PMTR0RGOj49xfHyM6XSKyWRSsiOYiPk5J2OHYzNwEt5jSBV8fHyMXq+H\nk5OTgoBDCIU1wQE794Qdjs3CSXjPoNPR2As+OjpCr9fD6ekpDg4OCgKezWYlApZBPYfDcXM4Ce8p\npB/MdsTJyQlOT09BRJjP54Ut0el0iswI/l9Xww7HZrD2ijkieiUR/Ssi+iMiuiSi7zFe8xYi+jwR\nDYjoPUT02Gaa67gJrKCcpYRPTk5wfHyMo6MjdLvdIo3NlbDDsXlcZ9nyGYDfAvDDAJbkEBG9EcAb\nALwOwMsA9AE8Q0RHN2inY0OQRCo9YQ7MnZycoNfrFQR8cHBQImAnYYdjs1jbjggh/AqAXwEAsq/I\nJwE8HUJ499VrngBwD8BrALzz+k11bBI6PU0q4fl8jtFohMPDw4KE2Rfm/3U4HJvBRgv4ENGjAB4B\n8F5+LoTwAMCHAbxik5/luD5idoRWwoeHh+h2u0vBPIfDsTlsOjD3CBYWxT31/L2rvzkahvaF5UIN\nmaJ2fHy8pISdhB2OzcNLWe4hrNVyrIR7vV5BwJYn7HA4NotNk/BzAAjAw+r5h6/+5mgYmoBlzYjD\nw8MiK4JJWAflnIwdjs1ioyQcQvgMFmT7OD9HRHcAvBzABzf5WY71IcnTImLOF5Yq+ODgwMnX4dgi\n1vaEiegMwGNYKF4A+FoiegmAPwsh/CGAtwF4MxF9CsCzAJ4G8DkA79pIix03hq4jrOtHaCXMapj/\n1+FwbA7XCcx9E4B/g0UALgD4qavn/xmAvxtCeCsRnQJ4B4C7AN4P4NUhhMkG2uu4IbStYClhzxF2\nOHaH6+QJ/1ussDFCCE8BeOp6TXJsG1aKmvSEuaSlZUc4ETscm4VnR+wZtAqWSphVMFsRVmCO38Ph\ncGwGTsJ7hKrAHKtfHZjT6WlOwA7HZuEkvGe4jifsecIOx/bgJLyHiNkRUgnL1XLuCTsc24PXE94z\nWARsHTo1LSfIPsq+6hmA3EPPetT766UMXRlPH7JPVZu3attJ/4/8X68pvRk4Ce8RtAVRRcC5Kl9u\nMwcW+bD6KXcPmc1mxc/z+bz4WRNWarBWM/JMRs5o5vM5Li8vi0eLlK334u+ID/m/gBPxJuAkvGeI\nqWBLPeVEwLKtsh6GzPjgg62XEAImkwkmk0mxi8h0Oi1+zmWnaT274bKksg4I32jkDUerfgBLY4Bv\nRtPptPhfIlqaNTiuDyfhPUMdNZyrEpZTac74kGTEqwH5MYSA0WhUHOPxGKPRqFCAl5eXhRpOFZqA\n+eZzcnKCs7MznJ6e4ujoqHSTmUwmmM1mJSLlPQR5XPDjfD4vtrgCyjci3/h1M3AS3iPUJeAclTBD\nV4bjQvWnp6cFIXPN5Pl8juFwiMFgUEzbuc9MwHJanirh6HPKdaHPzs5w584d9Ho9jMfj0jGZTAry\n5QPAknXD3wGAJRsjx/GRIpyE9wx17YiclLDlZXa73YKMbt26VahCecznc1xcXEQJeDKZlNR1qgSs\nzykr4du3b+Pu3bs4OTnBaDTCcDjEcDgsfHL2iPmwvPTpdAoAJf/ciXizcBLeI+iMAUnAbckJlkpY\nK8Jbt26VjtlsVnimPN2WBHxwcFC8Z4oEDCzfVLnfp6enBQnfunUL/X6/lP/NVoMMQsqgHvvm0hvn\n70XaGI6bw0l4z1A3OyI3MpZt7XQ6Rc4zK+E7d+7gzp07uH37dvHzbDYrlmZLpTeZTDAajUrqOGVo\nItYkfOfOndLO2eyZ62AdERVBTLlycj6fF14yk3du4yNlOAnvEapyhGN2RE6Q/dN2xO3bt/HQQw/h\n7t27xeN0Oi36KIlmNBphMBiUihdJpKSKrZWPR0dHJRK+e/duiYBDCOh0OkXGAz8yCXPg8ujoqHid\nvDEdHBzg8vIy27hBanAS3jPoC1ZOT3P0g4GyCq6yI+7evYsv+ZIvKY7pdFooYCYZJmDtE6cM6+bD\nJMx9lQTMWQ2cjsfeLxEVWSR88Ov07ts5jY/UkS0JW8EYfTFe932tzwCwFE1mTyz1hH4JSwnXuahS\nTc6X5MM3lJOTkyL4dnZ2VhwyKHdycoJut1t67enpKfr9vrnJqTzXKUFu1MrEKbNA+JA50ByE49/5\n6HQ6S2l8RFS6QR0fHxdjn20Jx82QJQlLItFT6E0pOev9qhLegfQuUA0rRU0G5axpd8rLVLk/uvaF\nJCBJxEy+TFJM2PK1JycnBQHpUp6p9R/A0s4oFgGfnJwURMvjVpIwWxKshNmKYCU8Ho8xHA6Lv8nV\nha6Gb44sSRh4ISF/la95XVg5s7y6ajJ5YZMQJuBcVg5p4qpTJyLFfln2AxOHVLZ1lLBUw0zCVRud\npvR9SBKWKliS8enpaUk4sJ8rCZiVsFxleHR0BAAYDoelmxP/Xy52TerIkoSlEo7VBliXhPVr9fsR\nEUajUSmVSSrgFC9QjVhgziIajZQsF205aSWoiVUTMavhg4ODJdUol/rKG1SqObGc22v1Xx6SgC0S\nloE5eYQQSjbN0dFRIUTkzTuVsZEjsiRh4AUFwANQkjE/1r1o9OukUuSDP5MHHOdXAmkR1CpUZUbw\n9xCrmsV/SwnyXLES1D6vVLnSjuh0OkuKsUoJpwZpLVlKmPsqSZiJ+ODgoFQPwkpROzo6wuXlJQaD\nQfFeuuh/it9LbsiOhHUkmAcfe3gy0XzdoIFUWPwefPAA5epbPHABlCLOqZGUhJXOJJU+sEyyMTJu\nGlK1y6W6nI52+/btYlEGK+Ber1ciV5nSxf/PRCM9YVbCfI5TgJ4FcPul0pfFi46Pj0txjG63uxTf\nIFperDGfz9Hv95eCfJPJBOPx2ANzG0B2JAyU1Q9PwVi9yDv5TUhY50vyZwKLnFJefy/rzrIyThVW\nYM5Se1U1ZFODVbRGro7TFoQmV3kj15XHpGeeIgED5f73ej2cnZ0t2SrcP45bMNnKFXOxZcvz+bw0\nk+j1ekWmBH833KZUx0jqyIqEq4IxPOgkeTJprvveHCXm9+z1esXfmYAHgwFGo1FyCmkVZDaEDswB\ncQJOhZCtlERJwloJsxdsZT3ElLB8XcorCFkYxJQw9+Pw8LBEwFxfQtaO4DGsYyGz2Wwpq0RaEq6E\nb46sSJih05J4AGrirDtAdECKvULpqXU6Hcxms4KAz8/PiyIoKV6gFiwlXCc9rWnirYK2I87Ozkwr\nQithXvWlx5BFMimdXx2U1EpYp9nJABu/nr8vK+dd3nTYN9ZKWAcuWQW7Gr4esiRhAKYSlqTJ0e86\nsEhYqorT01N0Op1iNdWDBw+Ki3k2m5WUZOrQKWp1gnIpErGlhHu9XqlAT5UVwe8hMytkzWFtR6RK\nxKuUMPcDKKezcZ1kXYxH59yPx+NSGVB5k5J2hOP6yIaE5eC3AnM6NYnzQNd5f3lIAj49PS2e40Fo\nZRWkcIFWQaeoWfnBsa1vgHSyQLSil8V6OCXNmpLrpbt6iq5335CzhRRWh+kxGrsG+GbCRCnFiPzO\n9HnWn6FX0El1rbMjXAVfH9mQMGPVAOSL8PT0tDYJaz9YKmEm406nUyLhFKeqdaAvsioVbF2kTUO3\nXaenyfOv/d1YiqHOhkm5ngb3XcdFZI5wLJVMer6xWY4VG9EErGcJkoCdjNdHdiQMLGdH8FRML1Nd\nh4TXVcKShG+6Om+X0CpSk7D0CFO1JWJKWJOw9EYPDw9L7yH7KN9HEnGKdoTuuxYi8uYj+8G5wdaN\ntep86oI+VuaI42bIkoSB5cIlWgnfunVr6cKrgkXC8gCwRMJSYTR9cdZBHTsiVeJlWH3QJCTtCHm+\nJPHKYJS2I6qqy6XQdz0T0NeAznXWZGnlgsd+t7KOrPEfey/HamRFwnKqpFWQRcQ3IWG9lBXAUkAi\nJwKW0H1lWFXiUrMkLCvCWrJrBeKYfPXqMRmU0ilacjFL0+daWwqy7zrPOUbA8r00LO9f2zRWwNJx\nM2RFwgxrOqYH4snJSW0SttJ+tJdo1Q6oM51LFZJQtA+ccmDOugHLql86KAW8sGWRLt1obc8jvU1p\n2aSihJmA5SIMeVgztet+Fo95fb1ZNUeaHhc5IzsSlkQZI2FWQ1wFat33JqKlAASvKGLkTMAWtB8c\nI+KmIQNS+tzLm7BUwHJ/NF5uy3ulaWuGP0MSTypErAPS+gYUywa5yefpmUedgGUqYyUXZEfCQLyO\nrLwQ1yVhOYhktFwvW40NtlwGnu4nQyvhquBck9BqsEoJM6my7cA7RPARQiiRuVR0knhSCcxZY173\nu44VUedz5HlepYRX/b+jGtmSsF52avnC6yphCe0J6roQKRHTurBslVhmhPxZvr4pVFlRVXZECKFQ\nwcPhEIPBAABKxcslYVlKuGk7QgfjrH7XsSPq9EPaEZZXrgO70tpyrIcsSNgaNDo4o1UBD8brfp7O\nx2xaBW0asYyI1C0Ja0quVaFcTAOgCMKNx2OMRiMMh0P0+30AKAXlut2u6YOmpITr2BFWfvB1Py9m\nSaTwfbQFWZCwhhUljqmi67x3nedTIqbrQhMxF3SJZUWk0N9YZoRe1SWVmbWTcr/fLxEMryBjpOoJ\nx/qtf5aLU276mbGbkvaEmx4buSIbEtZ3XWtaKgemVsJ1p2AMK5+UoYNyKZDTTRBTwamlpwFYuvla\nipAL1sh+TKfTkhXBJMzKUpZ6BJbT1ZomYO67zg1etZhiE222LImYEnYyXh/ZkDBDT4+0Cq5SwqsG\npMyOkLmkvOtAm7IjYjnCUgmnaEmssiP4/PN29qyE2Q9mJXxxcVEoYL5pSxJO1Y6wvPBY8aFNKWGp\nfGPZEfzapsdHjsiOhBlaCcfsiHUUsBxQnEsKYEkJA1iapueEKmslVfJlxAJzmohkUJFzhLUdwSUd\ne70eZrNZqaBPqnaE5QlvUglbRFr1XTgB3xxr5a8Q0ZuI6CNE9ICI7hHRvyCiv2C87i1E9HkiGhDR\ne4josc01ubzyiS8sfYzH49Ihd0rWx3Q6XVo9xapQXsD8Wr3iKjWiWgcWAcd84RRuOpYa5EwIJgQe\nG2w/9Pt9XFxc4Pz8HBcXF+j3+8UxHA6L8TKZTEpbAAFY8j+b6rOVEaTzgm9a60KLEU2yWhU3bc+0\nBesq4VcC+McAfuPqf/8hgF8loheFEIYAQERvBPAGAE8AeBbAjwN45uo1E/Nda0CnSDEBy4tM50ry\njhh6QGnI/EeexvH7y4j6aDQqyNgi4lxRh4Dla5uElRFjLcxgAmbSffDgQXEwGR8eHhYLezQRW/V1\nmyAdSYJaBVvV0rbRXmumaP3uuB7WIuEQwnfJ34noBwD8fwBeCuDXr55+EsDTIYR3X73mCQD3ALwG\nwDuv00ieIsrf9YXGecEyUjydTpdSzKxBI6tncQI/2xGShFlVSzUsCStHVFkRKQbnLCUcWx3HY4PJ\n9/79+yU1zCR8enpaScJaEe6yr/yog3LShrBKS266nVrMaDJ2XB839YTvAggA/gwAiOhRAI8AeC+/\nIITwgIg+DOAVuCYJa+gLrd/v4/z8vBScOTo6wmQyWVqWag0YHtDS+40pYb5QNRGnQlLXgSThVUq4\nSeiALJ83TcJ88+QA3IMHD/D8888XKpiJ+OjoCKenpxgOh6WZjiRhfRNvos/8uVIJy1x4vUKuzuzv\npu1xFbw5XJuEaXEG3gbg10MIv3P19CNYkPI99fJ7V3+7MZgUtBLWe2odHh5iOp2WIty8guqq/cXP\nlgfIPrBlR2hfOCWiui6qKqgB6aThSTVYpYSlH3z//n3cv3+/ZEX0+33MZjMMBoNi01bLF141i9oV\nLCUc2xl6G3aEq+Dt4SZK+O0A/hKAv7ahtqwE2xLSEx6NRuj3+0skzHaEVZKQIQM5QNlvlCQslVKu\ndkSMSLQdkWpQTk7NtSdsVUyTG7Lev38fzz//fCk4d3FxgdlshrOzs+Lcjkaj4tzGlPCu7Qh5WEpY\nBua2UXIzFqyL+cNOyuvjWiRMRP8EwHcBeGUI4Y/Fn54DQAAeRlkNPwzgY9dtJFAmYCYLtiN0nqhU\nwrLgSLfbjd7FpcqYzWamHSGzLGIZEjmiTnYEv64J6OCUJOEqO0J6ws8//3wpK6Lf72M+n0dvsDFP\nuKn+65sP7yZjFezZdmCuiowd62NtEr4i4L8B4NtCCJ+VfwshfIaIngPwOIDfvnr9HQAvB/CzN21s\nTAlbOaO8E7LMI2YS1kEWHuBM3PwY84Q5aJeLErYuEvk91Fktl0qGhFSDlh3BaYXSE2YlzNYDH5eX\nlxgMBgURj8fj4gassyN2TcbWGI0p4arA3DYtidh78zXqqIe1SJiI3g7gbwP4HgB9Inr46k/3Qwij\nq5/fBuDNRPQpLFLUngbwOQDv2kSDdY4wX3xy9RMT7mw2K+0GwOUK9QDVPpusNcAkzIdUwnyx5kDE\nElWWhFT2KZSyjGUISEUoy1bKkpWShO/fv18i3OFwiBBC8RyfXx5beuFGk4pPBubkkmW9h55eXr0L\nAm76u2kD1lXCr8ci8Pb/qOd/EMDPA0AI4a1EdArgHVhkT7wfwKtvkiOsIZUwgKLWr94jTJIwH3yx\nyoEjk995Olo3RS21DAKNuheHpYYlEacCOWuJ5QmHEAo7QnrCekEPgBIJj8fjYmylkCesyU5nheg9\n9HT9YNneTZxDeSP0IN3msG6ecK0VdiGEpwA8dY32rNOWwr+UK9tkfYAQwtL+WBYJMwHL1XOshmOr\n5VJM4aqCvICkmtTqKUULQgen9LLlTqdTyphhEpYzF3n+5DnkscPjB1gub9lU6UbpBcs6KTpHOrZE\nWWe2rBqv7vU2gyxrR+hovq4NMBwOi/oB0o6QRU3kIDs+Psbp6WnhCcoLky/eNuQG6ym93qwxZkEw\nmu6rtiPk7IatlMlkgvl8Xlq6zudOevny5i1vuJ1OZ4mErZvVLvqqbzo6NU/bEFIFS8RmORq6UE8s\nluDYLLIkYWB5qa3OZjg8PMR8Pi8RsEXCRFQsW5VKWFoSVo0JnTXQNEHVhSYyXfg7JfK1lJlFSEw+\n8qap64doJaxnUXy+Dw4OSjnjsfKNu/hOLOWvg5JWAXedzaJnjTyG+TMY8nNcCe8O2ZMwbzuk83p5\nSyJJwHJwSW+Ll61qEq4q3qMXMqQM7StaZQljdoR1QTfdB01IUgkz4WolbC2y0VYWl7+M2RFNKGHu\nb6xaoCxbKZWwRcBylmdVBuQbEI8TC6mP9RyRJQlLO4IvCqmEZREeOZ3USph/Pjs7Ky1ZtS7OKhLO\nBVaGQR1PuOl+WjcRaTOx+uMbp5XNov3gKk+Ywd9LE3aE7q8uZB9TwhLyPHJfpSWjX9/tdot+y9Wl\nGk2Ph7YhSxIGlslCB+c6nU6hhK0ppXyUCzHkxSovTn3x5gp9QUsVJYmY0fQFZxGwJiRdiN46n9YS\ncyuuoLMhLDW8q35r+0XbEXzEbhBVsZPZbFb6fgEUxHxwcGCKDGsspDRWckW2JMzQ9X85d5iIisUa\n1tRbXmy6eIvlFVppaalDpw9pYlnlCde5CHeBmCfMfWAVzOeJ086YiOXNUwcedVyB1aBeaWnl4G77\n+9DKX+e8W4V7GDpeoUWKRcKXl5dFf1cJjVyugRyQLQlbimY2mxWDkTMjpAcaO7RykgSs1XBuCzMk\nYiSWKmKBOZ2uNZvNEEIoAnKxZcjWudMEJXOOeQGQlaa2i/Ovlb+2I2T6pTWT0TcaadlxmVf+nvn1\n/P6xMZ7SzbktSPcKrAE++VLJ8AXCd3XLfuCBzT9bFbS0J5y7HwzEo+110tJSyhPWNxEiWircY1W8\ns7JatBLm18kAoBXA5HZt63uJ5UVbecJWcaoqu459ck3CAEr2jvVeMeR4PaSCVpAw8EL1LH7+8vJy\nKR1NK2C+sCzVpKPnue6koX3CmB3BfbbS7pomYNnuKhJmJRyr+8zjwgpYyXMsU9QsJbzLvsvsCEsJ\nW5YSsByU07n0TML6OD4+Ln0HGnWCtrlcG6kgWxLmVBr+Wd7x5d9jeaaclM9ZFLpmglRIWiXlNsgs\nVSUDTvpmltpCFOtGqtUpn3v2PHWRpSqlL6frVllTDn5ZmSTbUsOWcNBlWfWCG3096PErFzNx8Fr+\nP6d1xs6/2xDbQbYkzNBTSwYTLFBeoswDjSPABwcH0bKUmoBzXSnHsKb0/D3IfqVExDqd0JrNAFhS\ne9Jaivn4+uYtzy8rUL3aMhYA22bfpRq2bkJ1MiP0itLRaFRK3eS+yu9Lv5/+zhybQdYkzAOBp6P8\nHHt6lueliYcvXn2xWko4x6BcLLAl86Zl2pXlmzYFef5ivj5PxZlIZeBJro6L9ck61/yZ0vKI+cL6\n/bbxHcTUcKyuRSzzg7+bwWBQ5NNzEI4DnNZszwl4u8iahBkyQMewvFAAhQrmgBSTcB1LIpf6wQwr\nRc26mHVAMxUVLNsds1SYgCwlrGs+a1i+cEwJx8iO27jp76rqfFURsOyXVsK6NvbR0VHpfFetBrUe\nUxgfbUArSJhh3b2t1xBRSfFUeWCShFMip3WgFaW8kKUStki46b5aBKx/57bLaXfdxTX6hsuv1QFM\nvWCDZ1u7sCNiVow8LCKWIkLuNtLv9zEajUrfjfSDV3nCda4zR320ioTXgR7cWnEBy4oipqhSRpWS\n1FF1vmBT6WuVHaHPUyzVbBVJaMWolXDMjpDHLohIfxfWz7JPsaAcF7rnAvbAC5sa1LnppmBTtQ17\nR8IWKVlZFIyYOsxtEMai7fy3WICq6X7GCFgvnOC263TCqj5Y3im/VvrmkoD5kYla217b6n+MfFf1\nK0bCw+Gw6KdcnBEb35an3vTYaAv2joQZVWTMsJRwCsRUB9aNxQrw8GuBZSWcSl+rbpgMne8rA6lV\niM10VtkR8rvZlhq2PH1NxBYhx4JymoRZAfOWXnVS06x+uj1xM+wlCcuBrD21KiWcEjGtA903GeBh\nRQegNIVNIfho3UgsMrKUX101rwlLkrAOzMXsiG31XX4H+vuQr9H9qauEea+609PTtce2+8Obw25K\nQiWIVRe29oVzJmB+tNSwDGzxlF7fdJqEvmHGSEhnstRpf2ymw++tMxF0Rsa27YhY31dZE9JWsrb9\n6vf7RXCOg5hyxlBlR1i2hP7ZsR72TgmvUlcxAs7RE7YUVJUnnJoSZqyahmvlt66vHSNhuapQe8Kc\nIcGv3Xbfq8ZprD86ZY+XdHNtDc6nrrtMWWeR5HQtpIy9V8I65clSFVoN54SY722ttEo5Ha8u8egs\nh7rZEZqIY/m5u1bC+sa5yg7RNoTccYZT1PhYRcJS8eYaG8kBe6eEgXja1ipPONc7v9VXTShSPaVk\nR3D7+THmi1pKft0UNcuOsIJysdzcbcBSv9ZnW9+DJmFeqMFKeDQalUg4RsR1ZoQpjJNcsbdKGEBU\nWehpbq4KIDaNl0RcRWIp9LNOgMqyI+qcL31udWBO2xGWEpZt3HS/VwkF63vgRx2UYxJmIq6jhK3v\nqkqUpDBecsTekfAqZWEp4tzv/FVBHKDsIVpLuFOBNT2OnY91222RdcwO2JUSjs1cqtIp+TzKbAhd\nX1mSrrzZ6OLw/L5aWVeNj5TGSy7YWzsithTU8oZzJmBGlZcqg1m63kLTfdWEGzs01iFHTe78/1UE\nbHnqm4T8/FW+tCRgAKUlyhcXF7i4uCiK9vAuJPK95bZJujyozr+29lvMaXaYIvaShIHlHOGqJbEy\nKJfbYKsiXz0V10o4pf5el4zrvK9+f0aVl14VHNsE9JjUBBxTrCGEgoQ5He38/BzD4bDYbw9Y3vDV\n2ipJWzxrjgPvAAAgAElEQVSrdh1PZazkhr0j4SqvrSrqnfNAq7Ih1p1u7hrXId11vdrYTMcaHxYR\nbwtVSjjmB+tiPayE2QPWSlhvHipzx/l9pVUlt/pyAt4M9o6EgXpTzSpPONcBV+UHp0jC8rPrkvF1\nSNF6L+tGvUtfWNsRsQwN3YfLyxe2eeKKaefn5xgMBiuVcJUdweQesyN0Oxz1sZckDKwu5CORO/kC\ny+Qk+xSzI5omYaut+rBsonWIcRWBVBHxrgNzVdsZ8aNUwrxE2VLC8n3r2BE8Lqo2T01hrOSIvcuO\nAOI1ai2/bVPeYxPQN5VVSpgv4JRIOEa6sYu/KoXLem/9GRJ1VPC2UCcwZ41TVsKTyaRQwkzCUgkT\nUaUSlt8Rjw+983gK46MN2DsSrppmxi5eS3GlPvg0QcSIuCowl8pFVqV+b2pFWJ/B7xWLH3B+9bbV\nsByb1tLpWCpZLDuCSVgqYU5NYyW8KjvClfDm4XZERBXHpu/8c06IEXAugTl+1Be8RQDrZi2s8per\n1PA2CXiVEo4JBT6PchcNLuLOxXr4/VfZEQCWxoZFwo6bYS9JWJKSdVFV5Qk3TUrrYlWmAF+8suBL\nKqvmLLugjj10XWJcRcCx8bJNIraUsDz0d8R2hCZhthGkEl5lR8ixoe0IXTQ/V4GSAvaShIHtVr5K\nDav6ak33c0LMYqpDkFXBWf3/ux4zdW4+8ubJWRFcIW08HpdWyQEopaVJ9csELPtsWVV6F+vcxkqK\n2FsSBvaPiGM2Cz9W+a0pIhaEq6tQ5SyhSuXq99LfzTa/pyr7hbMhdM1gScBMwgzu39HR0RIBWypY\n1iWO2VWugm+GvQvM7RPqKGB+rDvVTxUxX7+uEt6lzVAXqwhYEqUs0qNVsCzaXqWErYwLTcSpxQza\ngL0n4RQutm2ibt9yIl95zqrsBP3aVe+3LpFv83uqc4MEULIj2AdmIpZK2LIj9P55VlU9HZjzJcub\nx1okTESvJ6KPE9H9q+ODRPTX1WveQkSfJ6IBEb2HiB7bbJO3h7aScRWZWMoqByIG4nWG18lcWEW+\nVhbCrr4byyrSxBcrV8lq2FLCh4eHhR2hLYmq9DSrgE/qYyQHrKuE/xDAGwF8I4CXAngfgHcR0YsA\ngIjeCOANAF4H4GUA+gCeIaKjjbV4i7AGVA5kdB1oTzMnJRxDLLi26n9W+cH69buEJt+qwFyVHcHn\nkzcvPTo6wtHRUVQJA4h6wpLYPTB3c6xFwiGE/yuE8CshhN8PIXwqhPBmABcA/urVS54E8HQI4d0h\nhE8AeALAVwJ4zUZbvWVYJJQrMa0Di4Cb7nOdz99kYK4qS2KXBLxqhqLtCFbCbEcwAcuNPIloiYR1\nfnAsPc2qHZFrZcHUcG1PmIg6RPT9AE4BfJCIHgXwCID38mtCCA8AfBjAK27a0CYQU8Y5w8qOyDko\nF/OG6y4rjlkZKQToNPlapGcp4ZgdIUn4+Pi4MkWtTnaE2xGbwdopakT0YgD/HkAPwDmA7w0h/B4R\nvQJAAHBP/cs9LMg5S6SoDjeFKksiJWhvVKPKD65LxFVL2Ou+z6ZhecGrsiO4UE+VHcEqWKapsR3B\n78fvb1kRsvB/amMlR1wnT/h3AbwEwEMA/iaAnyeib91oqxKB5RfK5aOSFFIm6VWEYpFcamR8XVuC\nn1/1P7Imgzzk/+/ajuBHScT6kMXWWQVLO0IW6Oc+WoE5XY+CCVgrYV1bxD3hm2NtEg4hzAB8+urX\njxHRy7Dwgt8KgAA8jLIafhjAx27Yzo2hKpIuLzZeznl0dITj42OcnJyUdt89OjrCdDo1L4xUVGWd\nyD8jJytiHTW8ioBl/QRd2LzqO9oFtNq1FCmrXrmbMqem6Y1LJQGzHRFbqmwRr14pl/IYyQmbWDHX\nAXAcQvgMET0H4HEAvw0ARHQHwMsB/OwGPmej0Bervui0auj1esXvh4eHpToLMYWQwkCtuulISOWV\napparC36BlpXscqbE59bSxVa7dj2d6MJ2CJiJlvpBbMfrDf0lDcbaUfogj1VJCyVsdsRm8NaJExE\nPwHg/wbwWQC3AfwdAN8G4DuvXvI2AG8mok8BeBbA0wA+B+BdG2rvRrBKDfNzrIyOj4/R6/XQ7XbN\nvEme+vH/cmJ8CKEIcjTZx1X1FCwCTg2W7aOxbkaDVsLW6jH5+dZMYdtErDMUpBKWJCyVsJUfrGd3\nTMCxRRpVStgzIzaLdZXwfwLgnwH4TwHcx0LxfmcI4X0AEEJ4KxGdAngHgLsA3g/g1SGEyeaavBlU\nEbAkYZ669Xq9kiKRJQN1IWwGD+imsMqOkEg5O2IbBCz/R9oRmpiq3mMXSjhGihYBW0pYLtLQ6WlS\nCcfsCFmwR29rlOoNOzesRcIhhNfWeM1TAJ66Znu2jhjh6ue0FXF6eroUqZ5OpxgOh8UFqxVSExF1\nDd3HdYg4BVjtqJMlUZeA9TTdqqmrP3dX2TI6AGfZEFoFsyfMhMl91SSsPeFVSlgKD7ciNou9rKJm\nqUOtFHnAMgHfvn17adBNp1P0+310u10QUYmgeSA3ZUdY/axazhtTwqlcaKvIb53AnPSPpRWhp+lS\nHe7aM7dUMBOwVL5MvDozQi9VrtNPoFzEnW02q3yl2xGbw16TMF+ElhJmZdTr9XB2dobxeLx0kU+n\n0xIBy3Sg+XxuWhRN9jVmSeh0qNQuLE28VW201LAkYv2z9v4lOdWxbLYJS5WyCpYELI91SNja2JPH\nrs7CsHZYTm2c5Iq9I2F5YVYl6HNSOyvh6XS6tMPBbDYrdjeQymE+n6+MsO+in1VKOEYwKWVGWORb\n9ZqYtWRBnmcrd9ba8HKXMwVJwJoU2YKwSFjuqmz5wZqEq4r2yGXPMT+46THSBuwdCQOr9w6TQTkm\n4fl8XgredLvdYqDLPb04KGIpqab6uS4Rp3RxxVSwbGPM41+Fukq4KZtGE7G2I6QNIX9m4uS4RMx2\nkUqYYZWulHaEp6dtHq0l4dgUVO6rZUXE+QLWnvDl5eVSIezZbIbRaITBYIBer7e0BDQFO0LmwWoi\nlrAyEFIh4zrt0Dcaa8GFvvlIAuZg1bp2xLa+o1We8HA4xGAwWPKEY+Ur5WIUyxOWMY2qzAhrQZLj\nZmgVCVsXm35Op+jo9fNaCfNquRBC8Tom4ul0ipOTk1KkWZNck2pYq+CY/81IUQlrWG2zMh1iOxNL\npSzV4fHxcXHIabpFuLuailuWhPSDB4NBQcSxhRoy7mGVsbQUP8/urII9TsCbR2tIWF9o+oLjR+n7\nVa2h54uz1+sVe3TJ/zs8PMRkMkGv11u6cKuyEHb9fWgfO6aEJVK9uGIqXROwvqFaWRNWsKrX65W8\n0qoblDVz2HRfdaaNtiO0EuaiPTJADGDpu7HGO0PbH5YX7J7wZtEaEmZo4tWHpYT16iGthDkAJwcv\nP2olrC/+pr+LGBFXqWD5eyqItUUSjeyn1Vc9NjQJW0pYf/YulWBMCUs7QiphJmFZZjLWV77ZyO+I\nP08q4dhCjdTGR85oFQnri01Ow/hnTcK6rqq8mHmaenl5WYouy9enqoT5+9DktCqHNsWppr4xxOwI\n6fevsiP0Ig1eFRkj4VXHNvqsSVFWSxsOh+j3+xgMBks1hOX3ZZGwVsLajogVcbfsiJTGSa7InoRl\nkEUrIR5kMgB3dnaG09NTnJ2dFT+zmpVTUR6wugxgbIVRilM1i4TljYJfw2i6vdeBleGgz6W8GctH\neUNmEtZ2hOUFN+UJy8Ltg8EAR0dHhRKWVoQOSuvzrwPI8vU6OOfku31kTcI66KYJV3q+/Hjr1i3c\nuXMHt2/fLo6zs7MlW4EJmAefVI6sFmQKj5XYnkrdiFVTdMuWSBmx886+rnVTtQ5NwHzw/wDL+dO7\nutHq7AhJwsPhsLDQ+v1+KTWSFwlZs0KrRoa0zazPjGVF8OsdN0fWJAzEl6DqiDcfTMKSiDUJy/eR\nnqMciDJwoUlYenJNDlSphPUUXXvWTVsndaGzTriPMrhWRcKSgKQPXKWELQW8i3MrCV9va8/WmUXC\nAErnWNsvVSRch4CbHtdtQ7YkbE25rLzPk5OT4sLs9XolBXz79m3cunUrSsJSyRLRUpBCpvDo1UVN\n7zpgqaAqnzRHWEpYqlq5m7DMldUzJfk/cgxIO0Ir4V16wlyDhElYLrCQdgSTsE7NtGy6ukrY6wdv\nH9mSMLCcliY9L76wTk5OcHp6WhyahGN2BA9ISWYAlpSwdaSyCaKlgqyVc1Voug8W9A14HTtCzpSk\nHcE37OPj45Lvr4sy7SIox4iRsAwYyqXKkoT5NTErokoJ6wyJKjWc4vjIDa0hYSsZnS9IDsLdunUL\nt27dqqWErQHGQY8QyrUi5H5eKRW9tjxhayVZDClfYLL90o64vLzEeDwukbBWvzolTSthmcpW5Qlv\nE/Jziai46UuFPp/Plzb15Lol8nuq6wnz58piVL5IY/vIloR1cEan4fCFxZkQknSZiPnx9PS0NH1l\nEub3Z1UxmSxq0+uUIWuNfQp2hF7GWycwlxu0Eg4hFNaTJmGtgK2g3PHx8ZLSs4Jxu1LC/PkAihol\ncvGG3GVDC4CYHSVJWAectSdsKWF+rWMzyJaEAXtZrvb5WAlLBSx/5zQ16Qfye8n3DyGUpqiagPUS\nz1T8M71kObaQRPuCKbS9DmSKGhPPdDotparFSFgHbZm49Q4qVWQMbIeQtAKXn8UEyfVLZIxCtq9K\nBdfxhLUa1mMjlzGSOlpBwvJCjClhzopgJczPyVxhHRnnqSArYZ7mSSUcqzSVgh1h5QintKLvJrDO\nPavi2WxWUsJaDccImNPT+HyuCsbtSgnLLA1OQeND56cDKM3AYkG5VZ5wlRJ28t0ssiXhKjtCesLS\njmASZo+YDzkVldM0OdABFAOWSVhbEXp1UdOD1cqQaJsdAaA4XwcHBwWByCXIsbxxbUXwZq7ACzda\nHZjbZRWx6yhO6WNXpafVTVGL2RGOzSFLEtbKl5cPn5ycFDbDQw89hIceemhJ7TLZWtXTGBx404W0\nHzx4gAcPHuD8/Bzn5+e4uLhAv9/HaDQqbayYwnRN36S096c9z1Sj3vK7jPm0erZydHSE09NT3Lp1\nC3fv3sWXfumXmuqXxwfPgrRHan1uat+PBt+M9GxQxzw08QJY2sk5VhDIsVlkT8I83eRUNFa9Dz30\nULEo49atWytJmC8+GR3m9fi8Vv/+/fvFwWQ8GAxKhVNSvUj1Igc9zdaP/JomYRGwtURcTs955sLp\niHfv3i3qPcuawXVIOHajstrXNGSmiFyIwtdFbCEKgEI8aAKWWReyPrFjs8iWhK2cYFY/d+7cKZSw\nTkPTUXNdSwFAMSDH4zEGgwH6/T76/X6JgJmEY0o4JTK2bIeclHCVCpYkzFNuACUSHo/HxXmXdoRF\nwkA5DzmH/Fg925FxEU3CnCsubQs+JAlLJSzHtmPzyJKEgfKuCFY6mlTBOgNilRJmO4JJ+OLioiBe\nebAS5i1lWAmn4p1p8pUKjx8lwaRKNlb7dDBKpqp1Op2ChHkRw8nJyVIVPDlLYhJme8MiqVRtiVi+\nPJOwnAFaS7Ll4gxJwJYdkUJ/24YsSVjaEXrapT1huVpOrqTSGx1aSng0GqHf7+PBgwd4/vnno3ZE\nlRJOBTr4VoeAUwAHmCzy1TmxTD7AC0qYp9G83FcG6ThTRith+dmpEi9Drx7UnrClhC1PWOYduxLe\nLbIjYZ2WJAu3WJ6wVAJ8scmi7HJ6FlPC5+fnuH//fkHEVXZEahfqKiuiioBT6YP2hnW2AlCeGbES\nlptdTiaTUmnTg4ODgqj55syLdCw7Qs9uUj3HUpzIa0J7wp1OpzRz04FoqYadhLeL7EgYWK6Ypj1h\naUfIYt06ZYkPfk+GVsLn5+dLBPzgwQNcXFyU7IjUVxVZRNxE/mtd6DZYBKztCD6/p6enBaF2u91i\nyS8T0MHBAXq93pISZjuCP2/Vd5PC9wQsly7VSljWxdBpaTolzQNzu0VWJCxJJBYFlkr49u3bS9sY\nWXVlgWW1xSQ8GAzw4MGDJSuClTDvapCqEmasQ8D6dakgFpjTdgQvX2ZV2Ov1ii2q5HF0dFQK1vJC\nj1gGSWo3KYalhGN2hN4/T9oRdVLUUup3W5AVCTOswWYp4du3by/tCaeX73JKk1ZZnB2hlbAk4ouL\ni9LyZTllS2HAxqwIftR9TpVkrBuDVsI6IMVFfXq9XjFLsQJYMlsitow3ZRUc84SlHaGX5a8iYfeE\nd4tWkDAPOFm6krMiqpbsykNe3Byk4Pxgzo7gBRq8SGMwGCzVGE7Zili1WCPFoKIMzPHvMjCndxVm\nP5QJWb9Gfg/Wkm59w4qtkkvl+wHs7Ahd2pOXZFt2RMwTllaEL9bYHrIhYR3d10V75FJMa/EFgCKn\nVC7n5GXIslCJVAJsN8jdbHWhnlRS0uoiZkfIv/HPqUHXNpB+pSRame0iSVxbWnIc6BtxbHcJjRS+\nJ+0J66JFsdKV+vusKuaeQj/biGxImGENNl2kRaacWVF0fcgqVEzCHJiQJMxEvGoH2tRQx45Iuf2s\nhvlnSRp6Pz+LgDVBy7/FMkcsYtJklML3pWd0euEKzxR1JhC336oVoa+HHIVGTsiOhAEsTbu05yvv\n+Bz95QuGyVsOVjn4dHRYErFUwlKB5TZIVxFwCuRiwSJIq2YHjw8+/1yKVEMuYokpYUnAqWe+SHES\nuzZipSu1Cq66+Tg2iyxJGFi2I2IkrP1DDsTx/4YQzL3itB0htxXXPlkOg7TK60xdyQNlb1irNm1H\nSNKtCqTFrAU9RdfZAal9R6uUMB91q6ZZtkROIiM3ZEXCWrnEyvNpP1gOLq0U+O+yHKVUwUzEWgXn\nYENoxKL9MSWcYr+qlLCM3uul2vL/+VGqX/ne63rCTUMTsEXCWpwA9g1nlR+cYv9zR1YkLGFZEnKA\nySwHVraTyaRUAnM+nxeJ/FUkrNfR50a+gJ3mlYsnLCFJQ+/vt4okNZFYN5+YCk6pTKmE5XFbu2lI\nAtZ2hCRg3V89U3JsHtmRcGzapX1gHlx8obKa5fxRVgk6P9IiYStdR6qunAZnFQHLv6cKSRq6kP46\nSjU2E9B+cA77rMXsCEnEdbYysgg49fHQBmRHwgxLCcs8TyZJGWgbDodFNgTvzMsLM/RKISZteWgv\nuMpvTAkWwVbZEfJ/UoMmDr2YINbuqptMjIjr7BuYyvdkEbG1i8YqT7hKBafS17YhSxKOTbm0HRFC\nKJQtL0E+ODhYutNLFSxVs2VF6J2UcxyYkmyqFiKk2Dfp81s7XNexI/jnVUo4ZkekREhWipq1p5yV\nIyzHgO5nqv1tIzqrXxIHEf0IEV0S0U+r599CRJ8nogERvYeIHrvh51T+zVr9I8mXCZiP4XBYCrZJ\nAtaq2No/LudgRZUXnMPFplWwzlhZNa220s/qHKkH5lZt6GnZEdrWqVqIlGK/24JrK2Ei+mYArwPw\ncfX8GwG8AcATAJ4F8OMAniGiF4UQJtdvajV4UEnr4fLysrQzxsXFRZERwYOq0+ksKSqdsG5d1PJz\nc4Gl+mIXWar90uQhZyrypipXRsrZUZX6t/Jlq8gohe/IWrRkVQqU+yhalktsNWhON+hccS0SJqJb\nAH4RwGsB/Kj685MAng4hvPvqtU8AuAfgNQDeef2m2tAKhwfUcDjEfD4vkXC/38fh4WGJgHmxht6y\n3krZaYtHVqWG+e+pQmdGWAHUyWSytCAHgDndBso3p1iubKrnWweouSBRbLmyTN3UMwqrYE+q6r9N\nuK4d8bMAfjmE8D75JBE9CuARAO/l50IIDwB8GMArrtvIGLQq0TbEcDgsLAhJxNKOkCo4NqXNZVpa\nB5qAc7uxSCUsSVcHUK2l5fyzTkmsOu8x9ZwKdIA6poZ1caLYzcxSwin2u01YWwkT0fcD+AYA32T8\n+REAAQvlK3Hv6m8bg452ayXM+b/ajuAyh3LQWiuwrOfk5+U8IGN2RA590iSsvX0+ut1u0R8mH31O\nreBqFRGnOFtgEl5lR1j1VHR6plXEPaexkSvWImEi+ioAbwPwHSGE6XaatIwQwlJU1xpIo9GotLR1\nMpng4uIC5+fnxTGbzUolD6fTaUklWXf/FC++66DKD646UoJWcDzjkbOewWBQFK5hoiWiJfVrrbDT\ntlSqCpghPWEuXak3s9VBOes75IC13s4o51lfLlhXCb8UwFcA+Ci9wIoHAL6ViN4A4C8CIAAPo6yG\nHwbwsRu2tQQrsMDkwoQ8nU6L+r98zOfzUtHrXIvw1IF187CsiBQXIFiQwTOZzz0cDkuzHd7Kh71c\n9kJlAIoLvQPlNC8r7zjlm7DeU07uFKI9YcDOIJI3MN6qS6f8pXoTagPWJeFfA/CX1XM/B+CTAH4y\nhPBpInoOwOMAfhsAiOgOgJdj4SNvBLHoLt/deQo2mUxKXnC/30cIoRiwfNcH0EoSZugLyTpyIWIr\nACt9fyZhScA8FdclSi8vL0sZFPya2AKNFL8Xa9dxVsOSiGUqp/Ud8iYFw+Ew6+X5OWItEg4h9AH8\njnyOiPoAvhBC+OTVU28D8GYi+hQWKWpPA/gcgHfduLUoE4n2BnWR7slkUpqiDgYDACgGK19wViGX\ntsEi3apVUal+B1YeOBOxJGFuP5OUJGH+H54VyXGTmycq/WCphOWmtno7I2nX8ffBJCyVcOp9bws2\nsWKudIZCCG8lolMA7wBwF8D7Abw6bDhH2FLCV59fPE4mk+ICZd+QiHBycoKzszOMx+OishqjamFI\nG7DKF+bXyMeUoO0I6QdLEgZQ1Anh7Y60j8wkLHdglnaETE1LlYysDW9ZBUslrPuxjh2Rat/bghuT\ncAjh243nngLw1E3f23jfaJ4j1wmWOaBMwnzRjUYjHB4eLu2QoXMoU7/wroPYctXc+mvlt0oS7vf7\nODk5KdVO4OCUTlucz+dLaVyrVoyl9v1YG97qTT3lZrasgmPBTR2Yk/1Nre9tQZa1I4CyrzWdTovC\nPDLfV6Yw8fLlo6OjJRLm6ZpeW587ZMCJf2fE1LD8e4rQNlS32y2R68XFBXq9XikF8fj4uCBhGci7\nvLwspXUdHh5Ga0XwZ6cG7QlbfjBvXgCUA3P6RsYqmEk45X63CVmSsPaEObgma0BYSfzj8Ri9Xq80\n2Dj/l6PH2jtrC2IEbFkRKfeb281evla4rITZhmBispQwjyGp+HR6WsozBLlQg282bEfIfGH2u5lY\nrYUacvcYScJOxNtHliQMLE9LLy8vzRKUOon/5ORkSQmzDQGglMojH3OGVsRANRGnDD7vsmqYJmHe\n3l2ScLfbXfKQNQlzGlsOy9T5XFp2BOcJSyU8n88BLKeoSTtC+sF1qtI5NoMsSVh7wnzh8AXGh1Uh\n7fT0tBhsk8kE8/m8pII1UcnHHLEOAadGNBaYOPlnACUSliTEypDTF6UdwSQsc4WlpZVynrA8l1V2\nRGzjW52iJu0IrYQd20eWJAwsr/rR1dBitR6sR3kxMqzABCOHwRnbbUGm8OXof+sbBo8DqegGg0GJ\nhIgI3W63WDXJC3f4e2ASs9K4Uls5pj1+a085aUFYBdzrLs127AbZkbC8+GSaDed4Xmc7Gk3Ecpqa\nerJ+FfQuC6yKLALOjZBj42A4HC4RMAAcHBwsLdzh7+P4+LikiIHyXnapEnFsc4M6m3quKlCUQj/3\nBdmQMHt2/LP2tWSivRXdrhpU/H56upa7MrCKfWuFlCN0ZocmYVaC/FrOB5Y546yWZf0QK40vpSCd\n3kVDV1BbdysjKxMkZ9GRK7IhYaCcJ6xzHWWBluvUg5AXlyThVNTPuoipJW1H5LZQxfKw5eovuc2V\nJFKZosbHbDbDyclJMV50rniKSlhbEfIGa+23qC0WmZ5mqeGmbzT7iKxIGLCnoUwsVmH2ukpYvkYX\n8+bX5AbtGVrbnucKaxzw0nVdupKLvMtsmfF4XGTU6Ipq/P6p+qXyvGpPWCthhqWE3RNOA9mRMFAe\nUJwvSkSlOsDrDCw5DVulfnIaoDG11AY7gh/lOJAKWC7k4ZiBLmUJoFi6LpUwEPeEm0SVHSEDc9r7\n18pe1822Zn45jfPckSUJAy/UENDPxQIOVZDTWj1dS2EKeh1YmRFt8YQZUglPJpPiOb2smQv565t0\np9MpKWE9fU+VnFZ5wtfNjnArohlkScJyQAEoLiB5sazjcenIsRWMyRF8oVoqOFdLQs9KeCXYdDot\nEfB0Oi3qhBweHhavkzdmXsDBFcMAVHrCKYwFK/XQsiP0dkZWdkQslTOFfu4TsiZhmdXAz/OjHEh1\nBpUVtMg9RU1epLHUpRwhzwOfM2lBsAes+6zHw9HR0ZISlp+RYoB2lR1RJztCb+Flpak5docsSRhA\nYTFUWQ3WYNLFbKSCtnzAXAlYLmvVR64qWEITMWfOWCpR95kfOVDHAdiYHZEKEVdlRsiKcXqHZSsz\nompRkyvh3SJbEq5C3QEkB3XVcuVcB6TunyahNiB2bph0GLHvwAp26fdIyYqI7aysN/nUKljaNLKu\nSpX11nR/9wXZk7D28uogdtG1EVUk0yboc395eVnU0QVeKMwExAlYE5e0vVK4IdclYWnBMPSillU7\nKzsB7w6d1S/JA9cdNJb6SeWi2xTaTsRWLEDHDaqsJev7kf+XghXBsEiYawdLJaxT1GQqn97ePsUM\nkH1C9kp4XVgXXBuJCVgu9iL72Lb+sqfLJMIqUAfcrO/EUsL8nil5pevaEXrp9jp2hBPx7rBXJKyJ\nZxX5tmUgtv1mw6iyppiM5Wv40fp+pJpORSGuQ8KWEpa503pjg1RuNPuIvSJhiSpV2KaBaAWk2kzE\nQHVWDP8sla/MppCvS82OkCSsrQi9saf0hHX6HtfOqNrU1LE7tMYTXgWLhNqaMRDDPvRRQxOLPvdW\nQSOd1pVKwIqISilpq5SwvKlYSljuLpN7TnzO2BsS1qhKS2rLAKyThtVmrBOIs5RwanaEXHjDhehj\n2egMBT4AABXESURBVBGxFLWYEnY7ojnsBQnHvOAqUmrLYNyXwJyGlest1bBFwLEUtVSIOOYJx7Ij\npB0hlTCTsFbCqSj+fcNekDCjKjJeRcY5QvbTmm7vAywCZlRZEZKIUyBfbq/2hKUHLMmXg3KSfOUu\n5DpP2P3gZrGXgTkrLSkWwMqZsCy1t2/g1DWgOj2xyppq2ieV7ZcqmINxvJ0TkzC3lclV7jYudyKX\nS5fdD24OrSdhHXiLTUPbUlNBQqsnXcRmX1Clgq0xIP/PCs7tErI92orgXZWrLAi9QMMiYSfgZtF6\nEpaIBWTa6pdaN5wUlF2TqKuG+btJzY6wlLAuYalT66QStlbLSTvCsXvsxfw0FpizVHBbCJgR6+e+\nQRJMncAc/0+TwSo9i5PZEUdHR0tK2NrgVHrBWg3zirkUcqD3GXtBwsDqC69t5BSzXNqs/Otg1fnX\ngblUAlZWUE6TsPT9rZQ0zojQxXvkXoqO3WNv7IgqL7hNW/5IaE+YgzbA/lkRdW0IPQb099SEGpbn\nUeYIW54wWylyB2q9y3SsboSjGbSehC3Vt09qWG5/Iz3htvW1LlbdjKsyJFLJEWYl3Ov1lpSwJGHe\ngXo4HC4pYR2YczSHvbMjeKDKxzZ6pRbZtGnL++tAeqwxG0ITsHxsor2yTayELTtC5wczCY/HYwyH\nwyUitlbLOZpBq0lYBzYsBaT342oTOVkE3Na+1kUsM6bKF5aPTbZ5lRK2SFgqYR2Us1LUHLtHq0lY\nwrIk9mElmTXt9qBcPCZQFZTbJVHFhAOnqMmFGrpehM6M0OTrBJwW9oaEgfqFW9qCWD/besOpgxgR\nx76TpolYHrpmBNsRVfWD2ZKQ5Gtt8uloDq0n4djFFSOmtpFTLAC5b6gTjFsVlGs6M8LyhKtyhGXp\nSksFxzb4dOwWrc+OYFR5f20k4JjidyIuL+Ouq4SbgrTNZC1hKzAn7QhdP1iqYD48KJcG1lLCRPRj\nRHSpjt9Rr3kLEX2eiAZE9B4iemyzTb4Z9tGOaLvarwvLX9XZMlW+cJNWhFbCbEfU2UlDH5Yd4UTc\nHK5jR3wCwMMAHrk6voX/QERvBPAGAK8D8DIAfQDPENHRzZu6PegLrI0Dcp/JlyFVsK63UGVF8O9N\ntVe3WRd0t4q4c90I9oUt+6GN4zxHXMeOmIUQ/iTytycBPB1CeDcAENETAO4BeA2Ad16vibuDvuh8\noLYHMrilSbjKkmjqBq0Dcta2RrJ4j0XC7Atr1etV09LCdZTw1xHRHxHR7xPRLxLRVwMAET2KhTJ+\nL78whPAAwIcBvGIjrd0CrEHoA7OdiKlKrSQZKQTmrDbX2c6ISVgTsUXAPt6bxbok/CEAPwDgVQBe\nD+BRAP+OiM6wIOCAhfKVuHf1t6TR9AXn2C50UE7bEauUcAp2hCZimZ6mg3KWEra2afJx3jzWsiNC\nCM+IXz9BRB8B8AcAvg/A726yYbtEbCC2bYC2rT/rgklNbwdUt2ZEU4E5i4Bj+8lxOzUJuyecLm6U\nJxxCuA/gPwB4DMBzAAiLoJ3Ew1d/ywLWhdeGwRrzNdvQt7qwFKVFxJqM5eMuUUcJ17UjLE94n859\nyrgRCRPRLSwI+PMhhM9gQbaPi7/fAfByAB+8yefsAtagbMsg3RelXwVtRzCh1akd0hQBW+2tUsJs\nR+jsiFV2xD6NgxSxlh1BRP8IwC9jYUH8ZwD+RwBTAP/86iVvA/BmIvoUgGcBPA3gcwDetaH2bh1t\nn6pZQZm29lWjjics0fSNWReakuo9ZqlwG6uUsPvBaWHdFLWvAvBLAL4MwJ8A+HUAfzWE8AUACCG8\nlYhOAbwDwF0A7wfw6hDCZHNN3j7aMjhXEWxblb+FqsDcqsUsTS7WkO2NpahVkbD0hD0wlybWDcz9\n7RqveQrAU9dsTyOwMiPapIh1f/TF2JZ+roJeeSZVsLYjUljYUrVYQ9aM4MUmQHmhhpUnvE/nOxfs\nTe0IhrUYQ5JTWwIYmnDlBcm1A/ZtnzFNsLH6IXVKXe6irfKGoQOJcqUf8AL5EtGSArZWzDnSwd6R\nMGBPw/U24W3YBlzfXPSFqevK7kNJQ0vpxoh3VQ7xttsZW+FnFR8CbC9Yl650Ek4Pe0nCjLpKOFfo\nPnFRF62E5VR1H1ClhGMquAkiljcDbT/EfOAQwhIB79uNNje0vp5wDNqW0Co496T2VXaEVeA7x35e\nFzEC1kTctBLWdoS0IqzdQFYp4VzHc5uxtyQM2ETcNiXsdsQytBWxioibVMJVi0t0+UqrXoTbEelj\nL+0IvWLMUsJt2HXAsiOswNw+XZwxK2KVJ7zrmtOral1YdgSAWiTsSAt7ScKAXbbSUsK5kpO1cqrT\n6Sx5hftkR1gkKtVvFSHHcoi32VZrdZ/2hIEXCFhbT9Y5znU8txl7bUcA+6WELTtCJvLn3NfrYJ2g\nXJNK2PKE9c1B205aDfsuGulir0k4lhnRZk9Yq2B9ge4DtKq1vOBYAKyJPGHLE7bU8CpPuA1juo3Y\naztC/mwp4tyzI/S25/P5vNh5V+6+uy92hIUYEWt1vOugnGyTVuNVpTetGIBeMedIC3tHwpYX3DYC\nBhZ94912R6MRBoMBut0uhsMhRqNRcejgXNsR84Xlz1VHk4i1xxq/MtWyLWO6rdg7EmZYy5at2gq5\ngpP2p9MpRqMRhsMhDg4OMBgMCiJmNbyvSngV4VqE1xQRV31uVa57W6y1NmNvSZhhEbB+LkdIEh6P\nxxgMBuh0OgUh76sSZsQUsZUznIoS5jbKRwaPVa1+rcVHjrSwlyS8Sv22odwfX5DSjmAS5kMr4X0h\n4VX2g3x+11kRGlVqfB07whrXjjSwlyTMiHnDbVHC7AkPh0P0+310Op1SYE4r4Zz7e1PU9YV3jVVC\nIEbA1rHP5zdl7C0JywT3mCWRMzgzgkn44uICnU6nyIiQxz7aERZS9YTriAUPyOWLvSThVYO5TUqY\nFW+32wURLeUI6/oR+4h1g3NNoGos6vQ0J+K8sHckHMsPrrIichy40hMeDocAFmQjV83pgt859vOm\n0MSaCvnGhICVYuk2RN7YOxIG6pFvzgQMlD1hosVuC/xorazaRzvC8oHlz1aGRBOIkbH8m7XkXgbk\nXA2ni70k4Vh6j2VJ5ApWwrPZrNRfK490n2oKxM6zZUOlGKS1xqpVO9jaYTmF9juWsTckXCfAkuqF\ndx3wBWqRcGwKm2tf68IKcFlEHCPnplAlGiQJTyYTM+2wDeO5zdgLEq4zjYxdiDmD1TCAwmqwCCgF\notkVVqlg6yaVIoFZKliSsC5fuU/nODfsBQkzYmRcZUfkOnA1oTAZa687936uC32eLd9UE3AKRKwX\nZmglzCRsbVvlBJw2Wk/Csei3hbZYEYx9C7StQh3PVxNuigQmb6C6bKVlR7RBVLQZe1FPuGpZaqwc\noA/Y9kGrx1i5R53G11Qt3qo0OW6PrpTX7/eL2iBMxvvk++eIvSBhYHX+p56iOhG3C9YUXh8W+VpZ\nBruAJRakJbGKhMfjsVnM3cdyemg9Ca9agmpFnn3Athd6ee8qMm5yV4rYrI3HpsyKYBK+uLjAYDAo\nKWF9A/FxnRZaTcLrVMuq8gYd7UCVEtbKV68mbKrAkVw0IvtRRwlPJhPPjsgArSZhRmxqF1PBbkW0\nE6v84Jgd0cSKwtgMTgsFScJcLU8r4TalXbYRrSfhOtWvVqlgH7j5Q3v+OqvAqiynCXjXJBZLR9N5\nwUzAg8GgIGD2hF0Jp4/Wk7DEqtVyXvik3YiRWYyEm7Qj5OesQ8KrAnP6vR3NYy9IeFVFrFW5oo52\nQNsRelt4udosBSXMbdZtt0iYPWFpRUgl7OM5XbR+scY6sFLU+HlH3ojZEXWUcFPZEXL86WCcJOHx\neIzhcFioYamEPTMifewVCcdyg6X69dVF7YVWk3KVGU/pZWaBZUc00V5dpEcG4iTxaiW8ryVKc8Ne\nkbAmXVYVOiHftwlvHyQBs4ocjUbo9/s4Pz/H6ekpjo+P8eDBA5yfn2MwGGA8Hje22EG2lZUut/X4\n+BiHh4fodDr44he/iAcPHuDi4qJEvvtcqD837BUJA8vqQpNvbL29I39oG2I8HhcLHHq9HrrdLi4u\nLgoS5jzbJm7KvBhDkvBgMMD5+XlBwAAKEuaAnN6uysdu+tg7EgbsfFG9J5dnR7QL2oqQCxzOz8/R\n7XbR6XQKtamV8K7HgvR+x+MxDg8P0e/3i3bya6QSliQsx7MjbewdCVsBGouMXQW3D3KK3+l0ChKW\nxMakzCTc1O4UrIQ5+HZwcLBEwNPpFPfv38f9+/dNJex2RB5Ym4SJ6CsB/M8AXg3gFMB/BPCDIYSP\nite8BcBrAdwF8AEAPxRC+NRGWnxDWMEOy5aQKsIHcv7QSrjT6RR2BBPbfD7HaDTC+fl5QWhNZUZo\nT1guXZYr5c7Pz3F+fl54wjo/2Mdu+liLhImISfW9AF4F4E8BfB2AL4rXvBHAGwA8AeBZAD8O4Bki\nelEIYbKZZt8MbkfsJ0IIpe2eeBNPqTonk0lRBEcr4SY84dlshvF4DGCR3SMJWGZEcIaEtCNcCeeB\ndZXwjwD4bAjhteK5P1CveRLA0yGEdwMAET0B4B6A1wB453Ubuilcx47wgZw/5I2XiZefZwIejUaY\nTqelVLWmsiNkYE62XxLw8fFxoX75kW8croTzwbok/N0AfoWI3gng2wD8EYC3hxD+KQAQ0aMAHsFC\nKQMAQggPiOjDAF6BxEjYKujtSri9kEEqPr9MdOwPS0VsrTjbFbht8mcm4G63i8PDQ3S73WJRiTyk\npebjN32sS8JfC+CHAPwUgP8JwMsA/AwRjUMIv4AFAQcslK/Evau/JYGYCrbsCEc7oBfkEFHhDcuD\nCU+OD/5/+biL9rKdMJ/PS21kf7jT6ZTGshYPPobzwLok3AHwkRDCj179/nEiejGA1wP4hY22bIPQ\nizR0rqgVUXYron3g88meMCvjWKnIJpf6yjYQUWGjrKqDLdvqYzcPrFvA548BfFI990kAf+7q5+cA\nEICH1WsevvrbziEvJGsV0qqCJ/I9HO2APq+pE1msnVWplCm021EP65LwBwB8vXru63EVnAshfAYL\nsn2c/0hEdwC8HMAHr9/M68EavDLizEtBdcETV8Lth0XEKRa6qbpRWETsYzY/rGtH/C8APkBEb8Ii\nyPZyLPKB/1vxmrcBeDMRfQqLFLWnAXwOwLtu3NprQA5WntbJ2gFW1SkfzPuBqvOb0rnntuisjtjr\nHHlhLRIOIfwGEX0vgJ8E8KMAPgPgyRDCPxeveSsRnQJ4BxaLNd4P4NWhoRxhrRC0HdHtdqMkzP/v\naDdyOce5tNOxHtZeMRdC+NcA/vWK1zwF4KnrNWnz0EE5uR7/4ODAlbDD4WgMra8dof0zmRVxcHAA\nACYJ6/93OByObaDVJMzpPUzAnU6nVMCFl4HySiNXwg6HY9doNQkDZR9N2hH8N2uFlJOww+HYFVpP\nwkCZiJmEJSHz0mW9WEP/r8PhcGwarSdhaUkAL6hhhl726QTscDh2idaTsIQM0PHvul6AE7DD4dgl\n9oKEdbK7JGG56sgrpzkcjl1jL0iYoZeAcnaEB+IcDkdT2CsSBnZfktDhcDiqsG4BH4fD4XBsECmQ\ncK/pBjgcDseWsJLfUiDhr2m6AQ6Hw7ElfM2qF1DT3igRfRkWOzc/C2DUaGMcDodjM+hhQcDPhBC+\nUPXCxknY4XA49hkp2BEOh8Oxt3ASdjgcjgbhJOxwOBwNwknY4XA4GkQyJExEf4+IPkNEQyL6EBF9\nc9NtioGIXklE/4qI/oiILonoe4zXvIWIPk9EAyJ6DxE91kRbLRDRm4joI0T0gIjuEdG/IKK/YLwu\nyT4Q0euJ6ONEdP/q+CAR/XX1miTbboGIfuRqHP20ej7ZPhDRj121WR6/o16TbPsBgIi+koh+gYj+\n9KqNHyeib1Sv2XofkiBhIvpbAH4KwI8B+CsAPg7gGSL68kYbFscZgN8C8MMAltJLiOiNAN4A4HUA\nXgagj0V/jnbZyAq8EsA/xmK37O8AcAjgV4nohF+QeB/+EMAbAXwjgJcCeB+AdxHRi4Dk217Cldh4\nHRZjXj6fQx8+AeBhAI9cHd/Cf0i9/UR0F8AHAIyxSJF9EYD/HsAXxWt20we9G3ETB4APAfhfxe8E\n4HMA/kHTbavR9ksA36Oe+zyAvy9+vwNgCOD7mm5vpA9fftWPb8m4D18A8IM5tR3ALQC/B+DbAfwb\nAD+dy/ePhWD6aMXfU2//TwL4tytes5M+NK6EiegQCzXzXn4uLHr8awBe0VS7rgsiehQLVSD78wDA\nh5Fuf+5ioej/DMirD0TUIaLvB3AK4IM5tR3AzwL45RDC++STGfXh664sud8nol8koq8Gsmn/dwP4\nDSJ655Ul91Eiei3/cZd9aJyEsVBhBwDuqefvYfEl5IZHsCC0LPpDRATgbQB+PYTAnl7yfSCiFxPR\nORbTybcD+N4Qwu8hg7YDwNWN4xsAvMn4cw59+BCAH8BiKv96AI8C+HdEdIY82v+1AH4Ii5nIdwL4\n3wD8DBH9N1d/31kf9q6UpWMJbwfwlwD8taYbsiZ+F8BLADwE4G8C+Hki+tZmm1QPRPRVWNz4viOE\nMG26PddBCOEZ8esniOgjAP4AwPdhcW5SRwfAR0IIP3r1+8eJ6MVY3FB+YdcNaRp/CmCOhcEv8TCA\n53bfnBvjOSw87eT7Q0T/BMB3AfgvQwh/LP6UfB9CCLMQwqdDCB8LIfwPWAS2nkQGbcfCfvsKAB8l\noikRTQF8G4AniWiChdpKvQ8lhBDuA/gPAB5DHufgjwF8Uj33SQB/7urnnfWhcRK+UgK/CeBxfu5q\nivw4gA821a7rIoTwGSxOkuzPHSwyEZLpzxUB/w0A/1UI4bPyb7n0QaED4DiTtv8agL+MhR3xkqvj\nNwD8IoCXhBA+jfT7UAIR3cKCgD+fyTn4AICvV899PRZqfrfXQNNRyquo4/cBGAB4AsBfBPAOLKLd\nX9F02yLtPcPiwvkGLLIK/rur37/66u//4Kr9343FxfYvAfxHAEdNt/2qfW/HIhXnlVjc2fnoidck\n2wcAP3HV9v8cwIsB/EMAMwDfnnrbK/qksyOS7gOAfwTgW6/OwX8B4D1YKPgvy6T934RFPOFNAP48\ngP8awDmA79/1OWj8yxAd/mEsylkOAfx7AN/UdJsq2vptV+Q7V8f/Ll7zFBYpLgMAzwB4rOl2i7ZZ\nbZ8DeEK9Lsk+APinAD59NVaeA/CrTMCpt72iT++TJJx6HwD8n1ikkQ4BfBbALwF4NJf2X7XvuwD8\n9lX7/l8Af9d4zdb74KUsHQ6Ho0E07gk7HA7HPsNJ2OFwOBqEk7DD4XA0CCdhh8PhaBBOwg6Hw9Eg\nnIQdDoejQTgJOxwOR4NwEnY4HI4G4STscDgcDcJJ2OFwOBqEk7DD4XA0CCdhh8PhaBD/P3f0nZlw\n5/ZOAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11abf8d10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#checking a sample\n",
    "plt.figure()\n",
    "plt.imshow(X_synth_train[232], cmap='gray')\n",
    "\n",
    "y_synth_train[232]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Looks like things work as we expect them to. Let's prepare the datset and labels so that keras can handle them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Preparatory Preprocessing\n",
    "\n",
    "#### Preprocessing Labels for model\n",
    "\n",
    "The labels are going to be encoded to \"One Hot\" arrays, to make them compatible with Keras. Note that, as the our Deep Learning model will have 5 classifiers, we'll need 5 such One Hot arrays, one for each digit position in the image. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Converting labels to One-hot representations of shape (set_size,digits,classes)\n",
    "possible_classes = 11\n",
    "\n",
    "def convert_labels(labels):\n",
    "    \n",
    "    #As per Keras conventions, the multiple labels need to be of the form [array_digit1,...5]\n",
    "    #Each digit array will be of shape (60000,11)\n",
    "    \n",
    "    #Code below could be better, but cba for now. \n",
    "    \n",
    "    #Declare output ndarrays\n",
    "    dig0_arr = np.ndarray(shape=(len(labels),possible_classes))\n",
    "    dig1_arr = np.ndarray(shape=(len(labels),possible_classes))\n",
    "    dig2_arr = np.ndarray(shape=(len(labels),possible_classes))\n",
    "    dig3_arr = np.ndarray(shape=(len(labels),possible_classes)) #5 for digits, 11 for possible classes  \n",
    "    dig4_arr = np.ndarray(shape=(len(labels),possible_classes))\n",
    "    \n",
    "    for index,label in enumerate(labels):\n",
    "        \n",
    "        #Using np_utils from keras to OHE the labels in the image\n",
    "        dig0_arr[index,:] = np_utils.to_categorical(label[0],possible_classes)\n",
    "        dig1_arr[index,:] = np_utils.to_categorical(label[1],possible_classes)\n",
    "        dig2_arr[index,:] = np_utils.to_categorical(label[2],possible_classes)\n",
    "        dig3_arr[index,:] = np_utils.to_categorical(label[3],possible_classes)\n",
    "        dig4_arr[index,:] = np_utils.to_categorical(label[4],possible_classes)\n",
    "        \n",
    "    return [dig0_arr,dig1_arr,dig2_arr,dig3_arr,dig4_arr]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_labels = convert_labels(y_synth_train)\n",
    "test_labels = convert_labels(y_synth_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 11)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Checking the shape of the OHE array for the first digit position\n",
    "np.shape(train_labels[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.,  0.]])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np_utils.to_categorical(y_synth_train[234][0],11)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Preprocessing Images for model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The function below will pre-process the images so that they can be handled by keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def prep_data_keras(img_data):\n",
    "    \n",
    "    #Reshaping data for keras, with tensorflow as backend\n",
    "    img_data = img_data.reshape(len(img_data),64,64,1)\n",
    "    \n",
    "    #Converting everything to floats\n",
    "    img_data = img_data.astype('float32')\n",
    "    \n",
    "    #Normalizing values between 0 and 1\n",
    "    img_data /= 255\n",
    "    \n",
    "    return img_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "train_images = prep_data_keras(X_synth_train)\n",
    "test_images = prep_data_keras(X_synth_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 64, 64, 1)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(train_images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 64, 64, 1)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(test_images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Building"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "#Importing relevant keras modules\n",
    "from keras.models import Sequential, Model\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten, Input\n",
    "from keras.layers import Convolution2D, MaxPooling2D"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're going to use a Convolutional Neural Network for our network. \n",
    "\n",
    "Starting with a 2D Convolutional layer, we'll use ReLU activations after every Convolutional Layer. \n",
    "\n",
    "After the second CovLayer + ReLU, we'll add 2DMaxPooling, and a dropout to make the model robust to overfitting. A flattening layer will be added to make the data ready for classification layers, which were in the form of Dense Layers, of the same size as the no. of classes (11 for us), activated using softmax to give us the probability of each class."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/12\n",
      "60000/60000 [==============================] - 944s - loss: 3.7553 - dense_4_loss: 0.9496 - dense_5_loss: 0.9061 - dense_6_loss: 0.7953 - dense_7_loss: 0.6590 - dense_8_loss: 0.4454 - dense_4_acc: 0.6825 - dense_5_acc: 0.7028 - dense_6_acc: 0.7378 - dense_7_acc: 0.7884 - dense_8_acc: 0.8626 - val_loss: 1.0372 - val_dense_4_loss: 0.2358 - val_dense_5_loss: 0.2332 - val_dense_6_loss: 0.2181 - val_dense_7_loss: 0.1906 - val_dense_8_loss: 0.1596 - val_dense_4_acc: 0.9471 - val_dense_5_acc: 0.9511 - val_dense_6_acc: 0.9551 - val_dense_7_acc: 0.9600 - val_dense_8_acc: 0.9570\n",
      "Epoch 2/12\n",
      "60000/60000 [==============================] - 923s - loss: 2.3002 - dense_4_loss: 0.5586 - dense_5_loss: 0.5510 - dense_6_loss: 0.4836 - dense_7_loss: 0.4172 - dense_8_loss: 0.2899 - dense_4_acc: 0.8151 - dense_5_acc: 0.8167 - dense_6_acc: 0.8378 - dense_7_acc: 0.8628 - dense_8_acc: 0.9041 - val_loss: 0.6792 - val_dense_4_loss: 0.1599 - val_dense_5_loss: 0.1576 - val_dense_6_loss: 0.1411 - val_dense_7_loss: 0.1201 - val_dense_8_loss: 0.1005 - val_dense_4_acc: 0.9607 - val_dense_5_acc: 0.9678 - val_dense_6_acc: 0.9688 - val_dense_7_acc: 0.9757 - val_dense_8_acc: 0.9795\n",
      "Epoch 3/12\n",
      "60000/60000 [==============================] - 938s - loss: 1.9150 - dense_4_loss: 0.4602 - dense_5_loss: 0.4536 - dense_6_loss: 0.3994 - dense_7_loss: 0.3588 - dense_8_loss: 0.2430 - dense_4_acc: 0.8455 - dense_5_acc: 0.8470 - dense_6_acc: 0.8647 - dense_7_acc: 0.8813 - dense_8_acc: 0.9190 - val_loss: 0.5122 - val_dense_4_loss: 0.1225 - val_dense_5_loss: 0.1239 - val_dense_6_loss: 0.1018 - val_dense_7_loss: 0.0884 - val_dense_8_loss: 0.0756 - val_dense_4_acc: 0.9662 - val_dense_5_acc: 0.9706 - val_dense_6_acc: 0.9741 - val_dense_7_acc: 0.9801 - val_dense_8_acc: 0.9837\n",
      "Epoch 4/12\n",
      "60000/60000 [==============================] - 921s - loss: 1.7039 - dense_4_loss: 0.4060 - dense_5_loss: 0.3957 - dense_6_loss: 0.3528 - dense_7_loss: 0.3262 - dense_8_loss: 0.2231 - dense_4_acc: 0.8629 - dense_5_acc: 0.8660 - dense_6_acc: 0.8784 - dense_7_acc: 0.8904 - dense_8_acc: 0.9252 - val_loss: 0.4468 - val_dense_4_loss: 0.1054 - val_dense_5_loss: 0.1065 - val_dense_6_loss: 0.0908 - val_dense_7_loss: 0.0748 - val_dense_8_loss: 0.0694 - val_dense_4_acc: 0.9711 - val_dense_5_acc: 0.9726 - val_dense_6_acc: 0.9750 - val_dense_7_acc: 0.9833 - val_dense_8_acc: 0.9847\n",
      "Epoch 5/12\n",
      "60000/60000 [==============================] - 878s - loss: 1.5396 - dense_4_loss: 0.3605 - dense_5_loss: 0.3564 - dense_6_loss: 0.3216 - dense_7_loss: 0.2945 - dense_8_loss: 0.2066 - dense_4_acc: 0.8785 - dense_5_acc: 0.8788 - dense_6_acc: 0.8881 - dense_7_acc: 0.9003 - dense_8_acc: 0.9300 - val_loss: 0.4171 - val_dense_4_loss: 0.1031 - val_dense_5_loss: 0.0926 - val_dense_6_loss: 0.0790 - val_dense_7_loss: 0.0781 - val_dense_8_loss: 0.0643 - val_dense_4_acc: 0.9709 - val_dense_5_acc: 0.9744 - val_dense_6_acc: 0.9777 - val_dense_7_acc: 0.9808 - val_dense_8_acc: 0.9862\n",
      "Epoch 6/12\n",
      "60000/60000 [==============================] - 1074s - loss: 1.4021 - dense_4_loss: 0.3299 - dense_5_loss: 0.3179 - dense_6_loss: 0.2909 - dense_7_loss: 0.2743 - dense_8_loss: 0.1890 - dense_4_acc: 0.8875 - dense_5_acc: 0.8906 - dense_6_acc: 0.8978 - dense_7_acc: 0.9045 - dense_8_acc: 0.9349 - val_loss: 0.3621 - val_dense_4_loss: 0.0907 - val_dense_5_loss: 0.0813 - val_dense_6_loss: 0.0740 - val_dense_7_loss: 0.0582 - val_dense_8_loss: 0.0579 - val_dense_4_acc: 0.9746 - val_dense_5_acc: 0.9767 - val_dense_6_acc: 0.9779 - val_dense_7_acc: 0.9861 - val_dense_8_acc: 0.9876\n",
      "Epoch 7/12\n",
      "60000/60000 [==============================] - 1159s - loss: 1.2786 - dense_4_loss: 0.2945 - dense_5_loss: 0.2886 - dense_6_loss: 0.2667 - dense_7_loss: 0.2509 - dense_8_loss: 0.1779 - dense_4_acc: 0.8977 - dense_5_acc: 0.8999 - dense_6_acc: 0.9058 - dense_7_acc: 0.9132 - dense_8_acc: 0.9390 - val_loss: 0.3485 - val_dense_4_loss: 0.0841 - val_dense_5_loss: 0.0813 - val_dense_6_loss: 0.0674 - val_dense_7_loss: 0.0580 - val_dense_8_loss: 0.0577 - val_dense_4_acc: 0.9756 - val_dense_5_acc: 0.9787 - val_dense_6_acc: 0.9792 - val_dense_7_acc: 0.9866 - val_dense_8_acc: 0.9875\n",
      "Epoch 8/12\n",
      "60000/60000 [==============================] - 1161s - loss: 1.1724 - dense_4_loss: 0.2698 - dense_5_loss: 0.2653 - dense_6_loss: 0.2438 - dense_7_loss: 0.2342 - dense_8_loss: 0.1593 - dense_4_acc: 0.9058 - dense_5_acc: 0.9070 - dense_6_acc: 0.9126 - dense_7_acc: 0.9178 - dense_8_acc: 0.9451 - val_loss: 0.3301 - val_dense_4_loss: 0.0805 - val_dense_5_loss: 0.0803 - val_dense_6_loss: 0.0614 - val_dense_7_loss: 0.0528 - val_dense_8_loss: 0.0551 - val_dense_4_acc: 0.9777 - val_dense_5_acc: 0.9769 - val_dense_6_acc: 0.9814 - val_dense_7_acc: 0.9850 - val_dense_8_acc: 0.9865\n",
      "Epoch 9/12\n",
      "60000/60000 [==============================] - 1041s - loss: 1.0928 - dense_4_loss: 0.2506 - dense_5_loss: 0.2463 - dense_6_loss: 0.2316 - dense_7_loss: 0.2135 - dense_8_loss: 0.1507 - dense_4_acc: 0.9108 - dense_5_acc: 0.9126 - dense_6_acc: 0.9161 - dense_7_acc: 0.9248 - dense_8_acc: 0.9468 - val_loss: 0.3028 - val_dense_4_loss: 0.0738 - val_dense_5_loss: 0.0736 - val_dense_6_loss: 0.0570 - val_dense_7_loss: 0.0475 - val_dense_8_loss: 0.0508 - val_dense_4_acc: 0.9786 - val_dense_5_acc: 0.9802 - val_dense_6_acc: 0.9830 - val_dense_7_acc: 0.9876 - val_dense_8_acc: 0.9885\n",
      "Epoch 10/12\n",
      "60000/60000 [==============================] - 939s - loss: 1.0137 - dense_4_loss: 0.2337 - dense_5_loss: 0.2298 - dense_6_loss: 0.2112 - dense_7_loss: 0.1995 - dense_8_loss: 0.1395 - dense_4_acc: 0.9173 - dense_5_acc: 0.9183 - dense_6_acc: 0.9221 - dense_7_acc: 0.9289 - dense_8_acc: 0.9507 - val_loss: 0.3050 - val_dense_4_loss: 0.0747 - val_dense_5_loss: 0.0751 - val_dense_6_loss: 0.0581 - val_dense_7_loss: 0.0471 - val_dense_8_loss: 0.0500 - val_dense_4_acc: 0.9778 - val_dense_5_acc: 0.9794 - val_dense_6_acc: 0.9830 - val_dense_7_acc: 0.9875 - val_dense_8_acc: 0.9880\n",
      "Epoch 11/12\n",
      "60000/60000 [==============================] - 989s - loss: 0.9621 - dense_4_loss: 0.2213 - dense_5_loss: 0.2163 - dense_6_loss: 0.2024 - dense_7_loss: 0.1902 - dense_8_loss: 0.1319 - dense_4_acc: 0.9194 - dense_5_acc: 0.9226 - dense_6_acc: 0.9249 - dense_7_acc: 0.9315 - dense_8_acc: 0.9522 - val_loss: 0.2832 - val_dense_4_loss: 0.0675 - val_dense_5_loss: 0.0698 - val_dense_6_loss: 0.0516 - val_dense_7_loss: 0.0445 - val_dense_8_loss: 0.0498 - val_dense_4_acc: 0.9801 - val_dense_5_acc: 0.9806 - val_dense_6_acc: 0.9847 - val_dense_7_acc: 0.9871 - val_dense_8_acc: 0.9885\n",
      "Epoch 12/12\n",
      "60000/60000 [==============================] - 1003s - loss: 0.9082 - dense_4_loss: 0.2061 - dense_5_loss: 0.2059 - dense_6_loss: 0.1902 - dense_7_loss: 0.1798 - dense_8_loss: 0.1262 - dense_4_acc: 0.9246 - dense_5_acc: 0.9250 - dense_6_acc: 0.9273 - dense_7_acc: 0.9345 - dense_8_acc: 0.9545 - val_loss: 0.2760 - val_dense_4_loss: 0.0670 - val_dense_5_loss: 0.0719 - val_dense_6_loss: 0.0525 - val_dense_7_loss: 0.0404 - val_dense_8_loss: 0.0442 - val_dense_4_acc: 0.9810 - val_dense_5_acc: 0.9812 - val_dense_6_acc: 0.9846 - val_dense_7_acc: 0.9888 - val_dense_8_acc: 0.9901\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.History at 0x1228eaa90>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Building the model\n",
    "\n",
    "batch_size = 128\n",
    "nb_classes = 11\n",
    "nb_epoch = 12\n",
    "\n",
    "#image input dimensions\n",
    "img_rows = 64\n",
    "img_cols = 64\n",
    "img_channels = 1\n",
    "\n",
    "#number of convulation filters to use\n",
    "nb_filters = 32\n",
    "# size of pooling area for max pooling\n",
    "pool_size = (2, 2)\n",
    "# convolution kernel size\n",
    "kernel_size = (3, 3)\n",
    "\n",
    "#defining the input\n",
    "inputs = Input(shape=(img_rows,img_cols,img_channels))\n",
    "\n",
    "#Model taken from keras example. Worked well for a digit, dunno for multiple\n",
    "cov = Convolution2D(nb_filters,kernel_size[0],kernel_size[1],border_mode='same')(inputs)\n",
    "cov = Activation('relu')(cov)\n",
    "cov = Convolution2D(nb_filters,kernel_size[0],kernel_size[1])(cov)\n",
    "cov = Activation('relu')(cov)\n",
    "cov = MaxPooling2D(pool_size=pool_size)(cov)\n",
    "cov = Dropout(0.25)(cov)\n",
    "cov_out = Flatten()(cov)\n",
    "\n",
    "\n",
    "#Dense Layers\n",
    "cov2 = Dense(128, activation='relu')(cov_out)\n",
    "cov2 = Dropout(0.5)(cov2)\n",
    "\n",
    "\n",
    "\n",
    "#Prediction layers\n",
    "c0 = Dense(nb_classes, activation='softmax')(cov2)\n",
    "c1 = Dense(nb_classes, activation='softmax')(cov2)\n",
    "c2 = Dense(nb_classes, activation='softmax')(cov2)\n",
    "c3 = Dense(nb_classes, activation='softmax')(cov2)\n",
    "c4 = Dense(nb_classes, activation='softmax')(cov2)\n",
    "\n",
    "#Defining the model\n",
    "model = Model(input=inputs,output=[c0,c1,c2,c3,c4])\n",
    "\n",
    "#Compiling the model\n",
    "model.compile(loss='categorical_crossentropy',optimizer='adam',metrics=['accuracy'])\n",
    "\n",
    "#Fitting the model\n",
    "model.fit(train_images,train_labels,batch_size=batch_size,nb_epoch=nb_epoch,verbose=1,\n",
    "          validation_data=(test_images, test_labels))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "predictions = model.predict(test_images)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 10000, 11)"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(predictions)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10000"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(predictions[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5, 10000, 11)"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.shape(test_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll define a custom to calculate accuracy for predicting individual digits, as well as for predicting complete sequence of images."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def calculate_acc(predictions,real_labels):\n",
    "    \n",
    "    individual_counter = 0\n",
    "    global_sequence_counter = 0\n",
    "    for i in range(0,len(predictions[0])):\n",
    "        #Reset sequence counter at the start of each image\n",
    "        sequence_counter = 0 \n",
    "        \n",
    "        for j in range(0,5):\n",
    "            if np.argmax(predictions[j][i]) == np.argmax(real_labels[j][i]):\n",
    "                individual_counter += 1\n",
    "                sequence_counter +=1\n",
    "        \n",
    "        if sequence_counter == 5:\n",
    "            global_sequence_counter += 1\n",
    "         \n",
    "    ind_accuracy = individual_counter/50000.0\n",
    "    global_accuracy = global_sequence_counter/10000.0\n",
    "    \n",
    "    return ind_accuracy,global_accuracy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "ind_acc,glob_acc = calculate_acc(predictions,test_labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The individual accuracy is 98.514 %\n",
      "The sequence prediction accuracy is 92.86 %\n"
     ]
    }
   ],
   "source": [
    "print(\"The individual accuracy is {} %\".format(ind_acc*100))\n",
    "print(\"The sequence prediction accuracy is {} %\".format(glob_acc*100))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Actual labels: [0, 9, 7, 10, 10]\n",
      "Predicted labels: [0, 8, 7, 10, 10]\n",
      "\n",
      "Actual labels: [0, 5, 7, 10, 10]\n",
      "Predicted labels: [0, 5, 7, 10, 10]\n",
      "\n",
      "Actual labels: [3, 8, 1, 10, 10]\n",
      "Predicted labels: [3, 8, 1, 10, 10]\n",
      "\n",
      "Actual labels: [6, 8, 1, 10, 10]\n",
      "Predicted labels: [6, 8, 1, 10, 10]\n",
      "\n",
      "Actual labels: [7, 6, 3, 2, 9]\n",
      "Predicted labels: [7, 6, 3, 2, 9]\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#Printing some examples of real and predicted labels\n",
    "for i in random.sample(range(0,10000),5):\n",
    "    \n",
    "    actual_labels = []\n",
    "    predicted_labels = []\n",
    "    \n",
    "    for j in range(0,5):\n",
    "        actual_labels.append(np.argmax(test_labels[j][i]))\n",
    "        predicted_labels.append(np.argmax(predictions[j][i]))\n",
    "        \n",
    "    print(\"Actual labels: {}\".format(actual_labels))\n",
    "    print(\"Predicted labels: {}\\n\".format(predicted_labels))\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that model achieved good accuracy, with around 98.5% accurate for identifying individual digits or blanks, and around 92.8% for identifying whole sequences."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda env:py27]",
   "language": "python",
   "name": "conda-env-py27-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}