tensorflow/linear_regression/02_Inspecting_the_graph.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Note on running the notebook (solved)\n",
    "\n",
    "Since the graph is written in the default graph, it is better to reset it using:\n",
    "\n",
    "```\n",
    "tf.reset_default_graph()\n",
    "```\n",
    "\n",
    "~~What is currently a bit strange, is that this code only works once. If you want to let this notebook run for the second time, you seem to have to restart the notebook (Kernel->restart). This is not the case if you put all the code into a python file. ~~\n",
    "\n",
    "\n",
    "### Looking in the detail of how TensorFlow opimizes the problem\n",
    "The computation of residual sum of error of the linear regression\n",
    "$$\n",
    "    y = a x + b \n",
    "$$\n",
    "\n",
    "is encoded in the follwong TensorFlow graph, which reads in x_data and y_data and does some processing. The nodes of the graph, are operations called *ops*. For the documemation: \"An op\n",
    "takes zero or more `Tensors`, performs some computation, and produces zero or\n",
    "more `Tensors`.  A `Tensor` is a typed multi-dimensional array.\" In our example, the Tensor `x_data` is a simple vector of size N.\n",
    "\n",
    "\n",
    "In this notebook we investigate, who to write such a graph and also include additional information, while the graph is processed.\n",
    "![graph.png, width=140](graph.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Creating the data\n",
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "N = 30\n",
    "x_vals = (np.linspace(0,10,N)).astype('float32')\n",
    "y_vals = (2.42 * x_vals + 0.42 + np.random.normal(0,1,N)).astype('float32')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 992 16.5880947113\n",
      "epoch 993 16.5874900818\n",
      "epoch 994 16.5868778229\n",
      "epoch 995 16.5862636566\n",
      "epoch 996 16.5856647491\n",
      "epoch 998 16.5844516754\n",
      "[16.583851, 2.3717301, 0.80445063]\n",
      "Finished all\n"
     ]
    }
   ],
   "source": [
    "tf.reset_default_graph()\n",
    "a = tf.Variable(1.0, name = 'a') #Note that 1.0 is needed\n",
    "b = tf.Variable(0.01, name = 'b')\n",
    "x = tf.placeholder('float32', [N], name='x_data')\n",
    "y = tf.placeholder('float32', [N], name='y_data')\n",
    "\n",
    "\n",
    "resi = a*x + b - y\n",
    "loss = tf.reduce_sum(tf.square(resi), name='loss')\n",
    "init_op = tf.initialize_all_variables() #Initialization op \n",
    "\n",
    "train_op = tf.train.GradientDescentOptimizer(0.0001).minimize(loss)\n",
    "epochs = 1000\n",
    "\n",
    "loss_summary = tf.scalar_summary(\"loss_summary\", loss) #<--- Definition of ops to be stored\n",
    "resi_summart = tf.histogram_summary(\"resi_summart\", resi)\n",
    "merged_summary_op = tf.merge_all_summaries()        #<-----  Combine all ops to be stored\n",
    "sess = tf.Session()\n",
    "sess.run(init_op)\n",
    "writer = tf.train.SummaryWriter(\"/tmp/dumm/run1\", tf.get_default_graph(), 'graph.pbtxt') #<--- Where to store\n",
    "for e in range(epochs): #Fitting the data for 10 epochs\n",
    "    sess.run(train_op, feed_dict={x:x_vals, y:y_vals})\n",
    "    if (e < 5 | e > epochs - 5):\n",
    "        print(\"epoch {} {}\".format(e, sess.run(loss, feed_dict={x:x_vals, y:y_vals})))\n",
    "    sum_str = sess.run(merged_summary_op, feed_dict={x:x_vals, y:y_vals}) #<--- Running the graph to produce output\n",
    "    writer.add_summary(sum_str, e) #<--- writing out the output\n",
    "res = sess.run([loss, a, b], feed_dict={x:x_vals, y:y_vals})\n",
    "print(res)\n",
    "print('Finished all')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The graph is created and can be visualized using:\n",
    "```\n",
    "    tensorboard --logdir=/tmp/dumm\n",
    "```\n",
    "and explored using the browser with the URL: http://0.0.0.0:6006, you will find something like:\n",
    "![graph.png](tf_screenshot.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Getting hold of tensors\n",
    "\n",
    "Suppose you have not access to the graph construction but would like to feed and fetch the graph later for e.g. for debugging. You could do to following top get all necessary tensors for debugging"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "16.5839\n"
     ]
    }
   ],
   "source": [
    "loss_t = tf.Graph.get_tensor_by_name(tf.get_default_graph(), 'loss:0')\n",
    "x_1 = tf.Graph.get_tensor_by_name(tf.get_default_graph(), 'x_data:0')\n",
    "y_1 = tf.Graph.get_tensor_by_name(tf.get_default_graph(), 'y_data:0')\n",
    "loss_val = sess.run(loss_t, feed_dict={x_1:x_vals, y_1:y_vals})\n",
    "print(loss_val)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sess.close()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}