deep_learning/image-processing-keras-datacamp.ipynb

{
 "metadata": {
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": 3
  },
  "orig_nbformat": 2
 },
 "nbformat": 4,
 "nbformat_minor": 2,
 "cells": [
  {
   "source": [
    "# Image Processing with Keras in Python"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "source": [
    "### Images as data: visualizations\n",
    "To display image data, you will rely on Python's Matplotlib library, and specifically use matplotlib's pyplot sub-module, that contains many plotting commands. Some of these commands allow you to display the content of images stored in arrays."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load the image\n",
    "data = plt.imread('bricks.png')\n",
    "\n",
    "# Display the image\n",
    "plt.imshow(data)\n",
    "plt.show()"
   ]
  },
  {
   "source": [
    "### Images as data: changing images\n",
    "To modify an image, you can modify the existing numbers in the array. In a color image, you can change the values in one of the color channels without affecting the other colors, by indexing on the last dimension of the array.\n",
    "\n",
    "The image you imported in the previous exercise is available in data."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the red channel in this part of the image to 1\n",
    "data[:10, :10, 0] = 1\n",
    "\n",
    "# Set the green channel in this part of the image to 0\n",
    "data[:10, :10, 1] = 0\n",
    "\n",
    "# Set the blue channel in this part of the image to 0\n",
    "data[:10, :10, 2] = 0\n",
    "\n",
    "# Visualize the result\n",
    "plt.imshow(data)\n",
    "plt.show()"
   ]
  },
  {
   "source": [
    "Using one-hot encoding to represent images\n",
    "Neural networks expect the labels of classes in a dataset to be organized in a one-hot encoded manner: each row in the array contains zeros in all columns, except the column corresponding to a unique label, which is set to 1.\n",
    "\n",
    "The fashion dataset contains three categories:\n",
    "- Shirts\n",
    "- Dresses\n",
    "- Shoes\n",
    "\n",
    "In this exercise, you will create a one-hot encoding of a small sample of these labels."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The number of image categories\n",
    "n_categories = 3\n",
    "\n",
    "# The unique values of categories in the data\n",
    "categories = np.array([\"shirt\", \"dress\", \"shoe\"])\n",
    "\n",
    "# Initialize ohe_labels as all zeros\n",
    "ohe_labels = np.zeros((len(labels), n_categories))\n",
    "\n",
    "# Loop over the labels\n",
    "for ii in range(len(labels)):\n",
    "    # Find the location of this label in the categories variable\n",
    "    jj = np.where(categories == labels[ii])\n",
    "    # Set the corresponding zero to one\n",
    "    ohe_labels[ii, jj] = 1"
   ]
  },
  {
   "source": [
    "### Evaluating a classifier\n",
    "To evaluate a classifier, we need to test it on images that were not used during training. This is called \"cross-validation\": a prediction of the class (e.g., t-shirt, dress or shoe) is made from each of the test images, and these predictions are compared with the true labels of these images.\n",
    "\n",
    "The results of cross-validation are provided as one-hot encoded arrays: test_labels and predictions.\n",
    "\n",
    "- Multiply the arrays with each other and sum the result to find the total number of correct predictions.\n",
    "- Divide the number of correct answers (the sum) by the length of predictions array to calculate the proportion of correct predictions."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calculate the number of correct predictions\n",
    "number_correct = (test_labels * predictions).sum()\n",
    "print(number_correct)\n",
    "\n",
    "# Calculate the proportion of correct predictions\n",
    "proportion_correct = number_correct / len(predictions)\n",
    "print(proportion_correct)"
   ]
  },
  {
   "source": [
    "### Build a neural network\n",
    "We will use the Keras library to create neural networks and to train these neural networks to classify images. These models will all be of the Sequential type, meaning that the outputs of one layer are provided as inputs only to the next layer.\n",
    "\n",
    "In this exercise, you will create a neural network with Dense layers, meaning that each unit in each layer is connected to all of the units in the previous layer. For example, each unit in the first layer is connected to all of the pixels in the input images. The Dense layer object receives as arguments the number of units in that layer, and the activation function for the units. For the first layer in the network, it also receives an input_shape keyword argument.\n",
    "\n",
    "### Compile a neural network\n",
    "Once you have constructed a model in Keras, the model needs to be compiled before you can fit it to data. This means that you need to specify the optimizer that will be used to fit the model and the loss function that will be used in optimization. Optionally, you can also specify a list of metrics that the model will keep track of. For example, if you want to know the classification accuracy, you will provide the list ['accuracy'] to the metrics keyword argument.\n",
    "\n",
    "### Fitting a neural network model to clothing data\n",
    "In this exercise, you will fit the fully connected neural network that you constructed in the previous exercise to image data. The training data is provided as two variables: train_data that contains the pixel data for 50 images of the three clothing classes and train_labels, which contains one-hot encoded representations of the labels for each one of these 50 images. Transform the data into the network's expected input and then fit the model on training data and training labels.\n",
    "\n",
    "The model you compiled in the previous exercise, and train_data and train_labels are available in your workspace.\n",
    "\n",
    "### Cross-validation for neural network evaluation\n",
    "To evaluate the model, we use a separate test data-set. As in the train data, the images in the test data also need to be reshaped before they can be provided to the fully-connected network because the network expects one column per pixel in the input.\n",
    "\n",
    "The model you fit in the previous exercise, and test_data and test_labels are available in your workspace."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Imports components from Keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense\n",
    "\n",
    "# Initializes a sequential model\n",
    "model = Sequential()\n",
    "\n",
    "# First layer\n",
    "model.add(Dense(10, activation='relu', input_shape=(784,)))\n",
    "\n",
    "# Second layer\n",
    "model.add(Dense(10, activation='relu'))\n",
    "\n",
    "# Output layer\n",
    "model.add(Dense(3, activation='softmax'))\n",
    "\n",
    "# Compile the model\n",
    "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n",
    "\n",
    "# The image data are provided in an array with shape (50, 28, 28, 1), but the network expects \n",
    "# data as a two-dimensional array with one row per sample, and one column per pixel.\n",
    "# Reshape the data to two-dimensional array\n",
    "train_data = train_data.reshape(50, 784)\n",
    "\n",
    "# Fit the model\n",
    "model.fit(train_data, train_labels, validation_split=0.2, epochs=3)\n",
    "\n",
    "# Reshape test data\n",
    "test_data = test_data.reshape(10, 784)\n",
    "\n",
    "# Evaluate the model\n",
    "model.evaluate(test_data, test_labels)"
   ]
  },
  {
   "source": [
    "### One dimensional convolutions\n",
    "A convolution of an one-dimensional array with a kernel comprises of taking the kernel, sliding it along the array, multiplying it with the items in the array that overlap with the kernel in that location and summing this product.\n",
    "\n",
    "Multiply each window in the input array with the kernel and sum the multiplied result and allocate the result into the correct entry in the output array (conv)."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "array = np.array([1, 0, 1, 0, 1, 0, 1, 0, 1, 0])\n",
    "kernel = np.array([1, -1, 0])\n",
    "conv = np.array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])\n",
    "\n",
    "# Output array\n",
    "for ii in range(8):\n",
    "    conv[ii] = (kernel * array[ii:ii+3]).sum()\n",
    "\n",
    "# Print conv\n",
    "print(conv)"
   ]
  },
  {
   "source": [
    "### Image convolutions\n",
    "The convolution of an image with a kernel summarizes a part of the image as the sum of the multiplication of that part of the image with the kernel. In this exercise, you will write the code that executes a convolution of an image with a kernel using Numpy. Given a black and white image that is stored in the variable im, write the operations inside the loop that would execute the convolution with the provided kernel.\n",
    "\n",
    "- Select the right window from the image in each iteration and multiply this part of the image with the kernel.\n",
    "- Sum the result and allocate the sum to the correct entry in the output array (results)."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "kernel = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]])\n",
    "result = np.zeros(im.shape)\n",
    "\n",
    "# Output array\n",
    "for ii in range(im.shape[0] - 3):\n",
    "    for jj in range(im.shape[1] - 3):\n",
    "        result[ii, jj] = (im[ii:ii+3, jj:jj+3] * kernel).sum()\n",
    "\n",
    "# Print result\n",
    "print(result)"
   ]
  },
  {
   "source": [
    "### Defining image convolution kernels\n",
    "In the previous exercise, you wrote code that performs a convolution given an image and a kernel. This code is now stored in a function called convolution() that takes two inputs: image and kernel and produces the convolved image. In this exercise, you will be asked to define the kernel that finds a particular feature in the image.\n",
    "\n",
    "For example, the following kernel finds a vertical line in images:"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define a kernel that finds horizontal lines in images.\n",
    "kernel = np.array([[-1, -1, -1], \n",
    "                   [1, 1, 1],\n",
    "                   [-1, -1, -1]])\n",
    "\n",
    "# Define a kernel that finds a light spot surrounded by dark pixels.\n",
    "kernel = np.array([[-1, -1, -1], \n",
    "                   [-1, 1, -1],\n",
    "                   [-1, -1, -1]])\n",
    "\n",
    "# Define a kernel that finds a dark spot surrounded by bright pixels.\n",
    "kernel = np.array([[1, 1, 1], \n",
    "                   [1, -1, 1],\n",
    "                   [1, 1, 1]])"
   ]
  },
  {
   "source": [
    "### Convolutional network for image classification\n",
    "Convolutional networks for classification are constructed from a sequence of convolutional layers (for image processing) and fully connected (Dense) layers (for readout). In this exercise, you will construct a small convolutional network for classification of the data from the fashion dataset.\n",
    "\n",
    "### Training a CNN to classify clothing types\n",
    "Before training a neural network it needs to be compiled with the right cost function, using the right optimizer. During compilation, you can also define metrics that the network calculates and reports in every epoch. Model fitting requires a training data set, together with the training labels to the network.\n",
    "\n",
    "The Conv2D model you built in the previous exercise is available in your workspace.\n",
    "\n",
    "### Evaluating a CNN with test data\n",
    "To evaluate a trained neural network, you should provide a separate testing data set of labeled images. The model you fit in the previous exercise is available in your workspace."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the necessary components from Keras\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Conv2D, Flatten\n",
    "\n",
    "# Initialize the model object\n",
    "model = Sequential()\n",
    "\n",
    "# Add a convolutional layer\n",
    "model.add(Conv2D(10, kernel_size=3, activation='relu', \n",
    "               input_shape=(img_rows, img_cols, 1)))\n",
    "\n",
    "# Flatten the output of the convolutional layer\n",
    "model.add(Flatten())\n",
    "# Add an output layer for the 3 categories\n",
    "model.add(Dense(3, activation='softmax'))\n",
    "\n",
    "# Compile the model \n",
    "model.compile(optimizer='adam', \n",
    "              loss='categorical_crossentropy', \n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Fit the model on a training set\n",
    "model.fit(train_data, train_labels, \n",
    "          validation_split=0.2, \n",
    "          epochs=3, batch_size=10)\n",
    "\n",
    "# Evaluate the model on separate test data\n",
    "model.evaluate(test_data, test_labels, batch_size=10)"
   ]
  },
  {
   "source": [
    "### Add padding to a CNN\n",
    "Padding allows a convolutional layer to retain the resolution of the input into this layer. This is done by adding zeros around the edges of the input image, so that the convolution kernel can overlap with the pixels on the edge of the image.\n",
    "\n",
    "### Add strides to a convolutional network\n",
    "The size of the strides of the convolution kernel determines whether the kernel will skip over some of the pixels as it slides along the image. This affects the size of the output because when strides are larger than one, the kernel will be centered on only some of the pixels.\n",
    "\n",
    "- Calculating the size of the output\n",
    "- O = ((I - K + 2P)/S)+1\n",
    "- I = size of the input\n",
    "- K = size of the kernel\n",
    "- P = size of the zero padding\n",
    "- S = strides"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the model\n",
    "model = Sequential()\n",
    "\n",
    "# Add the convolutional layer\n",
    "model.add(Conv2D(10, kernel_size=3, activation='relu', \n",
    "                 input_shape=(img_rows, img_cols, 1), \n",
    "                 padding='same'))\n",
    "\n",
    "# Feed into output layer\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))\n",
    "\n",
    "# Initialize the model\n",
    "model = Sequential()\n",
    "\n",
    "# Add the convolutional layer\n",
    "model.add(Conv2D(10, kernel_size=3, activation='relu', \n",
    "              input_shape=(img_rows, img_cols, 1), \n",
    "              strides=2))\n",
    "\n",
    "# Feed into output layer\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))"
   ]
  },
  {
   "source": [
    "### Creating a deep learning network\n",
    "A deep convolutional neural network is a network that has more than one layer. Each layer in a deep network receives its input from the preceding layer, with the very first layer receiving its input from the images used as training or test data.\n",
    "\n",
    "Here, you will create a network that has two convolutional layers.\n",
    "\n",
    "### Train a deep CNN to classify clothing images\n",
    "Training a deep learning model is very similar to training a single layer network. Once the model is constructed (as you have done in the previous exercise), the model needs to be compiled with the right set of parameters. Then, the model is fit by providing it with training data, as well as training labels. After training is done, the model can be evaluated on test data."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Conv2D, Flatten\n",
    "\n",
    "model = Sequential()\n",
    "\n",
    "# Add a convolutional layer (15 units)\n",
    "model.add(Conv2D(15, kernel_size=2, activation='relu', input_shape=(img_rows, img_cols, 1)))\n",
    "\n",
    "# Add another convolutional layer (5 units)\n",
    "model.add(Conv2D(5, kernel_size=2, activation='relu'))\n",
    "\n",
    "# Flatten and feed to output layer\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))\n",
    "\n",
    "# Compile model\n",
    "model.compile(optimizer='adam', \n",
    "              loss='categorical_crossentropy', \n",
    "              metrics=['accuracy'])\n",
    "\n",
    "# Fit the model to training data \n",
    "model.fit(train_data, train_labels, \n",
    "          validation_split=0.2, \n",
    "          epochs=3, batch_size=10)\n",
    "\n",
    "# Evaluate the model on test data\n",
    "model.evaluate(test_data, test_labels, batch_size=10)"
   ]
  },
  {
   "source": [
    "### How many parameters in a deep CNN?\n",
    "In this exercise, you will use Keras to calculate the total number of parameters along with the number of parameters in each layer of the network."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# CNN model\n",
    "model = Sequential()\n",
    "model.add(Conv2D(10, kernel_size=2, activation='relu', \n",
    "                 input_shape=(28, 28, 1)))\n",
    "model.add(Conv2D(10, kernel_size=2, activation='relu'))\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))\n",
    "\n",
    "# Summarize the model \n",
    "model.summary()"
   ]
  },
  {
   "source": [
    "### Write your own pooling operation\n",
    "As we have seen before, CNNs can have a lot of parameters. Pooling layers are often added between the convolutional layers of a neural network to summarize their outputs in a condensed manner, and reduce the number of parameters in the next layer in the network. This can help us if we want to train the network more rapidly, or if we don't have enough data to learn a very large number of parameters.\n",
    "\n",
    "A pooling layer can be described as a particular kind of convolution. For every window in the input it finds the maximal pixel value and passes only this pixel through. In this exercise, you will write your own max pooling operation, based on the code that you previously used to write a two-dimensional convolution operation."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Result placeholder\n",
    "result = np.zeros((im.shape[0]//2, im.shape[1]//2))\n",
    "\n",
    "# Pooling operation\n",
    "for ii in range(result.shape[0]):\n",
    "    for jj in range(result.shape[1]):\n",
    "        result[ii, jj] = np.max(im[ii*2:ii*2+2, jj*2:jj*2+2])"
   ]
  },
  {
   "source": [
    "### Keras pooling layers\n",
    "Keras implements a pooling operation as a layer that can be added to CNNs between other layers. In this exercise, you will construct a convolutional neural network similar to the one you have constructed before:\n",
    "\n",
    "Convolution => Convolution => Flatten => Dense\n",
    "\n",
    "However, you will also add a pooling layer. The architecture will add a single max-pooling layer between the convolutional layer and the dense layer with a pooling of 2x2:\n",
    "\n",
    "Convolution => Max pooling => Convolution => Flatten => Dense\n",
    "\n",
    "A Sequential model along with Dense, Conv2D, Flatten, and MaxPool2D objects are available in your workspace."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a convolutional layer\n",
    "model.add(Conv2D(15, kernel_size=2, activation='relu', \n",
    "                 input_shape=(img_rows, img_cols, 1)))\n",
    "\n",
    "# Add a pooling operation\n",
    "model.add(MaxPool2D(2))\n",
    "\n",
    "# Add another convolutional layer\n",
    "model.add(Conv2D(5, kernel_size=2, activation='relu'))\n",
    "\n",
    "# Flatten and feed to output layer\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))\n",
    "model.summary()"
   ]
  },
  {
   "source": [
    "### Train a deep CNN with pooling to classify images\n",
    "Training a CNN with pooling layers is very similar to training of the deep networks that y have seen before. Once the network is constructed (as you did in the previous exercise), the model needs to be appropriately compiled, and then training data needs to be provided, together with the other arguments that control the fitting procedure.\n",
    "\n",
    "The following model from the previous exercise is available in your workspace:\n",
    "\n",
    "Convolution => Max pooling => Convolution => Flatten => Dense"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compile the model\n",
    "model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])\n",
    "\n",
    "# Fit to training data\n",
    "model.fit(train_data, train_labels, \n",
    "          validation_split=0.2, \n",
    "          epochs=3, batch_size=10)\n",
    "\n",
    "# Evaluate on test data \n",
    "model.evaluate(test_data, test_labels, batch_size=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Plot the learning curves\n",
    "During learning, the model will store the loss function evaluated in each epoch. Looking at the learning curves can tell us quite a bit about the learning process. In this exercise, you will plot the learning and validation loss curves for a model that you will train."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Train the model and store the training object\n",
    "training = model.fit(train_data, train_labels, validation_split=0.2, epochs=3, batch_size=10)\n",
    "\n",
    "# Extract the history from the training object\n",
    "history = training.history\n",
    "\n",
    "# Plot the training loss \n",
    "plt.plot(history['loss'])\n",
    "# Plot the validation loss\n",
    "plt.plot(history['val_loss'])\n",
    "\n",
    "# Show the figure\n",
    "plt.show()"
   ]
  },
  {
   "source": [
    "### Using stored weights to predict in a test set\n",
    "Model weights stored in an hdf5 file can be reused to populate an untrained model. Once the weights are loaded into this model, it behaves just like a model that has been trained to reach these weights. For example, you can use this model to make predictions from an unseen data set (e.g. test_data)."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the weights from file\n",
    "model.load_weights('weights.hdf5')\n",
    "\n",
    "# Predict from the first three images in the test data\n",
    "model.predict(test_data[:3])"
   ]
  },
  {
   "source": [
    "## Regularization\n",
    "### Adding dropout to your network\n",
    "Dropout is a form of regularization that removes a different random subset of the units in a layer in each round of training. In this exercise, we will add dropout to the convolutional neural network that we have used in previous exercises."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a convolutional layer\n",
    "model.add(Conv2D(15, kernel_size=2, activation='relu', \n",
    "                 input_shape=(img_rows, img_cols, 1)))\n",
    "\n",
    "# Add a dropout layer\n",
    "model.add(Dropout(0.2))\n",
    "\n",
    "# Add another convolutional layer\n",
    "model.add(Conv2D(5, kernel_size=2, activation='relu'))\n",
    "\n",
    "# Flatten and feed to output layer\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))"
   ]
  },
  {
   "source": [
    "### Add batch normalization to your network\n",
    "Batch normalization is another form of regularization that rescales the outputs of a layer to make sure that they have mean 0 and standard deviation 1. In this exercise, we will add batch normalization to the convolutional neural network that we have used in previous exercises"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Add a convolutional layer\n",
    "model.add(Conv2D(15, kernel_size=2, activation='relu', \n",
    "                 input_shape=(img_rows, img_cols, 1)))\n",
    "\n",
    "\n",
    "# Add batch normalization layer\n",
    "model.add(BatchNormalization())\n",
    "\n",
    "# Add another convolutional layer\n",
    "model.add(Conv2D(5, kernel_size=2, activation='relu'))\n",
    "\n",
    "# Flatten and feed to output layer\n",
    "model.add(Flatten())\n",
    "model.add(Dense(3, activation='softmax'))"
   ]
  },
  {
   "source": [
    "## Interpreting the model\n",
    "### Extracting a kernel from a trained network\n",
    "One way to interpret models is to examine the properties of the kernels in the convolutional layers. In this exercise, you will extract one of the kernels from a convolutional neural network with weights that you saved in a hdf5 file.\n",
    "\n",
    "A Keras neural network stores its layers in a list called model.layers. For the convolutional layers, you can get the weights using the .get_weights() method. This returns a list, and the first item in this list is an array representing the weights of the convolutional kernels. If the shape of this array is (2, 2, 1, 5), what does the first number (2) represent? Each of the 2s in this shape is one of the dimensions of the kernel"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load the weights into the model\n",
    "model.load_weights('weights.hdf5')\n",
    "\n",
    "# Get the first convolutional layer from the model\n",
    "c1 = model.layers[0]\n",
    "\n",
    "# Get the weights of the first convolutional layer\n",
    "weights1 = c1.get_weights()\n",
    "\n",
    "# Pull out the first channel of the first kernel in the first layer\n",
    "kernel = weights1[0][...,0, 0]\n",
    "print(kernel)"
   ]
  },
  {
   "source": [
    "### Visualizing kernel responses\n",
    "One of the ways to interpret the weights of a neural network is to see how the kernels stored in these weights \"see\" the world. That is, what properties of an image are emphasized by this kernel. In this exercise, we will do that by convolving an image with the kernel and visualizing the result. Given images in the test_data variable, a function called extract_kernel() that extracts a kernel from the provided network, and the function called convolution() that we defined in the first chapter, extract the kernel, load the data from a file and visualize it with matplotlib.\n",
    "\n",
    "A deep CNN model, a function convolution(), along with the kernel you extracted in an earlier exercise is available in your workspace. Use the convolution() function to convolve the extracted kernel with the first channel of the fourth item in the image array.\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Convolve with the fourth image in test_data\n",
    "out = convolution(test_data[3, :, :, 0], kernel)\n",
    "\n",
    "# Visualize the result\n",
    "plt.imshow(out)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ]
}