diff --git a/09_up_and_running_with_tensorflow.ipynb b/09_up_and_running_with_tensorflow.ipynb
index 252a7e0..ad610a7 100644
--- a/09_up_and_running_with_tensorflow.ipynb
+++ b/09_up_and_running_with_tensorflow.ipynb
@@ -2,40 +2,28 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "**Chapter 9 – Up and running with TensorFlow**"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "_This notebook contains all the sample code and solutions to the exercises in chapter 9._"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Setup"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "First, let's make sure this notebook works well in both python 2 and 3, import a few common modules, ensure MatplotLib plots figures inline and prepare a function to save the figures:"
    ]
@@ -43,11 +31,7 @@
   {
    "cell_type": "code",
    "execution_count": 1,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "# To support both python 2 and python 3\n",
@@ -85,10 +69,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Creating and running a graph"
    ]
@@ -96,11 +77,7 @@
   {
    "cell_type": "code",
    "execution_count": 2,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import tensorflow as tf\n",
@@ -115,11 +92,7 @@
   {
    "cell_type": "code",
    "execution_count": 3,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "f"
@@ -128,11 +101,7 @@
   {
    "cell_type": "code",
    "execution_count": 4,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "sess = tf.Session()\n",
@@ -146,9 +115,7 @@
    "cell_type": "code",
    "execution_count": 5,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -158,11 +125,7 @@
   {
    "cell_type": "code",
    "execution_count": 6,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "with tf.Session() as sess:\n",
@@ -174,11 +137,7 @@
   {
    "cell_type": "code",
    "execution_count": 7,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "result"
@@ -187,11 +146,7 @@
   {
    "cell_type": "code",
    "execution_count": 8,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "init = tf.global_variables_initializer()\n",
@@ -204,11 +159,7 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "result"
@@ -218,9 +169,7 @@
    "cell_type": "code",
    "execution_count": 10,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -230,11 +179,7 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "sess = tf.InteractiveSession()\n",
@@ -247,9 +192,7 @@
    "cell_type": "code",
    "execution_count": 12,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -259,11 +202,7 @@
   {
    "cell_type": "code",
    "execution_count": 13,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "result"
@@ -271,10 +210,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Managing graphs"
    ]
@@ -282,11 +218,7 @@
   {
    "cell_type": "code",
    "execution_count": 14,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -298,11 +230,7 @@
   {
    "cell_type": "code",
    "execution_count": 15,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "graph = tf.Graph()\n",
@@ -316,9 +244,6 @@
    "cell_type": "code",
    "execution_count": 16,
    "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true,
     "scrolled": true
    },
    "outputs": [],
@@ -329,11 +254,7 @@
   {
    "cell_type": "code",
    "execution_count": 17,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "w = tf.constant(3)\n",
@@ -349,11 +270,7 @@
   {
    "cell_type": "code",
    "execution_count": 18,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "with tf.Session() as sess:\n",
@@ -364,20 +281,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Linear Regression"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Using the Normal Equation"
    ]
@@ -385,11 +296,7 @@
   {
    "cell_type": "code",
    "execution_count": 19,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "import numpy as np\n",
@@ -413,11 +320,7 @@
   {
    "cell_type": "code",
    "execution_count": 20,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "theta_value"
@@ -425,10 +328,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Compare with pure NumPy"
    ]
@@ -436,11 +336,7 @@
   {
    "cell_type": "code",
    "execution_count": 21,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "X = housing_data_plus_bias\n",
@@ -452,10 +348,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Compare with Scikit-Learn"
    ]
@@ -463,11 +356,7 @@
   {
    "cell_type": "code",
    "execution_count": 22,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "from sklearn.linear_model import LinearRegression\n",
@@ -479,20 +368,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Using Batch Gradient Descent"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Gradient Descent requires scaling the feature vectors first. We could do this using TF, but let's just use Scikit-Learn for now."
    ]
@@ -501,9 +384,7 @@
    "cell_type": "code",
    "execution_count": 23,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -516,11 +397,7 @@
   {
    "cell_type": "code",
    "execution_count": 24,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "print(scaled_housing_data_plus_bias.mean(axis=0))\n",
@@ -531,10 +408,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Manually computing the gradients"
    ]
@@ -542,11 +416,7 @@
   {
    "cell_type": "code",
    "execution_count": 25,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -579,11 +449,7 @@
   {
    "cell_type": "code",
    "execution_count": 26,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "best_theta"
@@ -591,20 +457,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Using autodiff"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Same as above except for the `gradients = ...` line:"
    ]
@@ -612,11 +472,7 @@
   {
    "cell_type": "code",
    "execution_count": 27,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -636,9 +492,7 @@
    "cell_type": "code",
    "execution_count": 28,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -648,11 +502,7 @@
   {
    "cell_type": "code",
    "execution_count": 29,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "training_op = tf.assign(theta, theta - learning_rate * gradients)\n",
@@ -675,10 +525,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "How could you find the partial derivatives of the following function with regards to `a` and `b`?"
    ]
@@ -687,9 +534,7 @@
    "cell_type": "code",
    "execution_count": 30,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -703,11 +548,7 @@
   {
    "cell_type": "code",
    "execution_count": 31,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "my_func(0.2, 0.3)"
@@ -716,11 +557,7 @@
   {
    "cell_type": "code",
    "execution_count": 32,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -737,10 +574,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's compute the function at $a=0.2$ and $b=0.3$, and the partial derivatives at that point with regards to $a$ and with regards to $b$:"
    ]
@@ -748,11 +582,7 @@
   {
    "cell_type": "code",
    "execution_count": 33,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "with tf.Session() as sess:\n",
@@ -763,10 +593,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Using a `GradientDescentOptimizer`"
    ]
@@ -774,11 +601,7 @@
   {
    "cell_type": "code",
    "execution_count": 34,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -798,9 +621,7 @@
    "cell_type": "code",
    "execution_count": 35,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -811,11 +632,7 @@
   {
    "cell_type": "code",
    "execution_count": 36,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "init = tf.global_variables_initializer()\n",
@@ -836,10 +653,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Using a momentum optimizer"
    ]
@@ -847,11 +661,7 @@
   {
    "cell_type": "code",
    "execution_count": 37,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -871,9 +681,7 @@
    "cell_type": "code",
    "execution_count": 38,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -885,9 +693,7 @@
    "cell_type": "code",
    "execution_count": 39,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -899,11 +705,7 @@
   {
    "cell_type": "code",
    "execution_count": 40,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "with tf.Session() as sess:\n",
@@ -920,20 +722,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Feeding data to the training algorithm"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Placeholder nodes"
    ]
@@ -941,11 +737,7 @@
   {
    "cell_type": "code",
    "execution_count": 41,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -962,11 +754,7 @@
   {
    "cell_type": "code",
    "execution_count": 42,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "print(B_val_2)"
@@ -974,10 +762,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Mini-batch Gradient Descent"
    ]
@@ -985,11 +770,7 @@
   {
    "cell_type": "code",
    "execution_count": 43,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "n_epochs = 1000\n",
@@ -1000,9 +781,7 @@
    "cell_type": "code",
    "execution_count": 44,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1016,9 +795,7 @@
    "cell_type": "code",
    "execution_count": 45,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1036,9 +813,7 @@
    "cell_type": "code",
    "execution_count": 46,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1049,9 +824,7 @@
    "cell_type": "code",
    "execution_count": 47,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1062,11 +835,7 @@
   {
    "cell_type": "code",
    "execution_count": 48,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "def fetch_batch(epoch, batch_index, batch_size):\n",
@@ -1090,11 +859,7 @@
   {
    "cell_type": "code",
    "execution_count": 49,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "best_theta"
@@ -1102,10 +867,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Saving and restoring a model"
    ]
@@ -1113,11 +875,7 @@
   {
    "cell_type": "code",
    "execution_count": 50,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1153,11 +911,7 @@
   {
    "cell_type": "code",
    "execution_count": 51,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "best_theta"
@@ -1166,11 +920,7 @@
   {
    "cell_type": "code",
    "execution_count": 52,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "with tf.Session() as sess:\n",
@@ -1181,11 +931,7 @@
   {
    "cell_type": "code",
    "execution_count": 53,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "np.allclose(best_theta, best_theta_restored)"
@@ -1193,10 +939,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "If you want to have a saver that loads and restores `theta` with a different name, such as `\"weights\"`:"
    ]
@@ -1205,9 +948,7 @@
    "cell_type": "code",
    "execution_count": 54,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1216,10 +957,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "By default the saver also saves the graph structure itself in a second file with the extension `.meta`. You can use the function `tf.train.import_meta_graph()` to restore the graph structure. This function loads the graph into the default graph and returns a `Saver` that can then be used to restore the graph state (i.e., the variable values):"
    ]
@@ -1227,11 +965,7 @@
   {
    "cell_type": "code",
    "execution_count": 55,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1248,11 +982,7 @@
   {
    "cell_type": "code",
    "execution_count": 56,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "np.allclose(best_theta, best_theta_restored)"
@@ -1260,20 +990,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "This means that you can import a pretrained model without having to have the corresponding Python code to build the graph. This is very handy when you keep tweaking and saving your model: you can load a previously saved model without having to search for the version of the code that built it."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Visualizing the graph\n",
     "## inside Jupyter"
@@ -1283,9 +1007,7 @@
    "cell_type": "code",
    "execution_count": 57,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1331,9 +1053,6 @@
    "cell_type": "code",
    "execution_count": 58,
    "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true,
     "scrolled": true
    },
    "outputs": [],
@@ -1343,10 +1062,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Using TensorBoard"
    ]
@@ -1354,11 +1070,7 @@
   {
    "cell_type": "code",
    "execution_count": 59,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1374,9 +1086,7 @@
    "cell_type": "code",
    "execution_count": 60,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1398,11 +1108,7 @@
   {
    "cell_type": "code",
    "execution_count": 61,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "mse_summary = tf.summary.scalar('MSE', mse)\n",
@@ -1413,9 +1119,7 @@
    "cell_type": "code",
    "execution_count": 62,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1427,11 +1131,7 @@
   {
    "cell_type": "code",
    "execution_count": 63,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "with tf.Session() as sess:                                                        # not shown in the book\n",
@@ -1453,9 +1153,7 @@
    "cell_type": "code",
    "execution_count": 64,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1465,11 +1163,7 @@
   {
    "cell_type": "code",
    "execution_count": 65,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "best_theta"
@@ -1477,10 +1171,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Name scopes"
    ]
@@ -1488,11 +1179,7 @@
   {
    "cell_type": "code",
    "execution_count": 66,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1514,9 +1201,7 @@
    "cell_type": "code",
    "execution_count": 67,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1529,9 +1214,7 @@
    "cell_type": "code",
    "execution_count": 68,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1547,11 +1230,7 @@
   {
    "cell_type": "code",
    "execution_count": 69,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "n_epochs = 10\n",
@@ -1581,11 +1260,7 @@
   {
    "cell_type": "code",
    "execution_count": 70,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "print(error.op.name)"
@@ -1594,11 +1269,7 @@
   {
    "cell_type": "code",
    "execution_count": 71,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "print(mse.op.name)"
@@ -1607,11 +1278,7 @@
   {
    "cell_type": "code",
    "execution_count": 72,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1631,20 +1298,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Modularity"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "An ugly flat code:"
    ]
@@ -1652,11 +1313,7 @@
   {
    "cell_type": "code",
    "execution_count": 73,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1680,10 +1337,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Much better, using a function to build the ReLUs:"
    ]
@@ -1691,11 +1345,7 @@
   {
    "cell_type": "code",
    "execution_count": 74,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1717,9 +1367,7 @@
    "cell_type": "code",
    "execution_count": 75,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1728,10 +1376,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Even better using name scopes:"
    ]
@@ -1739,11 +1384,7 @@
   {
    "cell_type": "code",
    "execution_count": 76,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1761,9 +1402,7 @@
    "cell_type": "code",
    "execution_count": 77,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1778,20 +1417,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Sharing Variables"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Sharing a `threshold` variable the classic way, by defining it outside of the `relu()` function then passing it as a parameter:"
    ]
@@ -1799,11 +1432,7 @@
   {
    "cell_type": "code",
    "execution_count": 78,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1825,11 +1454,7 @@
   {
    "cell_type": "code",
    "execution_count": 79,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1849,9 +1474,7 @@
    "cell_type": "code",
    "execution_count": 80,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1864,9 +1487,7 @@
    "cell_type": "code",
    "execution_count": 81,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1881,9 +1502,7 @@
    "cell_type": "code",
    "execution_count": 82,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1895,9 +1514,7 @@
    "cell_type": "code",
    "execution_count": 83,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -1909,11 +1526,7 @@
   {
    "cell_type": "code",
    "execution_count": 84,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1938,11 +1551,7 @@
   {
    "cell_type": "code",
    "execution_count": 85,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "file_writer = tf.summary.FileWriter(\"logs/relu6\", tf.get_default_graph())\n",
@@ -1952,11 +1561,7 @@
   {
    "cell_type": "code",
    "execution_count": 86,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -1984,11 +1589,7 @@
   {
    "cell_type": "code",
    "execution_count": 87,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -2014,9 +1615,7 @@
    "cell_type": "code",
    "execution_count": 88,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2026,10 +1625,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Extra material"
    ]
@@ -2037,11 +1633,7 @@
   {
    "cell_type": "code",
    "execution_count": 89,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -2069,10 +1661,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "The first `variable_scope()` block first creates the shared variable `x0`, named `my_scope/x`. For all operations other than shared variables (including non-shared variables), the variable scope acts like a regular name scope, which is why the two variables `x1` and `x2` have a name with a prefix `my_scope/`. Note however that TensorFlow makes their names unique by adding an index: `my_scope/x_1` and `my_scope/x_2`.\n",
     "\n",
@@ -2083,10 +1672,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Strings"
    ]
@@ -2094,11 +1680,7 @@
   {
    "cell_type": "code",
    "execution_count": 90,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -2112,10 +1694,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Implementing a Home-Made Computation Graph"
    ]
@@ -2123,11 +1702,7 @@
   {
    "cell_type": "code",
    "execution_count": 91,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "class Const(object):\n",
@@ -2173,10 +1748,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## Computing gradients\n",
     "### Mathematical differentiation"
@@ -2185,11 +1757,7 @@
   {
    "cell_type": "code",
    "execution_count": 92,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "df_dx = Mul(Const(2), Mul(x, y))  # df/dx = 2xy\n",
@@ -2200,10 +1768,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Numerical differentiation"
    ]
@@ -2211,11 +1776,7 @@
   {
    "cell_type": "code",
    "execution_count": 93,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "def gradients(func, vars_list, eps=0.0001):\n",
@@ -2237,10 +1798,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Symbolic differentiation"
    ]
@@ -2248,11 +1806,7 @@
   {
    "cell_type": "code",
    "execution_count": 94,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "Const.derive = lambda self, var: Const(0)\n",
@@ -2272,10 +1826,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Automatic differentiation (autodiff) – forward mode"
    ]
@@ -2284,9 +1835,7 @@
    "cell_type": "code",
    "execution_count": 95,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2321,10 +1870,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$3 + (3 + 4 \\epsilon) = 6 + 4\\epsilon$"
    ]
@@ -2332,11 +1878,7 @@
   {
    "cell_type": "code",
    "execution_count": 96,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "3 + DualNumber(3, 4)"
@@ -2344,10 +1886,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "$(3 + 4ε)\\times(5 + 7ε) = 3 \\times 5 + 3 \\times 7ε + 4ε \\times 5 + 4ε \\times 7ε = 15 + 21ε + 20ε + 28ε^2 = 15 + 41ε + 28 \\times 0 = 15 + 41ε$"
    ]
@@ -2355,11 +1894,7 @@
   {
    "cell_type": "code",
    "execution_count": 97,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "DualNumber(3, 4) * DualNumber(5, 7)"
@@ -2368,11 +1903,7 @@
   {
    "cell_type": "code",
    "execution_count": 98,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "x.value = DualNumber(3.0)\n",
@@ -2384,11 +1915,7 @@
   {
    "cell_type": "code",
    "execution_count": 99,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "x.value = DualNumber(3.0, 1.0)  # 3 + ε\n",
@@ -2405,11 +1932,7 @@
   {
    "cell_type": "code",
    "execution_count": 100,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "df_dx"
@@ -2418,11 +1941,7 @@
   {
    "cell_type": "code",
    "execution_count": 101,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "df_dy"
@@ -2430,10 +1949,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Autodiff – Reverse mode"
    ]
@@ -2441,11 +1957,7 @@
   {
    "cell_type": "code",
    "execution_count": 102,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "class Const(object):\n",
@@ -2510,10 +2022,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "### Autodiff – reverse mode (using TensorFlow)"
    ]
@@ -2521,11 +2030,7 @@
   {
    "cell_type": "code",
    "execution_count": 103,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "reset_graph()\n",
@@ -2547,20 +2052,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "# Exercise solutions"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## 1. to 11."
    ]
@@ -2568,9 +2067,7 @@
   {
    "cell_type": "markdown",
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "source": [
     "See appendix A."
@@ -2578,20 +2075,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "## 12. Logistic Regression with Mini-Batch Gradient Descent using TensorFlow"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "First, let's create the moons dataset using Scikit-Learn's `make_moons()` function:"
    ]
@@ -2599,11 +2090,7 @@
   {
    "cell_type": "code",
    "execution_count": 104,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "from sklearn.datasets import make_moons\n",
@@ -2614,10 +2101,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's take a peek at the dataset:"
    ]
@@ -2625,11 +2109,7 @@
   {
    "cell_type": "code",
    "execution_count": 105,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "plt.plot(X_moons[y_moons == 1, 0], X_moons[y_moons == 1, 1], 'go', label=\"Positive\")\n",
@@ -2640,10 +2120,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "We must not forget to add an extra bias feature ($x_0 = 1$) to every instance. For this, we just need to add a column full of 1s on the left of the input matrix $\\mathbf{X}$:"
    ]
@@ -2651,11 +2128,7 @@
   {
    "cell_type": "code",
    "execution_count": 106,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "X_moons_with_bias = np.c_[np.ones((m, 1)), X_moons]"
@@ -2663,10 +2136,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's check:"
    ]
@@ -2674,11 +2144,7 @@
   {
    "cell_type": "code",
    "execution_count": 107,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "X_moons_with_bias[:5]"
@@ -2686,10 +2152,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Looks good. Now let's reshape `y_train` to make it a column vector (i.e. a 2D array with a single column):"
    ]
@@ -2698,9 +2161,7 @@
    "cell_type": "code",
    "execution_count": 108,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2709,10 +2170,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Now let's split the data into a training set and a test set:"
    ]
@@ -2721,9 +2179,7 @@
    "cell_type": "code",
    "execution_count": 109,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2737,10 +2193,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Ok, now let's create a small function to generate training batches. In this implementation we will just pick random instances from the training set for each batch. This means that a single batch may contain the same instance multiple times, and also a single epoch may not cover all the training instances (in fact it will generally cover only about two thirds of the instances). However, in practice this is not an issue and it simplifies the code:"
    ]
@@ -2749,9 +2202,7 @@
    "cell_type": "code",
    "execution_count": 110,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2764,10 +2215,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's look at a small batch:"
    ]
@@ -2775,11 +2223,7 @@
   {
    "cell_type": "code",
    "execution_count": 111,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "X_batch, y_batch = random_batch(X_train, y_train, 5)\n",
@@ -2789,11 +2233,7 @@
   {
    "cell_type": "code",
    "execution_count": 112,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "y_batch"
@@ -2801,20 +2241,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Great! Now that the data is ready to be fed to the model, we need to build that model. Let's start with a simple implementation, then we will add all the bells and whistles."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "First let's reset the default graph."
    ]
@@ -2823,9 +2257,7 @@
    "cell_type": "code",
    "execution_count": 113,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2834,10 +2266,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "The _moons_ dataset has two input features, since each instance is a point on a plane (i.e., 2-Dimensional):"
    ]
@@ -2846,9 +2275,7 @@
    "cell_type": "code",
    "execution_count": 114,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2857,28 +2284,22 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Now let's build the Logistic Regression model. As we saw in chapter 4, this model first computes a weighted sum of the inputs (just like the Linear Regression model), and then it applies the sigmoid function to the result, which gives us the estimated probability for the positive class:\n",
     "\n",
-    "$\\hat{p} = h_\\mathbf{\\theta}(\\mathbf{x}) = \\sigma(\\mathbf{\\theta}^T \\cdot \\mathbf{x})$\n"
+    "$\\hat{p} = h_\\boldsymbol{\\theta}(\\mathbf{x}) = \\sigma(\\boldsymbol{\\theta}^T \\mathbf{x})$\n"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
-    "Recall that $\\mathbf{\\theta}$ is the parameter vector, containing the bias term $\\theta_0$ and the weights $\\theta_1, \\theta_2, \\dots, \\theta_n$. The input vector $\\mathbf{x}$ contains a constant term $x_0 = 1$, as well as all the input features $x_1, x_2, \\dots, x_n$.\n",
+    "Recall that $\\boldsymbol{\\theta}$ is the parameter vector, containing the bias term $\\theta_0$ and the weights $\\theta_1, \\theta_2, \\dots, \\theta_n$. The input vector $\\mathbf{x}$ contains a constant term $x_0 = 1$, as well as all the input features $x_1, x_2, \\dots, x_n$.\n",
     "\n",
     "Since we want to be able to make predictions for multiple instances at a time, we will use an input matrix $\\mathbf{X}$ rather than a single input vector. The $i^{th}$ row will contain the transpose of the $i^{th}$ input vector $(\\mathbf{x}^{(i)})^T$. It is then possible to estimate the probability that each instance belongs to the positive class using the following equation:\n",
     "\n",
-    "$ \\hat{\\mathbf{p}} = \\sigma(\\mathbf{X} \\cdot \\mathbf{\\theta})$\n",
+    "$ \\hat{\\mathbf{p}} = \\sigma(\\mathbf{X} \\boldsymbol{\\theta})$\n",
     "\n",
     "That's all we need to build the model:"
    ]
@@ -2887,9 +2308,7 @@
    "cell_type": "code",
    "execution_count": 115,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2902,10 +2321,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "In fact, TensorFlow has a nice function `tf.sigmoid()` that we can use to simplify the last line of the previous code:"
    ]
@@ -2914,9 +2330,7 @@
    "cell_type": "code",
    "execution_count": 116,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2925,14 +2339,11 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "As we saw in chapter 4, the log loss is a good cost function to use for Logistic Regression:\n",
     "\n",
-    "$J(\\mathbf{\\theta}) = -\\dfrac{1}{m} \\sum\\limits_{i=1}^{m}{\\left[ y^{(i)} log\\left(\\hat{p}^{(i)}\\right) + (1 - y^{(i)}) log\\left(1 - \\hat{p}^{(i)}\\right)\\right]}$\n",
+    "$J(\\boldsymbol{\\theta}) = -\\dfrac{1}{m} \\sum\\limits_{i=1}^{m}{\\left[ y^{(i)} \\log\\left(\\hat{p}^{(i)}\\right) + (1 - y^{(i)}) \\log\\left(1 - \\hat{p}^{(i)}\\right)\\right]}$\n",
     "\n",
     "One option is to implement it ourselves:"
    ]
@@ -2940,11 +2351,7 @@
   {
    "cell_type": "code",
    "execution_count": 117,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "epsilon = 1e-7  # to avoid an overflow when computing the log\n",
@@ -2953,10 +2360,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "But we might as well use TensorFlow's `tf.losses.log_loss()` function:"
    ]
@@ -2965,9 +2369,7 @@
    "cell_type": "code",
    "execution_count": 118,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -2976,10 +2378,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "The rest is pretty standard: let's create the optimizer and tell it to minimize the cost function:"
    ]
@@ -2988,9 +2387,7 @@
    "cell_type": "code",
    "execution_count": 119,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3001,10 +2398,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "All we need now (in this minimal version) is the variable initializer:"
    ]
@@ -3013,9 +2407,7 @@
    "cell_type": "code",
    "execution_count": 120,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3024,20 +2416,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "And we are ready to train the model and use it for predictions!"
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "There's really nothing special about this code, it's virtually the same as the one we used earlier for Linear Regression:"
    ]
@@ -3045,11 +2431,7 @@
   {
    "cell_type": "code",
    "execution_count": 121,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "n_epochs = 1000\n",
@@ -3072,20 +2454,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Note: we don't use the epoch number when generating batches, so we could just have a single `for` loop rather than 2 nested `for` loops, but it's convenient to think of training time in terms of number of epochs (i.e., roughly the number of times the algorithm went through the training set)."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "For each instance in the test set, `y_proba_val` contains the estimated probability that it belongs to the positive class, according to the model. For example, here are the first 5 estimated probabilities:"
    ]
@@ -3093,11 +2469,7 @@
   {
    "cell_type": "code",
    "execution_count": 122,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "y_proba_val[:5]"
@@ -3105,10 +2477,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "To classify each instance, we can go for maximum likelihood: classify as positive any instance whose estimated probability is greater or equal to 0.5:"
    ]
@@ -3116,11 +2485,7 @@
   {
    "cell_type": "code",
    "execution_count": 123,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "y_pred = (y_proba_val >= 0.5)\n",
@@ -3129,20 +2494,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Depending on the use case, you may want to choose a different threshold than 0.5: make it higher if you want high precision (but lower recall), and make it lower if you want high recall (but lower precision). See chapter 3 for more details."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's compute the model's precision and recall:"
    ]
@@ -3150,11 +2509,7 @@
   {
    "cell_type": "code",
    "execution_count": 124,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "from sklearn.metrics import precision_score, recall_score\n",
@@ -3165,11 +2520,7 @@
   {
    "cell_type": "code",
    "execution_count": 125,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "recall_score(y_test, y_pred)"
@@ -3177,10 +2528,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's plot these predictions to see what they look like:"
    ]
@@ -3188,11 +2536,7 @@
   {
    "cell_type": "code",
    "execution_count": 126,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "y_pred_idx = y_pred.reshape(-1) # a 1D array rather than a column vector\n",
@@ -3204,20 +2548,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Well, that looks pretty bad, doesn't it? But let's not forget that the Logistic Regression model has a linear decision boundary, so this is actually close to the best we can do with this model (unless we add more features, as we will show in a second)."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Now let's start over, but this time we will add all the bells and whistles, as listed in the exercise:\n",
     "* Define the graph within a `logistic_regression()` function that can be reused easily.\n",
@@ -3230,10 +2568,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Before we start, we will add 4 more features to the inputs: ${x_1}^2$, ${x_2}^2$, ${x_1}^3$ and ${x_2}^3$. This was not part of the exercise, but it will demonstrate how adding features can improve the model. We will do this manually, but you could also add them using `sklearn.preprocessing.PolynomialFeatures`."
    ]
@@ -3242,9 +2577,7 @@
    "cell_type": "code",
    "execution_count": 127,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3262,10 +2595,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "This is what the \"enhanced\" training set looks like:"
    ]
@@ -3273,11 +2603,7 @@
   {
    "cell_type": "code",
    "execution_count": 128,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "X_train_enhanced[:5]"
@@ -3285,10 +2611,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Ok, next let's reset the default graph:"
    ]
@@ -3297,9 +2620,7 @@
    "cell_type": "code",
    "execution_count": 129,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3308,10 +2629,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Now let's define the `logistic_regression()` function to create the graph. We will leave out the definition of the inputs `X` and the targets `y`. We could include them here, but leaving them out will make it easier to use this function in a wide range of use cases (e.g. perhaps we will want to add some preprocessing steps for the inputs before we feed them to the Logistic Regression model)."
    ]
@@ -3320,9 +2638,7 @@
    "cell_type": "code",
    "execution_count": 130,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3349,10 +2665,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Let's create a little function to get the name of the log directory to save the summaries for Tensorboard:"
    ]
@@ -3361,9 +2674,7 @@
    "cell_type": "code",
    "execution_count": 131,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3380,10 +2691,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Next, let's create the graph, using the `logistic_regression()` function. We will also create the `FileWriter` to save the summaries to the log directory for Tensorboard:"
    ]
@@ -3391,11 +2699,7 @@
   {
    "cell_type": "code",
    "execution_count": 132,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "n_inputs = 2 + 4\n",
@@ -3411,10 +2715,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "At last we can train the model! We will start by checking whether a previous training session was interrupted, and if so we will load the checkpoint and continue training from the epoch number we saved. In this example we just save the epoch number to a separate file, but in chapter 11 we will see how to store the training step directly as part of the model, using a non-trainable variable called `global_step` that we pass to the optimizer's `minimize()` method.\n",
     "\n",
@@ -3424,11 +2725,7 @@
   {
    "cell_type": "code",
    "execution_count": 133,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "n_epochs = 10001\n",
@@ -3469,10 +2766,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Once again, we can make predictions by just classifying as positive all the instances whose estimated probability is greater or equal to 0.5:"
    ]
@@ -3481,9 +2775,7 @@
    "cell_type": "code",
    "execution_count": 134,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": [
@@ -3493,11 +2785,7 @@
   {
    "cell_type": "code",
    "execution_count": 135,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "precision_score(y_test, y_pred)"
@@ -3506,11 +2794,7 @@
   {
    "cell_type": "code",
    "execution_count": 136,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "recall_score(y_test, y_pred)"
@@ -3519,11 +2803,7 @@
   {
    "cell_type": "code",
    "execution_count": 137,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "y_pred_idx = y_pred.reshape(-1) # a 1D array rather than a column vector\n",
@@ -3535,20 +2815,14 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Now that's much, much better! Apparently the new features really helped a lot."
    ]
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Try starting the tensorboard server, find the latest run and look at the learning curve (i.e., how the loss evaluated on the test set evolves as a function of the epoch number):\n",
     "\n",
@@ -3559,10 +2833,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "Now you can play around with the hyperparameters (e.g. the `batch_size` or the `learning_rate`) and run training again and again, comparing the learning curves. You can even automate this process by implementing grid search or randomized search. Below is a simple implementation of a randomized search on both the batch size and the learning rate. For the sake of simplicity, the checkpoint mechanism was removed."
    ]
@@ -3570,11 +2841,7 @@
   {
    "cell_type": "code",
    "execution_count": 138,
-   "metadata": {
-    "collapsed": false,
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "outputs": [],
    "source": [
     "from scipy.stats import reciprocal\n",
@@ -3633,10 +2900,7 @@
   },
   {
    "cell_type": "markdown",
-   "metadata": {
-    "deletable": true,
-    "editable": true
-   },
+   "metadata": {},
    "source": [
     "The `reciprocal()` function from SciPy's `stats` module returns a random distribution that is commonly used when you have no idea of the optimal scale of a hyperparameter. See the exercise solutions for chapter 2 for more details. "
    ]
@@ -3645,9 +2909,7 @@
    "cell_type": "code",
    "execution_count": null,
    "metadata": {
-    "collapsed": true,
-    "deletable": true,
-    "editable": true
+    "collapsed": true
    },
    "outputs": [],
    "source": []
@@ -3669,7 +2931,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.5.3"
+   "version": "3.6.5"
   },
   "nav_menu": {
    "height": "603px",
@@ -3686,5 +2948,5 @@
   }
  },
  "nbformat": 4,
- "nbformat_minor": 0
+ "nbformat_minor": 1
 }