To become more proficient in Tensorflow, to become more proficient in the construction of computation graphs and image classification, and to create and debug your first full DNN.
There are two parts to this lab:
You should turn in an iPython notebook that shows a tensorboard screenshot of your classifier's computation graph, as well as a of visualization of classification accuracy (on a held-out test set) going up over time.
An example plot is shown to the right.
NOTE: You are welcome to look at code on the internet to help with this lab, including the tensorflow tutorials and code developed in class, but all code that you turn in must be your own work!
Your notebook will be graded on the following:
You now have all of the tools you need to become a real deep learning ninja – you understand the basics of vectorized code, computation graphs, automatic differentiation, convolutions, and optimization. We're now going to put all of those pieces together!
There are two parts to this lab.
Part 1: Create helper functions
To simplify your life now and in the future, your first task is to create two helper functions that will be used to create convolution and fully connected layers. The API for these two functions, along with a description of what they're supposed to do, is provided below.
The challenge of this part of the lab is ensuring that sizes of everything matches, and that variables are properly initialized. To do that, you will need to examine the size of the input tensor
x. I used code like
x.get_shape().as_list() to get a list of dimensions.
You may also find the
tf.get_variable functions helpful.
If you use the
tf.get_variable method of creating and initializing variables, you can specify an optional initializer for the variable. I used
tf.contrib.layers.variance_scaling_initializer() with good results.
Make sure that when you call
tf.nn.conv2d, you use the
def conv( x, filter_size=3, stride=2, num_filters=64, is_output=False, name="conv" ): ''' x is an input tensor Declare a name scope using the "name" parameter Within that scope: Create a W filter variable with the proper size Create a B bias variable with the proper size Convolve x with W by calling the tf.nn.conv2d function Add the bias If is_output is False, Call the tf.nn.relu function Return the final op ''' pass
def fc( x, out_size=50, is_output=False, name="fc" ): ''' x is an input tensor Declare a name scope using the "name" parameter Within that scope: Create a W filter variable with the proper size Create a B bias variable with the proper size Multiply x by W and add b If is_output is False, Call the tf.nn.relu function Return the final op ''' pass
Given these helper functions, we can define a DNN as follows:
input_data = tf.placeholder( tf.float32, [1,32,32,3] ) h0 = conv( input_data, name="h0" ) h1 = conv( h0, name="h1" ) h2 = conv( h1, name="h2" ) ...
Part 2: Implement a DNN on the CIFAR-10 dataset
This is it! Now it's time to implement your first DNN. Using the helper functions defined previously, you now need to implement an image classifier, end-to-end.
The steps for completion of this lab are:
fclayers you defined previously, create a computation graph to classify images.
tf.nn.sparse_softmax_cross_entropy_with_logits. Check the documentation for details.
For now, you are welcome to assume a batch size of 1, but if you're feeling adventurous, see if you can code your computation graph to support arbitrarily large batches.
You input placeholders will need to be four dimensional – ie, [1,32,32,3]. This is required by the convolution function. (The “1” is the batch size)
It is a little bit tricky to transition from the convolution layers to the fully connected layers. The most common way to accomplish this is to reshape the tensor, using
tf.reshape to “flatten” a tensor.
You are welcome (and encouraged!) to see what happens as you add more and more layers!
The Tensorflow documentation is quite helpful. A few things that you might need:
tf.nn.reluto create a
The CIFAR-10 dataset can be downloaded at https://www.cs.toronto.edu/~kriz/cifar.html
Note: make sure to download the python version of the data - it will simplify your life!