To become more proficient in Tensorflow, to become more proficient in the construction of computation graphs and image classification, and to create and debug your first full DNN.
There are two parts to this lab:
You should turn in an iPython notebook that shows a tensorboard screenshot of your classifier's computation graph, as well as a of visualization of classification accuracy (on a held-out test set) going up over time.
An example plot is shown to the right.
NOTE: You are welcome to look at code on the internet to help with this lab, including the tensorflow tutorials and code developed in class, but all code that you turn in must be your own work!
Your notebook will be graded on the following:
conv
and fc
helper functionsYou now have all of the tools you need to become a real deep learning ninja – you understand the basics of vectorized code, computation graphs, automatic differentiation, convolutions, and optimization. We're now going to put all of those pieces together!
There are two parts to this lab.
Part 1: Create helper functions
To simplify your life now and in the future, your first task is to create two helper functions that will be used to create convolution and fully connected layers. The API for these two functions, along with a description of what they're supposed to do, is provided below.
The challenge of this part of the lab is ensuring that sizes of everything matches, and that variables are properly initialized. To do that, you will need to examine the size of the input tensor x
. I used code like x.get_shape().as_list()
to get a list of dimensions.
You may also find the tf.nn.bias_add
and tf.get_variable
functions helpful.
If you use the tf.get_variable
method of creating and initializing variables, you can specify an optional initializer for the variable. I used tf.contrib.layers.variance_scaling_initializer()
with good results.
Make sure that when you call tf.nn.conv2d
, you use the padding=“SAME”
option!
def conv( x, filter_size=3, stride=2, num_filters=64, is_output=False, name="conv" ): ''' x is an input tensor Declare a name scope using the "name" parameter Within that scope: Create a W filter variable with the proper size Create a B bias variable with the proper size Convolve x with W by calling the tf.nn.conv2d function Add the bias If is_output is False, Call the tf.nn.relu function Return the final op ''' pass
def fc( x, out_size=50, is_output=False, name="fc" ): ''' x is an input tensor Declare a name scope using the "name" parameter Within that scope: Create a W filter variable with the proper size Create a B bias variable with the proper size Multiply x by W and add b If is_output is False, Call the tf.nn.relu function Return the final op ''' pass
Given these helper functions, we can define a DNN as follows:
input_data = tf.placeholder( tf.float32, [1,32,32,3] ) h0 = conv( input_data, name="h0" ) h1 = conv( h0, name="h1" ) h2 = conv( h1, name="h2" ) ...
etc.
Part 2: Implement a DNN on the CIFAR-10 dataset
This is it! Now it's time to implement your first DNN. Using the helper functions defined previously, you now need to implement an image classifier, end-to-end.
The steps for completion of this lab are:
conv
and fc
layers you defined previously, create a computation graph to classify images.tf.nn.sparse_softmax_cross_entropy_with_logits
. Check the documentation for details.For now, you are welcome to assume a batch size of 1, but if you're feeling adventurous, see if you can code your computation graph to support arbitrarily large batches.
You input placeholders will need to be four dimensional – ie, [1,32,32,3]. This is required by the convolution function. (The “1” is the batch size)
It is a little bit tricky to transition from the convolution layers to the fully connected layers. The most common way to accomplish this is to reshape the tensor, using tf.reshape
to “flatten” a tensor.
You are welcome (and encouraged!) to see what happens as you add more and more layers!
The Tensorflow documentation is quite helpful. A few things that you might need:
tf.nn.relu
to create a relu
layer.The CIFAR-10 dataset can be downloaded at https://www.cs.toronto.edu/~kriz/cifar.html
Note: make sure to download the python version of the data - it will simplify your life!