User Tools

Site Tools


cs501r_f2016:lab5

Objective:

To install and learn the basics of Tensorflow, and to become more proficient in the construction of computation graphs and image classification.


Deliverable:

For this lab, you will need to perform three steps:

  1. You need to Install Tensorflow
  2. You need to implement the MNIST for beginners tutorial
  3. You need to modify the tutorial code to improve classification performance by adding more layers. Your final classifier should have > 90% accuracy on the MNIST test set!

You should turn in an iPython notebook that shows your modified classifier, as well as some sort of visualization of the performance of your modified classifier; this could be a simple matplotlib plot show classification accuracy, or a Tensorboard screenshot.

An example plot is shown to the right.


Grading standards:

Your notebook will be graded on the following:

  • 40% Tensorflow installed and working
  • 50% Modified classifier to improve accuracy
  • 10% Tidy and legible plot of classification accuracy over time

Description:

You now have all of the tools you need to become a real deep learning ninja – you understand the basics of vectorized code, computation graphs, automatic differentiation, and optimization. Tensorflow lets you put all of those pieces together!

There are three parts to this lab.

Part 1: install Tensorflow

The main point of this lab is to get Tensorflow installed and working. To do that, please follow the installation instructions on the Tensorflow.org website. Note that there are several different installation methods; I have had good luck using pip.

Note: when using pip to install python packages, make sure that you're using the anaconda version of pip (as opposed to any pip programs that are part of your system distro)! You can always tell which version you're running by running which pip in a terminal.

Also note that there are two versions of Tensorflow – one that runs on GPUs, and one that runs only on the CPU. You may want to try the GPU version first; if it works (and you have a GPU in your computer!) it may be considerably faster than the CPU only version. Performance won't be a big deal for this lab, but it will matter more later on.

A few other notes:

  1. The GPU version of Tensorflow has considerably more external dependencies - you will need to install several Nvidia packages to support it. There be dragons there.
  2. It might seem natural to use the anaconda install instructions, since that's what we're using in class. However, I have found that the creation of virtual environments complicates life, and doesn't seem to work well with the notebooks we've been using. A much simpler way is just to use the pip installation method.

Part 2: implement basic MNIST tutorial

For this part of the lab, all you have to do is read through (and understand!) the MNIST tutorial on the Tensorflow website. There are several snippets of code; you should put all of these together into a single program, and make sure you can run it.

You may want to plot classification accuracy over time to debug your model. My classification curve is shown on the right; note that this accuracy is on the test set, not the training set. My accuracy got up to about 90%.

Part 3: modifying the basic MNIST classifier

Now that you have read through the MNIST classifier tutorial, and you understand the basics of variables, placeholders, and computation graphs, you must modify the basic classifier to improve classification accuracy. To do this, you must move from a single-layer classifier to a two-layer classifier.

You will need to create new variables for this new layer, and you will need to modify your computation graph accordingly. Remember that you will need some sort of nonlinearity between layers – I recommend using a relu layer.

Note that you will need to pick the size of the hidden layer. Try different values, and see what works.

Adding a second layer, adjusting my initialization, changing my step size to 0.05, and running for 2000 epochs, I was able to achieve 92% classification accuracy. Using a larger steps size (0.1) allowed me to get to 94% accuracy. My new classification curve is shown in the first section of this document.

You are welcome (and encouraged!) to see what happens as you add more and more layers!


Hints:

The Tensorflow documentation is quite helpful. A few things that you might need:

  • Use tf.nn.relu to create a relu layer.
  • Variable initialization matters. If your classifier seems stuck at 10% or 11% accuracy, make sure you're not initializing to all zeros! I usually initialize variables by drawing entries from a Gaussian with a small standard deviation: W = tf.Variable( tf.random_normal( [784, 100], stddev=0.001 ), name=“W” )
cs501r_f2016/lab5.txt · Last modified: 2016/09/19 08:48 by wingated