To install and learn the basics of Tensorflow, and to become more proficient in the construction of computation graphs and image classification.
For this lab, you will need to perform three steps:
You should turn in an iPython notebook that shows your modified classifier, as well as some sort of visualization of the performance of your modified classifier; this could be a simple matplotlib plot show classification accuracy, or a Tensorboard screenshot.
An example plot is shown to the right.
Your notebook will be graded on the following:
You now have all of the tools you need to become a real deep learning ninja – you understand the basics of vectorized code, computation graphs, automatic differentiation, and optimization. Tensorflow lets you put all of those pieces together!
There are three parts to this lab.
Part 1: install Tensorflow
The main point of this lab is to get Tensorflow installed and working. To do that, please follow the installation instructions on the Tensorflow.org website. Note that there are several different installation methods; I have had good luck using pip.
Note: when using pip
to install python packages, make sure that you're using the anaconda version of pip
(as opposed to any pip
programs that are part of your system distro)! You can always tell which version you're running by running which pip
in a terminal.
Also note that there are two versions of Tensorflow – one that runs on GPUs, and one that runs only on the CPU. You may want to try the GPU version first; if it works (and you have a GPU in your computer!) it may be considerably faster than the CPU only version. Performance won't be a big deal for this lab, but it will matter more later on.
A few other notes:
anaconda
install instructions, since that's what we're using in class. However, I have found that the creation of virtual environments complicates life, and doesn't seem to work well with the notebooks we've been using. A much simpler way is just to use the pip
installation method.Part 2: implement basic MNIST tutorial
For this part of the lab, all you have to do is read through (and understand!) the MNIST tutorial on the Tensorflow website. There are several snippets of code; you should put all of these together into a single program, and make sure you can run it.
You may want to plot classification accuracy over time to debug your model. My classification curve is shown on the right; note that this accuracy is on the test set, not the training set. My accuracy got up to about 90%.
Part 3: modifying the basic MNIST classifier
Now that you have read through the MNIST classifier tutorial, and you understand the basics of variables, placeholders, and computation graphs, you must modify the basic classifier to improve classification accuracy. To do this, you must move from a single-layer classifier to a two-layer classifier.
You will need to create new variables for this new layer, and you will need to modify your computation graph accordingly. Remember that you will need some sort of nonlinearity between layers – I recommend using a relu
layer.
Note that you will need to pick the size of the hidden layer. Try different values, and see what works.
Adding a second layer, adjusting my initialization, changing my step size to 0.05, and running for 2000 epochs, I was able to achieve 92% classification accuracy. Using a larger steps size (0.1) allowed me to get to 94% accuracy. My new classification curve is shown in the first section of this document.
You are welcome (and encouraged!) to see what happens as you add more and more layers!
The Tensorflow documentation is quite helpful. A few things that you might need:
tf.nn.relu
to create a relu
layer.W = tf.Variable( tf.random_normal( [784, 100], stddev=0.001 ), name=“W” )