This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
cs501r_f2017:lab5v2 [2017/09/28 16:36] wingated created |
cs501r_f2017:lab5v2 [2021/06/30 23:42] (current) |
||
---|---|---|---|
Line 6: | Line 6: | ||
====Deliverable:==== | ====Deliverable:==== | ||
- | For this lab, you will need to perform three steps: | + | There are two parts to this lab: |
- You need to implement several helper functions | - You need to implement several helper functions | ||
- You need to create and train your own DNN | - You need to create and train your own DNN | ||
- | |||
You should turn in an iPython notebook that shows a tensorboard screenshot of your classifier's computation graph, as well as a of visualization of classification accuracy (on a held-out test set) going up over time. | You should turn in an iPython notebook that shows a tensorboard screenshot of your classifier's computation graph, as well as a of visualization of classification accuracy (on a held-out test set) going up over time. | ||
Line 66: | Line 64: | ||
<code python> | <code python> | ||
- | def fc( x, out_size=50, name="fc" ): | + | def fc( x, out_size=50, is_output=False, name="fc" ): |
''' | ''' | ||
x is an input tensor | x is an input tensor | ||
Line 104: | Line 102: | ||
- Use tensorflow to create your DNN. | - Use tensorflow to create your DNN. | ||
- Using the ''conv'' and ''fc'' layers you defined previously, create a computation graph to classify images. | - Using the ''conv'' and ''fc'' layers you defined previously, create a computation graph to classify images. | ||
- | - This graph should have at least two convolution layers and two fully connection layers. | + | - This graph should have at least two convolution layers and two fully connected layers. |
- | - You may pick the number of filters in each convolution layer, and the size of the fully connected layer. | + | - You may pick the number of filters in each convolution layer, and the size of the fully connected layers. Typically, there are about 64 filters in a convolution layer, and about 256 neurons in the first fully connected layer and 64 in the second. |
+ | - You should use the cross entropy loss function. I implemented this using ''tf.nn.sparse_softmax_cross_entropy_with_logits''. Check the documentation for details. | ||
- Train the network using an optimizer of your choice | - Train the network using an optimizer of your choice | ||
- You might as well use the Adam optimizer | - You might as well use the Adam optimizer | ||
- | For now, just assume a batch size of 1. | + | For now, you are welcome to assume a batch size of 1, but if you're feeling adventurous, see if you can code your computation graph to support arbitrarily large batches. |
- | You input placeholders will need to be four dimensional -- ie, [1,32,32,3]. This is required by the convolution function. | + | You input placeholders will need to be four dimensional -- ie, [1,32,32,3]. This is required by the convolution function. (The "1" is the batch size) |
It is a little bit tricky to transition from the convolution layers to the fully connected layers. The most common way to accomplish this is to reshape the tensor, using ''tf.reshape'' to "flatten" a tensor. | It is a little bit tricky to transition from the convolution layers to the fully connected layers. The most common way to accomplish this is to reshape the tensor, using ''tf.reshape'' to "flatten" a tensor. | ||
Line 123: | Line 122: | ||
* Use ''tf.nn.relu'' to create a ''relu'' layer. | * Use ''tf.nn.relu'' to create a ''relu'' layer. | ||
- | * **Variable initialization matters**. If your classifier seems stuck at 10% or 11% accuracy, make sure you're not initializing to all zeros! I usually initialize variables by drawing entries from a Gaussian with a small standard deviation: ''W = tf.Variable( tf.random_normal( [784, 100], stddev=0.001 ), name="W" )'' | + | * **Variable initialization matters**. If your classifier seems stuck at 10% or 11% accuracy, make sure you're not initializing to all zeros! |
+ | * If you're having trouble debugging your DNN, first make sure that you can overfit on one image -- you should be able to achieve a loss of very close to 0. | ||
+ | * You will need to specify a learning rate. This is likely topology dependent. Try values like 0.001, 0.01, or 0.1. | ||
+ | |||
+ | The CIFAR-10 dataset can be downloaded at | ||
+ | [[https://www.cs.toronto.edu/~kriz/cifar.html|https://www.cs.toronto.edu/~kriz/cifar.html]] | ||
+ | |||
+ | **Note: make sure to download the python version of the data - it will simplify your life!** | ||