cs501r_f2017:lab5v2
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cs501r_f2017:lab5v2 [2017/09/28 16:40] wingated |
cs501r_f2017:lab5v2 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 6: | Line 6: | ||
| ====Deliverable:==== | ====Deliverable:==== | ||
| - | For this lab, you will need to perform three steps: | + | There are two parts to this lab: |
| - You need to implement several helper functions | - You need to implement several helper functions | ||
| - You need to create and train your own DNN | - You need to create and train your own DNN | ||
| - | |||
| You should turn in an iPython notebook that shows a tensorboard screenshot of your classifier's computation graph, as well as a of visualization of classification accuracy (on a held-out test set) going up over time. | You should turn in an iPython notebook that shows a tensorboard screenshot of your classifier's computation graph, as well as a of visualization of classification accuracy (on a held-out test set) going up over time. | ||
| Line 66: | Line 64: | ||
| <code python> | <code python> | ||
| - | def fc( x, out_size=50, name="fc" ): | + | def fc( x, out_size=50, is_output=False, name="fc" ): |
| ''' | ''' | ||
| x is an input tensor | x is an input tensor | ||
| Line 106: | Line 104: | ||
| - This graph should have at least two convolution layers and two fully connected layers. | - This graph should have at least two convolution layers and two fully connected layers. | ||
| - You may pick the number of filters in each convolution layer, and the size of the fully connected layers. Typically, there are about 64 filters in a convolution layer, and about 256 neurons in the first fully connected layer and 64 in the second. | - You may pick the number of filters in each convolution layer, and the size of the fully connected layers. Typically, there are about 64 filters in a convolution layer, and about 256 neurons in the first fully connected layer and 64 in the second. | ||
| - | - You should use the cross entropy loss function. | + | - You should use the cross entropy loss function. I implemented this using ''tf.nn.sparse_softmax_cross_entropy_with_logits''. Check the documentation for details. |
| - Train the network using an optimizer of your choice | - Train the network using an optimizer of your choice | ||
| - You might as well use the Adam optimizer | - You might as well use the Adam optimizer | ||
| - | For now, just assume a batch size of 1. | + | For now, you are welcome to assume a batch size of 1, but if you're feeling adventurous, see if you can code your computation graph to support arbitrarily large batches. |
| - | You input placeholders will need to be four dimensional -- ie, [1,32,32,3]. This is required by the convolution function. | + | You input placeholders will need to be four dimensional -- ie, [1,32,32,3]. This is required by the convolution function. (The "1" is the batch size) |
| It is a little bit tricky to transition from the convolution layers to the fully connected layers. The most common way to accomplish this is to reshape the tensor, using ''tf.reshape'' to "flatten" a tensor. | It is a little bit tricky to transition from the convolution layers to the fully connected layers. The most common way to accomplish this is to reshape the tensor, using ''tf.reshape'' to "flatten" a tensor. | ||
| Line 124: | Line 122: | ||
| * Use ''tf.nn.relu'' to create a ''relu'' layer. | * Use ''tf.nn.relu'' to create a ''relu'' layer. | ||
| - | * **Variable initialization matters**. If your classifier seems stuck at 10% or 11% accuracy, make sure you're not initializing to all zeros! I usually initialize variables by drawing entries from a Gaussian with a small standard deviation: ''W = tf.Variable( tf.random_normal( [784, 100], stddev=0.001 ), name="W" )'' | + | * **Variable initialization matters**. If your classifier seems stuck at 10% or 11% accuracy, make sure you're not initializing to all zeros! |
| + | * If you're having trouble debugging your DNN, first make sure that you can overfit on one image -- you should be able to achieve a loss of very close to 0. | ||
| + | * You will need to specify a learning rate. This is likely topology dependent. Try values like 0.001, 0.01, or 0.1. | ||
| + | |||
| + | The CIFAR-10 dataset can be downloaded at | ||
| + | [[https://www.cs.toronto.edu/~kriz/cifar.html|https://www.cs.toronto.edu/~kriz/cifar.html]] | ||
| + | |||
| + | **Note: make sure to download the python version of the data - it will simplify your life!** | ||
cs501r_f2017/lab5v2.1506616832.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
