cs501r_f2016:tmp
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision | Next revision Both sides next revision | ||
|
cs501r_f2016:tmp [2016/09/24 20:31] wingated |
cs501r_f2016:tmp [2016/09/24 20:36] wingated |
||
|---|---|---|---|
| Line 36: | Line 36: | ||
| This lab is a chance for you to start reading the literature on deep neural networks, and understand how to replicate methods from the literature. You will implement 4 different regularization methods, and will benchmark each one. | This lab is a chance for you to start reading the literature on deep neural networks, and understand how to replicate methods from the literature. You will implement 4 different regularization methods, and will benchmark each one. | ||
| - | |||
| - | Please note tat | ||
| To help ensure that everyone is starting off on the same footing, you should download the following scaffold code: | To help ensure that everyone is starting off on the same footing, you should download the following scaffold code: | ||
| - | **For all parts** | ||
| - | For all 4 methods, we will run on a single, deterministic batch of the first 1000 images from the MNIST dataset. This will help us to | ||
| + | For all 4 methods, we will run on a single, deterministic batch of the first 1000 images from the MNIST dataset. This will help us to overfit, and will hopefully be small enough not to tax your computers too much. | ||
| **Part 1: implement dropout** | **Part 1: implement dropout** | ||
| Line 56: | Line 53: | ||
| There are several notes to help you with this part: | There are several notes to help you with this part: | ||
| - | - First, you should run the provided code as-is. It will overfit on the first 1000 images (how do you know this?). Record the accuracy of the | + | - First, you should run the provided code as-is. It will overfit on the first 1000 images (how do you know this?). Record the test and training accuracy; this will be the "baseline" line in your plot. |
| - Second, you should add dropout to each of the ''h1'', ''h2'', and ''h3'' layers. | - Second, you should add dropout to each of the ''h1'', ''h2'', and ''h3'' layers. | ||
| - You must consider carefully how to use tensorflow to implement dropout. | - You must consider carefully how to use tensorflow to implement dropout. | ||
| - | - Remember that you must scale activations by the ''keep_probability'', as discussed in class and in the paper. | + | - Remember that when you test images (or when you compute training set accuracy), you must scale activations by the ''keep_probability'', as discussed in class and in the paper. |
| - You should use the Adam optimizer, and optimize for 150 steps. | - You should use the Adam optimizer, and optimize for 150 steps. | ||
cs501r_f2016/tmp.txt ยท Last modified: 2021/06/30 23:42 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
