cs501r_f2016:lab6
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
cs501r_f2016:lab6 [2016/09/24 20:59] wingated |
cs501r_f2016:lab6 [2016/09/26 23:33] wingated |
||
|---|---|---|---|
| Line 22: | Line 22: | ||
| An example of my training/test performance for dropout is shown at the right. | An example of my training/test performance for dropout is shown at the right. | ||
| - | **NOTE**: because this lab can be more computationally time consuming than the others (since we're scanning across parameters), you are welcome to turn in your plots and your code separately. (This means, for example, that you can develop and run all of your code using an IDE other than the Jupyter notebook, collect the data, and then run a separate little script to generate the plots. Or, a particularly enterprising student may use his or her new supercomputer account to sweep all of the parameter values in parallel (!) ). You will need to zip up your images and code into a single file for submission to Learning Suite. | + | **NOTE**: because this lab can be more computationally time consuming than the others (since we're scanning across parameters), you are welcome to turn in your plots and your code separately. (This means, for example, that you can develop and run all of your code using an IDE other than the Jupyter notebook, collect the data, and then run a separate little script to generate the plots. Or, a particularly enterprising student may use his or her new supercomputer account to sweep all of the parameter values in parallel (!) ). If you do this, you will need to zip up your images and code into a single file for submission to Learning Suite. |
| ---- | ---- | ||
| Line 37: | Line 37: | ||
| ====Description:==== | ====Description:==== | ||
| - | This lab is a chance for you to start reading the literature on deep neural networks, and understand how to replicate methods from the literature. You will implement 4 different regularization methods, and will benchmark each one. | + | This lab is a chance for you to start reading the literature on deep neural networks, and understand how to replicate methods from the literature. You will implement 3 different regularization methods, and will benchmark each one. |
| To help ensure that everyone is starting off on the same footing, you should download the following scaffold code: | To help ensure that everyone is starting off on the same footing, you should download the following scaffold code: | ||
| - | [[http://liftothers.org/byu/lab6_scaffold.py|Lab 6 scaffold code]] | + | [[http://liftothers.org/byu/lab6_scaffold.py|Lab 6 scaffold code (UPDATED WITH RELUs)]] |
| For all 3 methods, we will run on a single, deterministic batch of the first 1000 images from the MNIST dataset. This will help us to overfit, and will hopefully be small enough not to tax your computers too much. | For all 3 methods, we will run on a single, deterministic batch of the first 1000 images from the MNIST dataset. This will help us to overfit, and will hopefully be small enough not to tax your computers too much. | ||
| Line 97: | Line 97: | ||
| Note that you should **not** call your regularization variable "lambda" because that is a reserved keyword in python. | Note that you should **not** call your regularization variable "lambda" because that is a reserved keyword in python. | ||
| + | |||
| + | Remember that the "masks" for both dropout and dropconnect change for **every** step in training. | ||
cs501r_f2016/lab6.txt ยท Last modified: 2021/06/30 23:42 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
