cs601r_w2020:lab2
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cs601r_w2020:lab2 [2020/01/10 17:49] wingated |
cs601r_w2020:lab2 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 13: | Line 13: | ||
| Your notebook will be graded on the following: | Your notebook will be graded on the following: | ||
| - | * 40% Part 1: Clearly displayed 10 bars | + | * 35% Part 1: Clearly displayed 10 bars (one for baseline, one for each tweak independently) |
| + | * 5% Part 1: Small writeup of conclusions from independent tweaks | ||
| * 25% Part 2: Clear explanation of your tweaking strategy | * 25% Part 2: Clear explanation of your tweaking strategy | ||
| * 25% Part 2: Actually run your tweaking strategy and show the results | * 25% Part 2: Actually run your tweaking strategy and show the results | ||
| * 10% Tidy and legible figures, including labeled axes where appropriate | * 10% Tidy and legible figures, including labeled axes where appropriate | ||
| + | * 10% Extra credit - Error bars on your figure in Part 1. | ||
| ---- | ---- | ||
| Line 27: | Line 29: | ||
| For this lab, you will explore various tweaks to the basic classifier you coded in lab 1. There are two parts to the lab. | For this lab, you will explore various tweaks to the basic classifier you coded in lab 1. There are two parts to the lab. | ||
| - | **Part 1:** You must clearly show the individual effect of each tweak compared to the baseline. For this part, you should present a simple bar chart (or possibly two or more, depending on your layout), clearly labeled with the baseline performance, and then the performance of each tweak relative to baseline. You may plot absolute or relative performances; whichever is clearer. | + | ---- |
| + | ====Part 1==== | ||
| - | **Note:** I am not requiring error bars for this lab, although if we were doing this for real, they would be absolutely essential! | + | You must clearly show the individual effect of each tweak compared to the baseline. For this part, you should present a simple bar chart (or possibly two or more, depending on your layout), clearly labeled with the baseline performance, and then the performance of each tweak relative to baseline. You may plot absolute or relative performances; whichever is clearer. |
| - | **Part 2:** You must think about how to find the best combination of tweaks. There is no right answer to this part; I want you to think carefully about how to search the space of possible combinations, and come up with a reasonable method for settling on a final combination of tweaks. I have tried to provide enough tweaks that it should be impossible to brute-force try all possible combinations (although that is certainly a valid strategy!). | + | You must include a few sentences describing what you can conclude from evaluating all of these tweaks. |
| + | |||
| + | **Note:** I am not requiring error bars for this lab, because they are computationally intensive. I have made them extra credit -- although if we were doing this for real, they would be absolutely required! | ||
| + | |||
| + | ---- | ||
| + | ====Part 2==== | ||
| + | |||
| + | You must think about how to find the best combination of tweaks. There is no right answer to this part; I want you to think carefully about how to search the space of possible combinations, and come up with a reasonable method for settling on a final combination of tweaks. I have tried to provide enough tweaks that it should be impossible to brute-force try all possible combinations (although that is certainly a valid strategy!). | ||
| For this part, you must include in your notebook a simple writeup describing your strategy (just a paragraph or two), and then show the final performance of whatever combination you hit upon. | For this part, you must include in your notebook a simple writeup describing your strategy (just a paragraph or two), and then show the final performance of whatever combination you hit upon. | ||
| Line 37: | Line 47: | ||
| Note that you will not be graded on absolute performance of any run; what is important is thinking clearly through which tweaks make a difference. | Note that you will not be graded on absolute performance of any run; what is important is thinking clearly through which tweaks make a difference. | ||
| - | **The Tweaks** | + | ---- |
| + | ====The Tweaks==== | ||
| Your baseline classifier must be a "vanilla" classifier, with none of the features listed below. We will systematically add them in. | Your baseline classifier must be a "vanilla" classifier, with none of the features listed below. We will systematically add them in. | ||
| Line 53: | Line 64: | ||
| Some of these tweaks require additional parameters. You should either leave them at their default values, or think of some reasonable way to set them. | Some of these tweaks require additional parameters. You should either leave them at their default values, or think of some reasonable way to set them. | ||
| + | |||
| + | Note: pytorch does not (AFAIK) natively implement label smoothing. In the interests of focusing on hyperparameter searching, **you may verbatim copy any internet code you like to help implement label smoothing.** | ||
| ---- | ---- | ||
| - | ====Hints:==== | + | ====Hints==== |
| Activation functions and dropout can all be found in torch.nn | Activation functions and dropout can all be found in torch.nn | ||
| Line 63: | Line 76: | ||
| This lab should be pretty straightforward, with the right script -- you should be able to iterate over tweaks and run your classifier in a tidy loop. Ideally, you'll code it up, let it run, and come back in a few hours to find the results! | This lab should be pretty straightforward, with the right script -- you should be able to iterate over tweaks and run your classifier in a tidy loop. Ideally, you'll code it up, let it run, and come back in a few hours to find the results! | ||
| - | If you find yourself cutting-and-pasting, you might want to rethink your strategy! | + | If you find yourself cutting-and-pasting, you might want to rethink your strategy. |
cs601r_w2020/lab2.1578678580.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
