cs501r_f2018:lab7
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cs501r_f2018:lab7 [2018/10/15 16:06] carr |
cs501r_f2018:lab7 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 23: | Line 23: | ||
| * 50% Read and Study the annotation found in the Harvard notebook (on your honor) | * 50% Read and Study the annotation found in the Harvard notebook (on your honor) | ||
| - | * 20% Clean, transform, load, and train on provided General Conference NMT dataset | + | * 40% Clean, transform, load, and train on provided General Conference NMT dataset |
| - | * 20% Try 1, 2, 4, 6 layers for both encoder and decoder pieces, report results in a few short paragraphs | + | |
| * 10% Good coding style, readable output | * 10% Good coding style, readable output | ||
| + | * 20% EXTRA CREDIT Try 1, 2, 4, 6 layers for both encoder and decoder pieces, report results in a few short paragraphs | ||
| ---- | ---- | ||
| Line 31: | Line 31: | ||
| For this lab, you will modify the | For this lab, you will modify the | ||
| - | [[https://github.com/harvardnlp/annotated-transformer/blob/master/The%20Annotated%20Transformer.ipynb|Annotated Transformer]]. There is link to a coloab notebook in the jupyter notebook that you can use. The code is slightly different between the notebook linked above, and the colab link provided by Harvard. Both will work, you may very likely need to mix and match pieces from each to get a working implementation. While this may feel slightly frustrating, it is good practice for deep learning research strategies. | + | [[https://github.com/harvardnlp/annotated-transformer/blob/master/The%20Annotated%20Transformer.ipynb|Annotated Transformer]]. |
| + | |||
| + | **There is a link to a coloab notebook in the jupyter notebook that you can use.** | ||
| + | |||
| + | |||
| + | |||
| + | The code is slightly different between the notebook linked above, and the colab link provided by Harvard. Both will work, you may very likely need to mix and match pieces from each to get a working implementation. While this may feel slightly frustrating, it is good practice for deep learning research strategies. | ||
| + | |||
| + | If you are experiencing difficulties, you may need to install specific versions of the necessary packages. One student contributed this: | ||
| + | |||
| + | <code python> | ||
| + | !pip install http://download.pytorch.org/whl/cu80/torch-0.3.0.post4-cp36-cp36m-linux_x86_64.whl numpy matplotlib spacy torchtext==0.2.3 seaborn | ||
| + | </code> | ||
| Often when implementing a novel deep learning method, you will start by using someone's implementation as a reference. This is an extremely valuable, and potentially time-saving skill, for producing workable solutions to many problems solved by deep learning methods. | Often when implementing a novel deep learning method, you will start by using someone's implementation as a reference. This is an extremely valuable, and potentially time-saving skill, for producing workable solutions to many problems solved by deep learning methods. | ||
| Line 57: | Line 70: | ||
| You should expect to see reasonable results after 2-4 hours of training in colab. | You should expect to see reasonable results after 2-4 hours of training in colab. | ||
| + | |||
| + | [[http://mlexplained.com/2018/02/08/a-comprehensive-tutorial-to-torchtext/|Here is a good tutorial on torchtext.]] | ||
| ---- | ---- | ||
| - | **Part 3: Experiment with a different number of stacked layers in the encoder and decoder** | + | **Part 3: EXTRA CREDIT: Experiment with a different number of stacked layers in the encoder and decoder** |
| Now it's time to put on your scientist hats. Try stacking a different number of layers (e.g., 1, 2, 4) for the encoder and decoder. This will require you to understand their implementation and be able to work with it reliably. | Now it's time to put on your scientist hats. Try stacking a different number of layers (e.g., 1, 2, 4) for the encoder and decoder. This will require you to understand their implementation and be able to work with it reliably. | ||
cs501r_f2018/lab7.1539619586.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
