cs501r_f2018:lab7
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
cs501r_f2018:lab7 [2018/10/15 16:05] carr |
cs501r_f2018:lab7 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 23: | Line 23: | ||
| * 50% Read and Study the annotation found in the Harvard notebook (on your honor) | * 50% Read and Study the annotation found in the Harvard notebook (on your honor) | ||
| - | * 20% Clean, transform, load, and train on provided General Conference NMT dataset | + | * 40% Clean, transform, load, and train on provided General Conference NMT dataset |
| - | * 20% Try 1, 2, 4, 6 layers for both encoder and decoder pieces, report results in a few short paragraphs | + | |
| * 10% Good coding style, readable output | * 10% Good coding style, readable output | ||
| + | * 20% EXTRA CREDIT Try 1, 2, 4, 6 layers for both encoder and decoder pieces, report results in a few short paragraphs | ||
| ---- | ---- | ||
| Line 31: | Line 31: | ||
| For this lab, you will modify the | For this lab, you will modify the | ||
| - | [[https://github.com/harvardnlp/annotated-transformer/blob/master/The%20Annotated%20Transformer.ipynb|Annotated Transformer]]. There is link to a coloab notebook in the jupyter notebook that you can use. The code is slightly different between the notebook linked above, and the colab link provided by Harvard. Both will work, you may very likely need to mix and match pieces from each to get a working implementation. While this may feel slightly frustrating, it is good practice for deep learning research strategies. | + | [[https://github.com/harvardnlp/annotated-transformer/blob/master/The%20Annotated%20Transformer.ipynb|Annotated Transformer]]. |
| + | |||
| + | **There is a link to a coloab notebook in the jupyter notebook that you can use.** | ||
| + | |||
| + | |||
| + | |||
| + | The code is slightly different between the notebook linked above, and the colab link provided by Harvard. Both will work, you may very likely need to mix and match pieces from each to get a working implementation. While this may feel slightly frustrating, it is good practice for deep learning research strategies. | ||
| + | |||
| + | If you are experiencing difficulties, you may need to install specific versions of the necessary packages. One student contributed this: | ||
| + | |||
| + | <code python> | ||
| + | !pip install http://download.pytorch.org/whl/cu80/torch-0.3.0.post4-cp36-cp36m-linux_x86_64.whl numpy matplotlib spacy torchtext==0.2.3 seaborn | ||
| + | </code> | ||
| Often when implementing a novel deep learning method, you will start by using someone's implementation as a reference. This is an extremely valuable, and potentially time-saving skill, for producing workable solutions to many problems solved by deep learning methods. | Often when implementing a novel deep learning method, you will start by using someone's implementation as a reference. This is an extremely valuable, and potentially time-saving skill, for producing workable solutions to many problems solved by deep learning methods. | ||
| Line 50: | Line 63: | ||
| We will be translating from Spanish to English, which means Spanish is our source language and English is the target language. | We will be translating from Spanish to English, which means Spanish is our source language and English is the target language. | ||
| - | The annotated jupyter notebook has an example application translating from Dutch to English. This may be useful to copy, however, you will need to do some work to get the data into a useable form to fit into the model. This can be tricky, so make sure you understand the various torch functions being called. | + | The annotated jupyter notebook has an example application translating from Dutch to English. This may be useful to copy, however, you will need to do some work to get our data into a useable form to fit into the model. This can be tricky, so make sure you understand the various torch functions being called. |
| - | You may find the torchtext module to be useful, but feel free to load the data as you see fit. | + | You may find the torchtext and data modules useful, but feel free to load the data as you see fit. |
| Train the transformer network using their implementation and print out 5-10 translated sentences with your translation and the ground truth. Feel free to comment on why you think it did well or underperformed. | Train the transformer network using their implementation and print out 5-10 translated sentences with your translation and the ground truth. Feel free to comment on why you think it did well or underperformed. | ||
| You should expect to see reasonable results after 2-4 hours of training in colab. | You should expect to see reasonable results after 2-4 hours of training in colab. | ||
| + | |||
| + | [[http://mlexplained.com/2018/02/08/a-comprehensive-tutorial-to-torchtext/|Here is a good tutorial on torchtext.]] | ||
| ---- | ---- | ||
| - | **Part 3: Experiment with a different number of stacked layers in the encoder and decoder** | + | **Part 3: EXTRA CREDIT: Experiment with a different number of stacked layers in the encoder and decoder** |
| Now it's time to put on your scientist hats. Try stacking a different number of layers (e.g., 1, 2, 4) for the encoder and decoder. This will require you to understand their implementation and be able to work with it reliably. | Now it's time to put on your scientist hats. Try stacking a different number of layers (e.g., 1, 2, 4) for the encoder and decoder. This will require you to understand their implementation and be able to work with it reliably. | ||
cs501r_f2018/lab7.1539619518.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
