This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
cs501r_f2018:lab6 [2018/10/08 19:07] carr |
cs501r_f2018:lab6 [2021/06/30 23:42] (current) |
||
---|---|---|---|
Line 10: | Line 10: | ||
You will train it on a text corpus that you're interested in, and then | You will train it on a text corpus that you're interested in, and then | ||
show samples from the model. | show samples from the model. | ||
+ | |||
+ | This lab is slightly different than previous labs in that we give you a large portion of the code, and you will just be filling in pieces of classes and functions. If you get suck, please get help from the TA's or your classmates. | ||
You should turn in your jupyter notebook | You should turn in your jupyter notebook | ||
Line 74: | Line 76: | ||
[[http://colah.github.io/posts/2015-08-Understanding-LSTMs/|Understanding LSTM Networks]] | [[http://colah.github.io/posts/2015-08-Understanding-LSTMs/|Understanding LSTM Networks]] | ||
+ | |||
+ | <code bash> | ||
+ | ! wget -O ./text_files.tar.gz 'https://piazza.com/redirect/s3?bucket=uploads&prefix=attach%2Fjlifkda6h0x5bk%2Fhzosotq4zil49m%2Fjn13x09arfeb%2Ftext_files.tar.gz' | ||
+ | ! tar -xzf text_files.tar.gz | ||
+ | ! pip install unidecode | ||
+ | ! pip install torch | ||
+ | </code> | ||
+ | |||
+ | |||
<code python> | <code python> | ||
Line 103: | Line 114: | ||
<code python> | <code python> | ||
+ | import torch | ||
+ | from torch.autograd import Variable | ||
# Turn string into list of longs | # Turn string into list of longs | ||
def char_tensor(string): | def char_tensor(string): | ||
Line 122: | Line 135: | ||
<code python> | <code python> | ||
+ | import time | ||
n_epochs = 2000 | n_epochs = 2000 | ||
print_every = 100 | print_every = 100 | ||
Line 138: | Line 152: | ||
for epoch in range(1, n_epochs + 1): | for epoch in range(1, n_epochs + 1): | ||
- | loss = train(*random_training_set()) | + | loss_ = train(*random_training_set()) |
- | loss_avg += loss | + | loss_avg += loss_ |
if epoch % print_every == 0: | if epoch % print_every == 0: | ||
- | print('[%s (%d %d%%) %.4f]' % (time_since(start), epoch, epoch / n_epochs * 100, loss)) | + | print('[%s (%d %d%%) %.4f]' % (time.time() - start, epoch, epoch / n_epochs * 100, loss_)) |
print(evaluate('Wh', 100), '\n') | print(evaluate('Wh', 100), '\n') | ||
Line 177: | Line 191: | ||
# decode output | # decode output | ||
| | ||
- | def forward(self, input, hidden): | + | def forward(self, input_char, hidden): |
# by reviewing the documentation, construct a forward function that properly uses the output | # by reviewing the documentation, construct a forward function that properly uses the output | ||
# of the GRU | # of the GRU | ||
Line 198: | Line 212: | ||
# your code here | # your code here | ||
## / | ## / | ||
+ | loss = 0 | ||
for c in range(chunk_len): | for c in range(chunk_len): | ||
output, hidden = # run the forward pass of your rnn with proper input | output, hidden = # run the forward pass of your rnn with proper input | ||
- | loss += criterion(output, target[c].view(1)) | + | loss += criterion(output, target[c].unsqueeze(0)) |
## calculate backwards loss and step the optimizer (globally) | ## calculate backwards loss and step the optimizer (globally) | ||
Line 216: | Line 230: | ||
<code python> | <code python> | ||
def evaluate(prime_str='A', predict_len=100, temperature=0.8): | def evaluate(prime_str='A', predict_len=100, temperature=0.8): | ||
- | ## initialize hidden variable, get tensor of input primer and initialize other useful variables | + | ## initialize hidden variable, initialize other useful variables |
# your code here | # your code here | ||
## / | ## / | ||
+ | | ||
+ | prime_input = char_tensor(prime_str) | ||
# Use priming string to "build up" hidden state | # Use priming string to "build up" hidden state |