Differences

This shows you the differences between two versions of the page.

--- cs501r_f2018:lab6 [2018/09/20 15:49]
carr
+++ cs501r_f2018:lab6 [2021/06/30 23:42]
@@ Line 1: / Line 1: @@
-====Objective:====
-Work with sequential data in Pytorch by building a Char-RNN for text generation
-----
-====Deliverable:====
-For this lab, you will submit an ipython notebook via learningsuite.
-There are many resources for character level recurrent neural networks. This [[http://karpathy.github.io/2015/05/21/rnn-effectiveness/|Blog Post]] will be helpful in understanding the potential, and getting basic understanding.
- This lab will have three parts:
-**Part 1:** Build RNN with built in methods, train on _textfile.txt_
-**Part 2:** Build your own LSTM
-**Part 3:** Build your own GRU
-**Part 4:** Generate awesome text
-This is an example output from lord of the rings, after only 20 minutes of training.
-[18m 46s (1800 90%) 1.2902]
-Who that ' said Gimli. 'Well
-was no much you gold. I do ride no one of grey of the few her
-command i
-%|███████████████████████████████████████████████████████████████████████████████████████████▏    | 1899/2000 [19:49<01:03,  1.60it/s][19m 50s (1900 95%) 1.3255]
-White so as these though I have
-the will of the Did and well's. You may indeed through his feet. They
-%|███████████████████████████████████████████████████████████████████████████████████████████████▉| 1999/2000 [20:58<00:00,  1.59it/s][20m 59s (2000 100%) 1.3048]
-Who now further here the learnest and
-south, looking slow you beastion, and that is all plainly day.
-----
-====Grading standards:====
-Your notebook will be graded on the following:
-  * 100% Build something amazing
-  * 20% Modified code to include a test/train split
-  * 20% Modified code to include a visualization of train/test losses
-  * 10% Tidy and legible figures, including labeled axes where appropriate
-----
-====Description:====
-**Not Updated **
-Throughout this class, we will be using pytorch to implement our deep neural networks.  Pytorch is a deep learning framework that handles the low-level details of GPU integration and automatic differentiation.
-The goal of this lab is to help you become familiar with pytorch.  The three parts of the lab are outlined above.
-For part 1, you should watch this video, and type in the code as it is explained to you.
-The video is here [[https://youtu.be/0P-YctShbwc|lab 2 tutorial video]]
-A more detailed outline of Part 1 is below.
-For part 2, you must add a validation (or testing) loop using the FashionMNIST dataset with train=False
-For part 3, you must plot the loss values and demonstrate overfitting.
-The easiest way to do this is to limit the size of your training dataset so that it only returns a single batch (ie len(dataloader) == batch_size, and train for multiple epochs. In the example graph above, I set my batch size to 42, and augmented my dataloader to produce only 42 unique items by overwriting the __len__ function to return 42. In my training loop, I performed a validation every epoch which basically corresponded to a validation every step.
-In practice, you will normally compute your validation loss every n steps, rather than at the end of every epoch. This is because some epochs can take hours, or even days and you don’t often want to wait that long to see your results.
-Testing your algorithm by using a single batch and training until overfitting is a great way of making sure that your model and optimizer are working the way they should!
-----
-====Part 1 detailed outline:====
-**Step 1.** Get a colab notebook up and running with GPUs enabled.
-**Step 2.** Install pytorch and torchvision
-<code python>
-!pip3 install torch
-!pip3 install torchvision
-!pip3 install tqdm
-</code>
-**Step 3.** Import pytorch and other important classes
-<code python>
-import torch
-import torch.nn as nn
-import torch.nn.functional as F
-import torch.optim as optim
-from torch.utils.data import Dataset, DataLoader
-import numpy as np
-import matplotlib.pyplot as plt
-from torchvision import transforms, utils, datasets
-from tqdm import tqdm
-assert torch.cuda.is_available() # You need to request a GPU from Runtime > Change Runtime Type
-</code>
-**Step 4.** Construct
-- a model class that inherits from “nn.Module”
-  * Check out [[https://pytorch.org/docs/stable/nn.html#torch.nn.Module]]
-  * Your model can contain any submodules you wish -- nn.Linear is a good, easy, starting point
-- a dataset class that inherits from “Dataset” and produces samples from [[https://pytorch.org/docs/stable/torchvision/datasets.html#fashion-mnist]]
-  * You may be tempted to use this dataset directly (as it already inherits from Dataset) but we want you to learn how a dataset is constructed. Your class should be pretty simple and output items from FashionMNIST
-**Step 5.** Create instances of the following objects:
-  * SGD optimizer Check out [[https://pytorch.org/docs/stable/optim.html#torch.optim.SGD]]
-  * your model
-  * the DataLoader class using your dataset
-  * MSE loss function [[https://pytorch.org/docs/stable/nn.html#torch.nn.MSELoss]]
-**Step 6.** Loop over your training dataloader, inside of this loop you should
-  * zero out your gradients
-  * compute the loss between your model and the true value
-  * take a step on the optimizer

BYU CS classes

User Tools

Site Tools

Differences

Page Tools