cs501r_f2018:lab2
Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
|
cs501r_f2018:lab2 [2018/09/10 16:58] wingated created |
cs501r_f2018:lab2 [2021/06/30 23:42] (current) |
||
|---|---|---|---|
| Line 6: | Line 6: | ||
| ====Deliverable:==== | ====Deliverable:==== | ||
| - | For this lab, you will submit an ipython notebook via learningsuite. This notebook will have two parts: | + | For this lab, you will submit an ipython notebook via learningsuite. This lab will be mostly boilerplate code, but you will be required to implement a few extras. |
| - | **Part 1:** Your notebook should generate a random image. We will run this | + | **NOTE: you almost certainly will not understand most of what's going on in this lab! That's ok - the point is just to get you going with pytorch. We'll be working on developing a deeper understanding of every part of this code over the course of the next two weeks.** |
| - | notebook 5 times; it should generate 5 different, moderately complex | + | |
| - | images. Each image should be 512 x 288. Have fun with it! | + | |
| - | The resulting image could, for example, look like this: | + | A major goal of this lab is to help you become conversant in working through pytorch tutorials and documentation. So, you should feel free to google whatever you want and need! |
| - | {{:cs401r_w2016:lab1.png?nolink|}} | + | https://youtu.be/0P-YctShbwc |
| - | **Part 2:** You must play with the Tensorflow playground neural network, and figure out how to create a classifier that successfully classifies the "spiral" dataset. | + | This notebook will have three parts: |
| - | [[http://playground.tensorflow.org/|Tensorflow playground]] | + | **Part 1:** Your notebook should contain the boilerplate code. See below. |
| - | [[http://colab.research.google.com/|Google colab]] | + | **Part 2:** Your notebook should extend the boilerplate code by adding a testing loop. |
| - | [[https://colab.research.google.com/drive/1TzaPS3jvRadN-URLbQ9nD1ZNoZktfNRy|A colab notebook teaching you how to use colab notebooks]] | + | **Part 3:** Your notebook should extend the boilerplate code by adding a visualization of test/training performance over time. |
| - | [[https://sites.google.com/site/artml2018/tutorials|Various colab tutorials]] | + | The resulting image could, for example, look like this: |
| + | |||
| + | {{:cs501r_f2018:lab2.png |}} | ||
| + | |||
| + | See the assigned readings for pointers to documentation on pytorch. | ||
| ---- | ---- | ||
| Line 33: | Line 35: | ||
| * 50% Successfully followed lab video and typed in code | * 50% Successfully followed lab video and typed in code | ||
| * 20% Modified code to include a test/train split | * 20% Modified code to include a test/train split | ||
| - | * 20% Modified code to include a visualization of train/test accuracies | + | * 20% Modified code to include a visualization of train/test losses |
| * 10% Tidy and legible figures, including labeled axes where appropriate | * 10% Tidy and legible figures, including labeled axes where appropriate | ||
| Line 39: | Line 41: | ||
| ====Description:==== | ====Description:==== | ||
| - | Throughout this class, we will be using Google's colab environment to develop and test our deep neural networks. This consists of ipython | + | Throughout this class, we will be using pytorch to implement our deep neural networks. Pytorch is a deep learning framework that handles the low-level details of GPU integration and automatic differentiation. |
| - | notebooks and a standardized python distribution. For this lab, you | + | |
| - | must create a google account (or use one you already have), start up colab, and write a simple python program in it. | + | |
| - | As described above, the notebook should do two things: | + | The goal of this lab is to help you become familiar with pytorch. The three parts of the lab are outlined above. |
| - | 1) generate simple random images, and 2) display an image that you generate using the Tensorflow playground. | + | |
| - | For part 1, you can generate any sort of random image that you want -- consider | + | For part 1, you should watch this video, and type in the code as it is explained to you. |
| - | random lines, random curves, random text, etc. Each time the program | + | |
| - | is run, it should generate a different random image. Your image | + | |
| - | should have at least 50 random elements (they can all be the same | + | |
| - | type, such as random lines, and can be created in a loop). We won't | + | |
| - | count the number of elements; this is just to encourage you to create | + | |
| - | random images with moderate complexity (ie, you can't just generate randomly colored pixels, as in the example below). | + | |
| - | For part 2, you should visit the Tensorflow playground (see link above), and play with different settings. Most of it will be unfamiliar, but don't worry -- you can't break it! | + | The video is here [[https://youtu.be/0P-YctShbwc|lab 2 tutorial video]] |
| - | Once you have a working classifier, take a screenshot. Then use your colab notebook to display that image in-line. | + | A more detailed outline of Part 1 is below. |
| - | ---- | + | For part 2, you must add a validation (or testing) loop using the FashionMNIST dataset with train=False |
| - | ====Starter code:==== | + | |
| - | Here's some starter code to help you generate an image. The ''nbimage'' function will display the image inline in the notebook: | ||
| - | <code python> | + | For part 3, you must plot the loss values and demonstrate overfitting. |
| - | import IPython.display | + | |
| - | import PIL.Image | + | |
| - | import numpy as np | + | |
| - | # A simple function to display an image in an ipython notebook | + | The easiest way to do this is to limit the size of your training dataset so that it only returns a single batch (ie len(dataloader) == batch_size, and train for multiple epochs. In the example graph above, I set my batch size to 42, and augmented my dataloader to produce only 42 unique items by overwriting the __len__ function to return 42. In my training loop, I performed a validation every epoch which basically corresponded to a validation every step. |
| - | def nbimage( data ): | + | |
| - | IPython.display.display(PIL.Image.fromarray(data)) | + | |
| + | In practice, you will normally compute your validation loss every n steps, rather than at the end of every epoch. This is because some epochs can take hours, or even days and you don’t often want to wait that long to see your results. | ||
| - | # create an image consisting of random colors | + | Testing your algorithm by using a single batch and training until overfitting is a great way of making sure that your model and optimizer are working the way they should! |
| - | data = np.random.rand(512,512,3) # a 512x512 image, with 3 color channels (R,G,B) | ||
| - | # by default, rand creates floating point numbers between [0,1]. We need to convert that to 8-bit bytes between [0,255] | ||
| - | data = (255*data).astype('uint8') | ||
| - | # display it! | + | ---- |
| - | nbimage( data ) | + | ====Part 1 detailed outline:==== |
| + | **Step 1.** Get a colab notebook up and running with GPUs enabled. | ||
| + | **Step 2.** Install pytorch and torchvision | ||
| + | <code python> | ||
| + | !pip3 install torch | ||
| + | !pip3 install torchvision | ||
| + | !pip3 install tqdm | ||
| </code> | </code> | ||
| - | ---- | + | **Step 3.** Import pytorch and other important classes |
| - | ====Hints:==== | + | |
| - | + | ||
| - | The following python functions might be helpful: | + | |
| <code python> | <code python> | ||
| + | import torch | ||
| + | import torch.nn as nn | ||
| + | import torch.nn.functional as F | ||
| + | import torch.optim as optim | ||
| + | from torch.utils.data import Dataset, DataLoader | ||
| + | import numpy as np | ||
| import matplotlib.pyplot as plt | import matplotlib.pyplot as plt | ||
| + | from torchvision import transforms, utils, datasets | ||
| + | from tqdm import tqdm | ||
| - | plt.legend | + | assert torch.cuda.is_available() # You need to request a GPU from Runtime > Change Runtime Type |
| - | plt.xlabel | + | </code> |
| - | plt.ylabel | + | |
| - | plt.tight_layout | + | **Step 4.** Construct |
| - | </code> | + | - a model class that inherits from “nn.Module” |
| + | * Check out [[https://pytorch.org/docs/stable/nn.html#torch.nn.Module]] | ||
| + | * Your model can contain any submodules you wish -- nn.Linear is a good, easy, starting point | ||
| + | - a dataset class that inherits from “Dataset” and produces samples from [[https://pytorch.org/docs/stable/torchvision/datasets.html#fashion-mnist]] | ||
| + | * You may be tempted to use this dataset directly (as it already inherits from Dataset) but we want you to learn how a dataset is constructed. Your class should be pretty simple and output items from FashionMNIST | ||
| + | |||
| + | **Step 5.** Create instances of the following objects: | ||
| + | |||
| + | * SGD optimizer Check out [[https://pytorch.org/docs/stable/optim.html#torch.optim.SGD]] | ||
| + | * your model | ||
| + | * the DataLoader class using your dataset | ||
| + | * MSE loss function [[https://pytorch.org/docs/stable/nn.html#torch.nn.MSELoss]] | ||
| + | |||
| + | **Step 6.** Loop over your training dataloader, inside of this loop you should | ||
| + | |||
| + | * zero out your gradients | ||
| + | * compute the loss between your model and the true value | ||
| + | * take a step on the optimizer | ||
cs501r_f2018/lab2.1536598682.txt.gz · Last modified: 2021/06/30 23:40 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
