http://liftothers.org/dokuwiki/ 2026-05-15T15:49:57+00:00 http://liftothers.org/dokuwiki/ http://liftothers.org/dokuwiki/lib/tpl/dokuwiki/images/favicon.ico text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:desc&rev=1625096525&do=diff CS501r, Fall 2018 - Deep Learning: Theory and Practice As big data and deep learning gain more prominence in both industry and academia, the time seems ripe for a class focused exclusively on the theory and practice of deep learning, both to understand why deep learning has had such a tremendous impact across so many disciplines, and also to spur research excellence in deep learning at BYU. text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:fp&rev=1625096525&do=diff Objective: To creatively apply knowledge gained through the course of the semester to a substantial learning problem of your own choosing. ---------- Deliverable: There are two deliverables for the final: * An excel spreadsheet (or CSV file) that shows the total amount of time you spent on your final, broken down by day text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab1&rev=1625096525&do=diff Objective: Get started with anaconda, python, ipython notebooks, and pandas. Begin producing simple visualizations of data and images. ---------- Deliverable: For this lab, you will submit an ipython notebook. This notebook will have two parts: text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab2&rev=1625096525&do=diff Objective: To gain experience with python, numpy, and linear classification. Oh, and to remember all of that linear algebra stuff. ;) ---------- Deliverable: You should turn in an iPython notebook that implements the perceptron algorithm on two different datasets: the Iris dataset, and the CIFAR-10 dataset. Because the perceptron is a binary classifier, we will preprocess the data and text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab3&rev=1625096525&do=diff Objective: To code a simple gradient descent based optimizer, to solidify understanding of score and loss functions, to improve ability to vectorize code, and to learn about numerical differentiation. ---------- Deliverable: You should turn in an iPython notebook that implements vanilla gradient descent on a 10-way CIFAR classifier. You should load the CIFAR-10 dataset, create a linear score function, and use the log soft-max loss function. text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab4&rev=1625096525&do=diff Objective: To gaze in awe at the power and speed of automatic differentiation, and to wonder why it took us all so long to figure this out. ---------- Deliverable: Tidings of joy: this should be a super simple lab! For this lab, you will modify your code from lab3. You will swap out the numerical gradient, and you will swap in the true gradient which will be calculated via automatic differentiation. text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab5&rev=1625096526&do=diff Objective: To install and learn the basics of Tensorflow, and to become more proficient in the construction of computation graphs and image classification. ---------- Deliverable: For this lab, you will need to perform three steps: * You need to text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab5b&rev=1625096526&do=diff Objective: To explore deeper networks, to leverage convolutions, and to explore Tensorboard. ---------- Deliverable: For this lab, you will need to perform three steps: * You need to implement the Deep MNIST for experts tutorial * You need to modify the tutorial code to deliver visualizations via Tensorboard. text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab6&rev=1625096526&do=diff Objective: To read current papers on DNN research and translate them into working models. To experiment with DNN-style regularization methods, including Dropout, Dropconnect, and L1 weight regularization. ---------- Deliverable: For this lab, you will need to implement three different regularization methods from the literature, and explore the parameters of each. text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab6v2&rev=1625096525&do=diff BYU CS 501R - Deep Learning:Theory and Practice - Lab 6 Objective: To build a dense prediction model, to begin to read current papers in DNN research, to experiment with different DNN topologies, and to experiment with different regularization techniques. text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab7&rev=1625096526&do=diff WARNING THIS LAB SPEC IS UNDER DEVELOPMENT: Objective: To learn about deconvolutions, variable sharing, trainable variables, and generative adversarial models. ---------- Deliverable: For this lab, you will need to implement a generative adversarial network (GAN). Specifically, we will be using the technique outlined in the paper text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab9&rev=1625096526&do=diff Objective: To gain experience coding a DNN architecture and learning program end-to-end, and to gain experience with Siamese network and ResNets. ---------- Deliverable: For this lab, you will need to implement a simple face similarity detector. text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab10&rev=1625096525&do=diff Objective: To learn about recurrent neural networks, LSTMs, GRUs and Tensorflow's sequence-to-sequence capabilities. To be able to read the core Tensorflow codebase. ---------- Deliverable: For this lab, you will need to implement the `char-rnn` model of Karpathy. You will train it on a text corpus that you're interested in, and then show samples from the model. text/html 2021-06-30T23:42:05+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab13&rev=1625096525&do=diff Objective: To explore an alternative use of DNNs by implementing the style transfer algorithm. ---------- Deliverable: For this lab, you will need to implement the style transfer algorithm of Gatys et al. * You must extract statistics from the content and style images text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab14&rev=1625096526&do=diff Objective: Implement a neural machine translation system using PyTorch. ---------- Deliverable: For this lab you will implement an autoencoder/decoder neural machine translation (NMT) system to translate between English and Spanish. Because of the difficulties inherent in processing variable-length sentences, we will be using PyTorch instead of Tensorflow. text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:lab_notes&rev=1625096526&do=diff Throughout this class, we'll be using several python packages extensively. If you've never used them before, we strongly encourage you to play around with them and get comfortable. They are: * numpy - Python's package that provides high-performance matrix, vector and linear algebra operations text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:openlabtf&rev=1625096526&do=diff Tensorflow on open lab machines Here are some notes for getting Tensorflow installed and running on the open lab machines. First of all, you can always just use my anaconda installation, which already has Tensorflow 0.10 installed. It can be found at text/html 2021-06-30T23:42:06+00:00 http://liftothers.org/dokuwiki/doku.php?id=cs501r_f2016:tmp&rev=1625096526&do=diff Objective: To gain experience coding a DNN architecture and learning program end-to-end, and to gain experience with Siamese network and ResNets. ---------- Deliverable: For this lab, you will need to implement a simple face similarity detector.