cs501r_f2016
Differences
This shows you the differences between two versions of the page.
|
cs501r_f2016 [2016/03/31 23:06] admin |
cs501r_f2016 [2021/06/30 23:42] |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | Preliminary Syllabus and topics to be covered: | ||
| - | |||
| - | - Basics of DNNs | ||
| - | - Convolution layers | ||
| - | - Maxpooling layers | ||
| - | - Relu units | ||
| - | - Softmax units | ||
| - | - Local response normalization / contrast normalization | ||
| - | - Regularization strategies | ||
| - | - Dropout | ||
| - | - Dropconnect | ||
| - | - Batch normalization | ||
| - | - Adversarial networks | ||
| - | - Data augmentation | ||
| - | |||
| - | High-level implementation packages - pros and cons | ||
| - | Tensorflow, Theano, Caffe, Keras, Torch, Mocha | ||
| - | |||
| - | Case studies / existing networks and why they're interesting | ||
| - | AlexNet | ||
| - | VGG | ||
| - | GoogLeNet / Inception | ||
| - | ZFNet | ||
| - | |||
| - | Training & initialization | ||
| - | Initialization strategies: Xavier, Gaussian, Identity, Sparse | ||
| - | Optimization theory and algorithms | ||
| - | Local minima; saddle points; plateaus | ||
| - | SGD | ||
| - | RPROP | ||
| - | RMS prop | ||
| - | Adagrad | ||
| - | Adam | ||
| - | Higher-order algorithms (LBFGS; Hessian-free; trust-region) | ||
| - | Nesterov and momentum | ||
| - | |||
| - | Large-scale distributed learning | ||
| - | Parameter servers | ||
| - | Asynchronous vs. synchronous architectures | ||
| - | |||
| - | Temporal networks and how to train them | ||
| - | Basic RNNs | ||
| - | LSTMs | ||
| - | Deep Memory Nets | ||
| - | |||
| - | Application areas | ||
| - | Deep reinforcement learning | ||
| - | NN models of style vs. content (deepart.io) | ||
| - | Imagenet classification | ||
| - | The Neural Turing Machine | ||
| - | Sentiment classification | ||
| - | Word embeddings | ||
| - | |||
| - | Understanding and visualizing CNNs | ||
| - | tSNE embeddings | ||
| - | deconvnets | ||
| - | data gradients / inceptionism | ||
| - | |||
| - | Misc | ||
| - | Network compression | ||
| - | Low bit-precision networks | ||
| - | Sum-product networks | ||
| - | Evolutionary approaches to topology discovery | ||
| - | Spatial transformer networks | ||
| - | Network-in-network | ||
| - | Regions-with-CNN | ||
cs501r_f2016.txt ยท Last modified: 2021/06/30 23:42 (external edit)
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
