This shows you the differences between two versions of the page.
cs501r_f2016 [2016/03/31 23:09] admin |
cs501r_f2016 [2021/06/30 23:42] |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | Preliminary Syllabus and topics to be covered: | ||
- | |||
- | - Basics of DNNs | ||
- | - Convolution layers | ||
- | - Maxpooling layers | ||
- | - Relu units | ||
- | - Softmax units | ||
- | - Local response normalization / contrast normalization | ||
- | - Regularization strategies | ||
- | - Dropout | ||
- | - Dropconnect | ||
- | - Batch normalization | ||
- | - Adversarial networks | ||
- | - Data augmentation | ||
- | - High-level implementation packages - pros and cons | ||
- | - Tensorflow, Theano, Caffe, Keras, Torch, Mocha | ||
- | - Case studies / existing networks and why they're interesting | ||
- | - AlexNet | ||
- | - VGG | ||
- | - GoogLeNet / Inception | ||
- | - ZFNet | ||
- | - Training & initialization | ||
- | - Initialization strategies: Xavier, Gaussian, Identity, Sparse | ||
- | - Optimization theory and algorithms | ||
- | - Local minima; saddle points; plateaus | ||
- | - SGD | ||
- | - RPROP | ||
- | - RMS prop | ||
- | - Adagrad | ||
- | - Adam | ||
- | - Higher-order algorithms (LBFGS; Hessian-free; trust-region) | ||
- | - Nesterov and momentum | ||
- | - Large-scale distributed learning | ||
- | - Parameter servers | ||
- | - Asynchronous vs. synchronous architectures | ||
- | - Temporal networks and how to train them | ||
- | - Basic RNNs and Backprop-through-time | ||
- | - LSTMs | ||
- | - Deep Memory Nets | ||
- | - Application areas | ||
- | - Deep reinforcement learning | ||
- | - NN models of style vs. content (deepart.io) | ||
- | - Imagenet classification | ||
- | - The Neural Turing Machine | ||
- | - Sentiment classification | ||
- | - Word embeddings | ||
- | - Understanding and visualizing CNNs | ||
- | - tSNE embeddings | ||
- | - deconvnets | ||
- | - data gradients / inceptionism | ||
- | - Misc | ||
- | - Network compression | ||
- | - Low bit-precision networks | ||
- | - Sum-product networks | ||
- | - Evolutionary approaches to topology discovery | ||
- | - Spatial transformer networks | ||
- | - Network-in-network | ||
- | - Regions-with-CNN | ||