This is an old revision of the document!
Preliminary Syllabus and topics to be covered:
High-level implementation packages - pros and cons
Tensorflow, Theano, Caffe, Keras, Torch, Mocha
Case studies / existing networks and why they're interesting
AlexNet VGG GoogLeNet / Inception ZFNet
Training & initialization
Initialization strategies: Xavier, Gaussian, Identity, Sparse Optimization theory and algorithms Local minima; saddle points; plateaus SGD RPROP RMS prop Adagrad Adam Higher-order algorithms (LBFGS; Hessian-free; trust-region) Nesterov and momentum
Large-scale distributed learning
Parameter servers Asynchronous vs. synchronous architectures
Temporal networks and how to train them
Basic RNNs LSTMs Deep Memory Nets
Application areas
Deep reinforcement learning NN models of style vs. content (deepart.io) Imagenet classification The Neural Turing Machine Sentiment classification Word embeddings
Understanding and visualizing CNNs
tSNE embeddings deconvnets data gradients / inceptionism
Misc
Network compression Low bit-precision networks Sum-product networks Evolutionary approaches to topology discovery Spatial transformer networks Network-in-network Regions-with-CNN