BYU CS classes

This is an old revision of the document!

---

Applications

• BERT - https://arxiv.org/abs/1810.04805
• Machine Theory of Mind - http://arxiv.org/pdf/1802.07740v2.pdf
• Video-to-Video Synthesis
• Video Prediction via Selective Sampling
• Learning to decompose & disadvantage representations for video prediction

GANs / unsupervised

IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesis
Wasserstein GAN
Text adaptive GAN: Manipulating images with natural language

Network design

Attention is all you need
Neural Ordinary Differential Equations - https://arxiv.org/pdf/1806.07366.pdf
Reversible neural networks - https://arxiv.org/abs/1807.03039 - https://arxiv.org/abs/1605.08803

Foundations / Philosophy

Troubling trends in ML scholarship - https://arxiv.org/pdf/1807.03341
A Theory of Local Learning, the Learning Channel, and the Optimality of Backpropagation - https://arxiv.org/pdf/1506.06472
Why and When Can Deep -- but Not Shallow -- Networks Avoid the Curse of Dimensionality: a Review - https://arxiv.org/pdf/1611.00740

RL

Curiosity-driven exploration by self-supervised prediction
Diversity is all you need: Learning skills without a reward function - http://arxiv.org/pdf/1802.06070v6.pdf
World Models - https://arxiv.org/pdf/1803.10122v4.pdf

Graph networks

Graph Neural Networks: A Review of Methods and Applications - https://arxiv.org/abs/1812.08434
Relational inductive biases, deep learning, and graph networks - https://arxiv.org/pdf/1806.01261

Optimization / training

Averaging weights leads to wider optima and better generalization - http://arxiv.org/pdf/1803.05407v2.pdf
The loss surface of multilayer networks - https://arxiv.org/pdf/1412.0233
Visualizing The Loss Landscape of Neural Nets - https://arxiv.org/pdf/1712.09913v3.pdf
The Matrix Calculus You Need For Deep Learning - https://arxiv.org/pdf/1802.01528v3.pdf
Group Norm - https://arxiv.org/pdf/1803.08494v3.pdf
Kalman Normalization: Normalizing internal representations across network layers
MetaReg: towards Domain Generalization using meta-regularization
AutoAugment - https://arxiv.org/abs/1805.09501
A Disciplined Approach To Neural Network Hyper-Parameters: part 1 - http://arxiv.org/pdf/1803.09820v2.pdf
(Direct) Feedback alignment

Geometric deep learning

Geometric deep learning: going beyond Euclidean data- https://arxiv.org/pdf/1611.08097.pdf
Convolutional Neural Networks on Surfaces via Seamless Toric Covers
SchNet: A continuous-filter convolutional neural network for modeling quantum interactions
Deriving Neural Architectures from Sequence and Graph Kernels
CayleyNets: Graph convolutional neural networks with complex rational spectral filters
Deep Functional Maps: Structured Prediction for Dense Shape Correspondence
Geometric matrix completion with recurrent multi-graph neural networks
Neural Message Passing for Quantum Chemistry
Deep Learning on Lie Groups for Skeleton-based Action Recognition

Other

Bayesian neural networks?