Stuff to read:

Visualizing representations

Unreasonable effectiveness of RNNs

Deepmind paper on DNNs and Atari

Inceptionism

Everything on the Deepmind publications page

Neural networks and deep learning - online book