Spring 2019: MA 59800 Mathematical Aspects of Neural Networks
Instructor: Greg Buzzard

Papers for weekly presentations

Theory
1. Understanding deep convolutional networks
2. Learning Functions: When Is Deep Better than Shallow? or Deep vs. shallow networks : An approximation theory perspective
3. Energy Propagation in Deep Convolutional Neural Networks (appendices not required for presentation) or A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
4. Approximation by Superpositions of a Sigmoidal Function or Multilayer feedforward networks are universal approximators
5. Why does deep and cheap learning work so well?
6. Visualizing the Loss Landscape of Neural Nets
7. Stronger generalization bounds for deep nets via a compression approach
8. Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems
   
   Regularization
9. Dropout: a simple way to prevent neural networks from overfitting
10. On the difficulty of training recurrent neural networks
    
    Architecture
11. Deep Residual Learning for Image Recognition
12. U-Net: Convolutional Networks for Biomedical Image Segmentation
    
    Adversarial networks and attacks
13. Improved Techniques for Training GANs
14. Stabilizing Adversarial Nets With Prediction Methods
15. Universal Adversarial Training
16. Explaining and harnessing adversarial examples
17. Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
18. Empirical study of the topology and geometry of deep networks
    
    Sequence learning
19. Attention Is All You Need
20. Convolutional Sequence to Sequence Learning
    
    Autoencoders
21. Representation Learning: A Review and New Perspectives (section 1-3 and 7 only for one presentation)
22. Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
23. Auto-Encoding Variational Bayes. This tutorial may be helpful.
    
    Reinforcement learning
24. Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. See also the NY Times article.
25. Learning to Optimize Neural Nets
26. An Actor-Critic Algorithm for Sequence Prediction
27. Designing Neural Network Architectures using Reinforcement Learning
28. Learning to Communicate with Deep Multi-Agent Reinforcement Learning
    
    Other papers
29. A topological insight into restricted Boltzmann machines
30. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
31. Emergence of Invariance and Disentanglement in Deep Representations
32. Sharp Minima Can Generalize For Deep Nets
33. Taskonomy: Disentangling Task Transfer Learning
34. Machine Learning: The High-Interest Credit Card of Technical Debt
35. Discriminative vs Informative Learning
36. How SGD Selects the Global Minima in Over-parameterized Learning: A Dynamical Stability Perspective