Spring 2019:
MA 59800 Mathematical Aspects of Neural Networks
Instructor: Greg Buzzard
Papers for weekly presentations
-
Theory
- Understanding deep convolutional networks
- Learning Functions: When Is Deep Better than Shallow? or Deep vs. shallow networks : An approximation theory perspective
- Energy Propagation in Deep Convolutional Neural Networks (appendices not required for presentation) or A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
- Approximation by Superpositions of a Sigmoidal Function or Multilayer feedforward networks are universal approximators
- Why does deep and cheap learning work so well?
- Visualizing the Loss Landscape of Neural Nets
- Stronger generalization bounds for deep nets via a compression approach
-
Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems
Regularization - Dropout: a simple way to prevent neural networks from overfitting
- On the difficulty of training recurrent neural networks
Architecture - Deep Residual Learning for Image Recognition
- U-Net: Convolutional Networks for Biomedical
Image Segmentation
Adversarial networks and attacks - Improved Techniques for Training GANs
- Stabilizing Adversarial Nets With Prediction Methods
- Universal Adversarial Training
- Explaining and harnessing adversarial examples
- Audio Adversarial Examples: Targeted Attacks on Speech-to-Text
- Empirical study of the topology and geometry of deep networks
Sequence learning - Attention Is All You Need
- Convolutional Sequence to Sequence Learning
Autoencoders - Representation Learning: A Review and New Perspectives (section 1-3 and 7 only for one presentation)
- Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion
- Auto-Encoding Variational Bayes. This tutorial may be helpful.
Reinforcement learning - Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm. See also the NY Times article.
- Learning to Optimize Neural Nets
- An Actor-Critic Algorithm for Sequence Prediction
- Designing Neural Network Architectures using Reinforcement Learning
- Learning to Communicate with Deep Multi-Agent Reinforcement Learning
Other papers - A topological insight into restricted Boltzmann machines
- GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
- Emergence of Invariance and Disentanglement in Deep Representations
- Sharp Minima Can Generalize For Deep Nets
- Taskonomy: Disentangling Task Transfer Learning
- Machine Learning: The High-Interest Credit Card of Technical Debt
- Discriminative vs Informative Learning
- How SGD Selects the Global Minima in Over-parameterized Learning: A Dynamical Stability Perspective