On the learning dynamics of neural networks

Neural networks have shown some unbelievable success over the years. A neural network is believed to be a universal function approximator meaning that even a single node of a network can learn any arbitrary function if left for training for a sufficient amount of time.

But these things need better explanation -

� Why can neural networks even achieve generalization? Or is it just memorization?

� How neural nets model uncertainty? Can these things be explained with information theory? Do mutual information between the subsequent layers influence this?

Throughout the session, I will be discussing several points to address the above questions from current research studies. Hopefully, this would give the audience a better perspective of the abstractions neural networks are known to model.

Author(s): Sayak Paul

Abstract | PDF

Share This Article