February 12, 2020: Seminar by Dr. Manuj Mukherjee, Telecom Paris, Paris, France

Distribution Generation Power of Neural Networks: A Separation Result
Manuj Mukherjee

Telecom Paris, Paris, France
Email: manuj.mukherjee@telecom-paris.fr, manujscientific@gmail.com
Web: https://sites.google.com/view/manuj-mukherjee/home

Date: Wednesday, February 12, 2020
Time: 15:00-16:00 HRS
Venue: Seminar Room, 
             Electronics and Communication Sciences Unit 
             9th Floor, S. N. Bose Bhavan (Library Building)


This paper revisits the classic question of whether deep nets have better approximating power than shallow nets, but in the context of distribution approximation. The paper establishes that feedforward neural networks with Lipschitz activation functions produce distributions with sub-Gaussian tails for a large class of input seeds, including the commonly used Gaussian and uniform seeds. The sub-Gaussian tail decay is explicitly determined by the parameters of the neural network. This is then used to establish separation results that emphasize the roles of depth, width, and seed dimension. For instance, there exists distributions which cannot be approximated by a network of depth $d$ with dimension $l$ seed, but that can be approximated by a depth $d+1$ network of similar width but with a seed of dimension $\log l$.

Bio: Manuj Mukherjee received the B.E. degree in Electronics and Telecommunication Engineering from Jadavpur University, Kolkata, India, in 2011 and the M.S. (Engineering) and Ph.D. degrees in Electrical Communication Engineering (ECE) from the Indian Institute of Science, Bangalore, India in 2017. From 2018-2020, he was a postdoctoral fellow at Telecom Paris, Paris, France. He will be joining as a postdoctoral fellow at the Bar Ilan University, Ramat Gan, Israel, in February 2020. He is a recipient of the Seshagiri Kaikini medal by the Indian Institute of Science, awarded for the best PhD thesis from the ECE department in the year 2017-18. His research interests include the broad umbrella of problems coming under information theory.

All are cordially invited.


Swagatam Das
Head (Acting)
Electronics and Communication Sciences Unit