Instructor: Sumeet Agarwal
1 credit (1-0-0)
Pre-requisite: Any course covering neural networks and backpropagation
I Semester 2018–19
F 18:00–19:20, LH 623
| Date | Topic(s) | Reading(s) |
| Wed 25th July (1 hr) | Introduction | Wolchover 2017 (Quanta) |
| Tue 31st July (1.5 hrs) | Deep learning overview | LeCun et al. 2015 (Nature) |
| Fri 10th August (1.5 hrs) | Notions of 'theory' in deep learning; information theory fundamentals | |
| Fri 31st August (1.5 hrs) | The information bottleneck method I (rate distortion theory) | Tishby et al. 2000 (arXiv) |
| Fri 7th September (1.5 hrs) | The information bottleneck method II (information bottlenecks) | Tishby et al. 2000 (arXiv) |
| Fri 14th September (1.5 hrs) | Deep learning and information bottlenecks I | Tishby & Zaslavsky 2015 (arXiv) |
| Fri 28th September (1.5 hrs) | Deep learning and information bottlenecks II; Quiz | Tishby & Zaslavsky 2015 (arXiv) |
| Mon 1st October (1.5 hrs) | Deep learning and information bottlenecks III | Tishby & Zaslavsky 2015 (arXiv) |
| Fri 26th October (1.5 hrs) | Information-plane analysis of deep learning models | Schwartz-Ziv & Tishby 2017 (arXiv) |
| Fri 2nd November (1.5 hrs) | Critique of the information bottleneck theory of deep learning | Saxe et al. 2018 (OpenReview) |