Course Contents
Below is a tentative list of topics:
- Sampling from Probability Distributions
- Foundations: Maximum Likelihood Estimation, Bias-Variance Tradeoff
- Classification
- Latent Variable Models and Expectation Maximisation
- Score-based Models
- Diffusion Models
- Flow-based Models
- Influence Functions
Textbooks:
- Pattern Recognition and Machine Learning, Bishop
- The Elements of Statistical Learning, Hastie, Tibshirani and Friedman
- Deep Learning, Bishop
Logistics
Time and Location:
- Slot: C
- Class Timings: Tue/Wed/Fri 8-9 am
- Venue: LH 521
Communication:
We will use piazza as the forum for students to ask questions about both the course material as well as logistics.
Teaching Team:
- Instructor: Raunak P. Bhattacharyya, Assistant Professor, Yardi School of Artificial Intelligence
- Graduate student instructors: Mahesh Keswani, Mohit Chhabra, Soubhagya Khan, Sai Krishna Podem, Parv Pratap Singh
Announcements
- [18 Aug] Assignment 1 released
- [5 Sep] Assignment 2 released
- [25 Sep] Assignment 3 released
- [19 Oct] Assignment 4 released
Schedule
The weekly schedule will be updated with lecture slides and pointers to additional references as we progress through the course.
| Monday | Tuesday | Wednesday | Thursday | Friday | Saturday | Sunday | |
|---|---|---|---|---|---|---|---|
| Week 1 | July 20 | July 21 | July 22 | July 23 | July 24 | July 25 | July 26 |
| Course overview |
Course overview Slides |
||||||
| Week 2 | July 27 | July 28 | July 29 | July 30 | July 31 | Aug 1 | Aug 2 |
|
Sampling PRML Chapter 11 |
Inverse Transform Sampling Notebook Notebook: Gaussian inverse CDF |
Sampling via change of variables |
Rejection Sampling Notebook |
||||
| Week 3 | Aug 3 | Aug 4 | Aug 5 | Aug 6 | Aug 7 | Aug 8 | Aug 9 |
|
Statistical Principles PRML Chapter 1 & 3 |
Polynomial curve fitting, Max likelihood estimation |
Logistic Regression, Decision Theory: Regression & Classification CS229 Notes Chapter 2 |
Decision Theory (contd.), Why MLE |
||||
| Week 4 | Aug 10 | Aug 11 | Aug 12 | Aug 13 | Aug 14 | Aug 15 | Aug 16 |
|
Model Selection PRML Chapter 1 |
Monte Carlo Estimator of log likelihood, MAP estimate |
Degree of polynomial fit |
Bias-Variance Decomposition Slides CS229 Notes Chapter 8 |
||||
| Week 5 | Aug 17 | Aug 18 | Aug 19 | Aug 20 | Aug 21 | Aug 22 | Aug 23 |
|
Regularisation, Generative Classification ESL Chapter 7 |
Variance of OLS and regularised least squares |
Bias OLS vs regularised, Regularisation visualisations, Cross Validation Slides |
Gaussian Discriminant Analysis |
||||
| Week 6 | Aug 24 | Aug 25 | Aug 26 | Aug 27 | Aug 28 | Aug 29 | Aug 30 |
|
Gaussian Discriminant Analysis PRML Section 4.2 CS229 Chapter 4 |
QDA & LDA with isotropic Gaussians, Gaussian density estimation Slides |
Multivariate Gaussians Slides |
|||||
| Week 7 | Aug 31 | Sep 1 | Sep 2 | Sep 3 | Sep 4 | Sep 5 | Sep 6 |
|
GDA, Naive Bayes CS229 Chapter 4.2 |
GDA with anisotropic Gaussians Slides |
QDA vs LDA vs Logistic Regression, Bayesian Networks Slides |
Naive Bayes Classifier Slides |
||||
| Week 8 | Sep 7 | Sep 8 | Sep 9 | Sep 10 | Sep 11 | Sep 12 | Sep 13 |
|
Mixture of Gaussians PRML Sections 2.3.9, 9.2 |
Mixture of Gaussians with complete data Slides |
Mixture of Gaussians with incomplete data | |||||
| Week 9 | Sep 14 | Sep 15 | Sep 16 | Sep 17 | Sep 18 | Sep 19 | Sep 20 |
| Midsem | |||||||
| Week 10 | Sep 21 | Sep 22 | Sep 23 | Sep 24 | Sep 25 | Sep 26 | Sep 27 |
|
Continous Latent Variables, Probabilistic PCA PRML Chapter 12 DL Chapter 16 |
Mixture of Bernoullis, Latent Variables, Manifolds Slides Goodfellow DL Section 5.11 |
Continous latent variables, Probabilistic PCA | |||||
| Week 11 | Sep 28 | Sep 29 | Sep 30 | Oct 1 | Oct 2 | Oct 3 | Oct 4 |
| Week 12 | Oct 5 | Oct 6 | Oct 7 | Oct 8 | Oct 9 | Oct 10 | Oct 11 |
|
Expectation Maximisation PRML Chapter 9 |
Minorisation Maximisation PRML 9.4 |
E-step and M-step, Why EM works |
EM for PPCA DL 16.3 |
||||
| Week 13 | Oct 12 | Oct 13 | Oct 14 | Oct 15 | Oct 16 | Oct 17 | Oct 18 |
|
Applying EM, Towards Nonlinear Latent Variable Models |
Normalisation of posterior, Why the name PPCA DL 16.3.2 |
MLE for MVN with missing data using EM Murphy advanced 6.5.4 |
MVN with missing data (Part 2), Nonlinear manifolds Slides DL 16.4 |
||||
| Week 14 | Oct 19 | Oct 20 | Oct 21 | Oct 22 | Oct 23 | Oct 24 | Oct 25 |
|
Nonlinearity, Neural Nets |
Nonlinear PPCA, Why nonlinearity causes problems for EM DL 16.4 |
Neural nets: Towards learned basis functions |
|||||
| Week 15 | Oct 26 | Oct 27 | Oct 28 | Oct 29 | Oct 30 | Oct 31 | Nov 1 |
|
Generalisation in Neural Nets, Notes |
Neural tangent kernels, Prof. Bhaskar Mukhoty, ScAI |
Neural tangent kernels (Part 2), Prof. Bhaskar Mukhoty, ScAI | |||||
| Week 16 | Nov 2 | Nov 3 | Nov 4 | Nov 5 | Nov 6 | Nov 7 | Nov 8 |
|
Variational Autoencoder, Modeling sequences |
VAE DL 19.2 HMM intro SLP |
Lecture |
Lecture |
||||
| Week 17 | Nov 9 | Nov 10 | Nov 11 | Nov 12 | Nov 13 | Nov 14 | Nov 15 |
| Topic |
Lecture |
Lecture |
Lecture |
||||
| Week 18 | Nov 16 | Nov 17 | Nov 18 | Nov 19 | Nov 20 | Nov 21 | Nov 22 |
| Topic |
Lecture |
Endsem |
Grading Policy
Grading policy (tentative, upto ±10% on each component):
- Assignments: 40%
- Exams: 60%
- Audit policy: At least 50% in each and every exam
Attendance policy:
- If a student’s attendance is less than 75%, the student will be awarded one grade less than the actual grade that he/she has earned.
- A student cannot get NP for an audit course if his/her attendance is less than 75%.